2024-04-23T19:00:35.001 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T19:00:35.002 DEBUG:teuthology.run:Teuthology command: teuthology --name suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi --description rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} --verbose --owner scheduled_suriarte@teuthology --archive /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982 -- /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/orig.config.yaml 2024-04-23T19:00:35.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T19:00:35.120 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982 branch: rgw-update-boost-redis description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} email: null first_in_suite: false job_id: '7669982' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rgw-update-boost-redis ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: 5bf3b3a9b161d70b9aa7806055bdae202e03111c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 5bf3b3a9b161d70b9aa7806055bdae202e03111c rgw: ec-data-pool: true frontend: beast s3tests: slow_backend: true selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rgw-update-boost-redis sha1: 5bf3b3a9b161d70b9aa7806055bdae202e03111c owner: scheduled_suriarte@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 sha1: 5bf3b3a9b161d70b9aa7806055bdae202e03111c sleep_before_teardown: 0 suite: rgw suite_branch: rgw-update-boost-redis suite_path: /home/teuthworker/src/github.com_samarahu_ceph_5bf3b3a9b161d70b9aa7806055bdae202e03111c/qa suite_relpath: qa suite_repo: https://github.com/samarahu/ceph.git suite_sha1: 5bf3b3a9b161d70b9aa7806055bdae202e03111c targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM5lFx6tyKbqLCCgBvx+a5yQJqhW6016rUZ7u/MxHiLyg08MFSLxgKNABTMQppRdYvMCQ3nCA9rXfxmUtkW9BVM= smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHI5yAFXtJ66L36Lk3+SgnZX0DQj4WS1jlVZP19Redn2QimOalffOT6U6/o6+b9Zv36tqndX4zx3eXzrI0n2Y/k= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_15:04:03 tube: smithi user: suriarte verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T19:00:35.120 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_samarahu_ceph_5bf3b3a9b161d70b9aa7806055bdae202e03111c/qa; will attempt to use it 2024-04-23T19:00:35.121 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_samarahu_ceph_5bf3b3a9b161d70b9aa7806055bdae202e03111c/qa/tasks 2024-04-23T19:00:35.121 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T19:00:35.122 INFO:teuthology.task.internal:Checking packages... 2024-04-23T19:00:35.148 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5bf3b3a9b161d70b9aa7806055bdae202e03111c' 2024-04-23T19:00:35.148 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T19:00:35.149 INFO:teuthology.packaging:ref: None 2024-04-23T19:00:35.149 INFO:teuthology.packaging:tag: None 2024-04-23T19:00:35.149 INFO:teuthology.packaging:branch: rgw-update-boost-redis 2024-04-23T19:00:35.149 INFO:teuthology.packaging:sha1: 5bf3b3a9b161d70b9aa7806055bdae202e03111c 2024-04-23T19:00:35.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rgw-update-boost-redis 2024-04-23T19:00:35.375 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3101.g5bf3b3a9 2024-04-23T19:00:35.377 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T19:00:35.383 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T19:00:35.383 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T19:00:35.424 INFO:teuthology.task.internal:Saving configuration 2024-04-23T19:00:35.438 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T19:00:35.459 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T19:00:35.485 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:55:27.185718', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM5lFx6tyKbqLCCgBvx+a5yQJqhW6016rUZ7u/MxHiLyg08MFSLxgKNABTMQppRdYvMCQ3nCA9rXfxmUtkW9BVM='} 2024-04-23T19:00:35.508 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:55:27.187225', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHI5yAFXtJ66L36Lk3+SgnZX0DQj4WS1jlVZP19Redn2QimOalffOT6U6/o6+b9Zv36tqndX4zx3eXzrI0n2Y/k='} 2024-04-23T19:00:35.508 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T19:00:35.531 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-04-23T19:00:35.531 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-04-23T19:00:35.532 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T19:00:35.626 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faf51ed70d0>, signals=[15]) 2024-04-23T19:00:35.626 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T19:00:35.632 INFO:teuthology.task.internal:Opening connections... 2024-04-23T19:00:35.632 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-04-23T19:00:35.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T19:00:35.712 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:00:35.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T19:00:35.789 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T19:00:35.794 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-04-23T19:00:35.812 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-04-23T19:00:35.813 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-04-23T19:00:35.871 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-04-23T19:00:35.871 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-04-23T19:00:35.871 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-04-23T19:00:35.871 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-04-23T19:00:35.871 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T19:00:35.872 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T19:00:35.873 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-04-23T19:00:35.914 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-04-23T19:00:35.933 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-04-23T19:00:35.933 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T19:00:35.991 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-04-23T19:00:35.992 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T19:00:35.992 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T19:00:35.992 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T19:00:35.992 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-04-23T19:00:36.021 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T19:00:36.029 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T19:00:36.059 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T19:00:36.060 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-04-23T19:00:36.063 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-04-23T19:00:36.079 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T19:00:36.084 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T19:00:36.084 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-04-23T19:00:36.121 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-04-23T19:00:36.143 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T19:00:36.196 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T19:00:36.210 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-23T19:00:36.210 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-23T19:00:36.211 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T19:00:36.211 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T19:00:36.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:00:36.227 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-04-23T19:00:36.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:00:36.229 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-04-23T19:00:36.284 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-437.el9.x86_64 2024-04-23T19:00:36.284 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.14.0-437.el9.x86_64 2024-04-23T19:00:36.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum install -y kernel 2024-04-23T19:00:36.286 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-437.el9.x86_64 2024-04-23T19:00:36.286 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-437.el9.x86_64 2024-04-23T19:00:36.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-04-23T19:00:38.980 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-04-23T19:00:39.839 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:01 2024-04-23T19:00:41.298 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-23T19:00:46.365 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-04-23T19:00:48.116 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-23T19:00:50.650 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 21 MB 00:02 2024-04-23T19:00:50.831 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:09 2024-04-23T19:00:55.876 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-23T19:00:57.782 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T19:00:57.782 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T19:00:57.887 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:00:57.888 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:00:57.888 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:00:57.980 DEBUG:teuthology.orchestra.run.smithi016:> echo no | sudo yum reinstall kernel || true 2024-04-23T19:00:58.500 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 07:00:55 PM UTC. 2024-04-23T19:00:58.632 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:Reinstalling: 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:00:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:00:58.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:00:58.648 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 255 k 2024-04-23T19:00:58.648 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 0 2024-04-23T19:00:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:Operation aborted. 2024-04-23T19:00:58.701 INFO:teuthology.orchestra.run.smithi016.stdout:Is this ok [y/N]: 2024-04-23T19:00:58.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum reinstall -y kernel || true 2024-04-23T19:00:59.217 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:04 ago on Tue 23 Apr 2024 07:00:55 PM UTC. 2024-04-23T19:00:59.337 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:00:59.337 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:00:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout:Reinstalling: 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 255 k 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 0 2024-04-23T19:00:59.338 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:00:59.586 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-23T19:00:59.586 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:00:59.586 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.0 MB/s | 255 kB 00:00 2024-04-23T19:00:59.602 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:00:59.773 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:00:59.773 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:00:59.778 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:00:59.778 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:00:59.920 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:00:59.987 INFO:teuthology.orchestra.run.smithi016.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T19:00:59.987 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T19:01:00.654 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T19:01:00.654 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T19:01:01.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T19:01:01.404 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:01:01.404 INFO:teuthology.orchestra.run.smithi016.stdout:Reinstalled: 2024-04-23T19:01:01.404 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T19:01:01.404 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:01:01.404 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:01:01.575 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T19:01:01.623 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T19:01:01.623 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T19:01:01.623 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T19:01:01.623 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-23T19:01:01.623 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T19:01:01.658 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-04-23T19:01:01.681 DEBUG:teuthology.parallel:result is None 2024-04-23T19:01:02.614 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 786 kB/s | 6.2 MB 00:08 2024-04-23T19:01:05.391 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-04-23T19:01:29.098 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 908 kB/s | 21 MB 00:23 2024-04-23T19:01:34.231 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-23T19:01:36.145 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T19:01:36.145 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T19:01:36.233 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:01:36.234 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:01:36.235 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:01:36.360 DEBUG:teuthology.orchestra.run.smithi063:> echo no | sudo yum reinstall kernel || true 2024-04-23T19:01:36.880 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 07:01:34 PM UTC. 2024-04-23T19:01:37.005 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:01:37.026 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:01:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-23T19:01:37.026 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:01:37.026 INFO:teuthology.orchestra.run.smithi063.stdout:Reinstalling: 2024-04-23T19:01:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T19:01:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:01:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:01:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:01:37.027 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:01:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 255 k 2024-04-23T19:01:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 0 2024-04-23T19:01:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:Operation aborted. 2024-04-23T19:01:37.088 INFO:teuthology.orchestra.run.smithi063.stdout:Is this ok [y/N]: 2024-04-23T19:01:37.089 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum reinstall -y kernel || true 2024-04-23T19:01:37.605 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 07:01:34 PM UTC. 2024-04-23T19:01:37.725 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout:Reinstalling: 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:01:37.726 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:01:37.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:01:37.727 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 255 k 2024-04-23T19:01:37.727 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 0 2024-04-23T19:01:37.727 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:01:39.524 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-437.el9.x86_64.rpm 272 kB/s | 255 kB 00:00 2024-04-23T19:01:39.524 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:01:39.525 INFO:teuthology.orchestra.run.smithi063.stdout:Total 142 kB/s | 255 kB 00:01 2024-04-23T19:01:39.543 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:01:39.732 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:01:39.732 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:01:39.737 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:01:39.737 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:01:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:01:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T19:01:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T19:01:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T19:01:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T19:01:41.166 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T19:01:41.166 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:01:41.167 INFO:teuthology.orchestra.run.smithi063.stdout:Reinstalled: 2024-04-23T19:01:41.167 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T19:01:41.167 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:01:41.167 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:01:41.350 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T19:01:41.398 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T19:01:41.398 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T19:01:41.398 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T19:01:41.398 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-23T19:01:41.399 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T19:01:41.433 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-04-23T19:01:41.455 DEBUG:teuthology.parallel:result is None 2024-04-23T19:01:41.456 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T19:01:41.463 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T19:01:41.463 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T19:01:41.467 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T19:01:41.510 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T19:01:41.516 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T19:01:41.543 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T19:01:41.543 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T19:01:41.545 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T19:01:41.588 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T19:01:41.594 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T19:01:41.594 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T19:01:41.613 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T19:01:41.654 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T19:01:41.663 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T19:01:41.666 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T19:01:41.673 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T19:01:41.675 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T19:01:41.686 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T19:01:41.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T19:01:41.689 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T19:01:41.740 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T19:01:41.750 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T19:01:41.750 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T19:01:41.756 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T19:01:41.795 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T19:01:41.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T19:01:41.912 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T19:01:41.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T19:01:42.036 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:01:42.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T19:01:42.101 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T19:01:42.144 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T19:01:42.191 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T19:01:42.274 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T19:01:42.313 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:01:42.313 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T19:01:42.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-04-23T19:01:42.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-04-23T19:01:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T19:01:42.457 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T19:01:42.875 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T19:01:42.883 INFO:teuthology.task.internal:Starting timer... 2024-04-23T19:01:42.883 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T19:01:42.931 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T19:01:42.963 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-23T19:01:42.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-04-23T19:01:43.032 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-04-23T19:01:43.034 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-04-23T19:01:43.098 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-04-23T19:01:43.100 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T19:01:43.100 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-04-23T19:01:43.122 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-04-23T19:01:43.123 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-04-23T19:01:43.147 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-04-23T19:01:43.147 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi063.front.sepia.ceph.com': 'permissive'} 2024-04-23T19:01:43.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T19:01:43.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:01:43.188 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T19:01:43.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:01:43.214 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T19:01:43.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-04-23T19:01:43.253 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-04-23T19:01:43.283 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T19:01:43.292 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T19:01:43.310 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T19:01:43.312 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi063.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T19:05:36.376 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi063.front.sepia.ceph.com')] 2024-04-23T19:05:36.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-23T19:05:36.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T19:05:36.456 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-04-23T19:05:36.542 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-23T19:05:36.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-04-23T19:05:36.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T19:05:36.625 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-04-23T19:05:36.706 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-04-23T19:05:36.707 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T19:05:36.717 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T19:05:36.717 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T19:05:36.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T19:05:36.720 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T19:05:36.721 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T19:05:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T19:05:36.766 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T19:05:36.779 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T19:05:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-04-23T19:05:36.793 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T19:05:36.803 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-04-23T19:05:36.818 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-04-23T19:05:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T19:05:36.828 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-04-23T19:05:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T19:05:36.842 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T19:05:36.856 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T19:05:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-23T19:05:36.891 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T19:05:36.891 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-23T19:05:36.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.909 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-04-23T19:05:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T19:05:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-04-23T19:05:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:05:36.913 INFO:teuthology.run_tasks:Running task install... 2024-04-23T19:05:36.922 DEBUG:teuthology.task.install:project ceph 2024-04-23T19:05:36.922 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c'}} 2024-04-23T19:05:36.922 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c'} 2024-04-23T19:05:36.922 INFO:teuthology.task.install:Using flavor: default 2024-04-23T19:05:36.928 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T19:05:36.928 INFO:teuthology.task.install:extra packages: [] 2024-04-23T19:05:36.929 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c', 'tag': None, 'wait_for_package': False} 2024-04-23T19:05:36.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5bf3b3a9b161d70b9aa7806055bdae202e03111c 2024-04-23T19:05:36.931 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c', 'tag': None, 'wait_for_package': False} 2024-04-23T19:05:36.931 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5bf3b3a9b161d70b9aa7806055bdae202e03111c 2024-04-23T19:05:37.083 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/ 2024-04-23T19:05:37.083 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g5bf3b3a9 2024-04-23T19:05:37.158 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/ 2024-04-23T19:05:37.158 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g5bf3b3a9 2024-04-23T19:05:37.258 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T19:05:37.258 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:05:37.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T19:05:37.269 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rgw-update-boost-redis/5bf3b3a9b161d70b9aa7806055bdae202e03111c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T19:05:37.270 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:05:37.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T19:05:37.285 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T19:05:37.285 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5bf3b3a9b161d70b9aa7806055bdae202e03111c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T19:05:37.296 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T19:05:37.296 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5bf3b3a9b161d70b9aa7806055bdae202e03111c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T19:05:37.351 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T19:05:37.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T19:05:37.425 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T19:05:37.439 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T19:05:37.488 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-04-23T19:05:37.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-04-23T19:05:37.504 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-04-23T19:05:37.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-04-23T19:05:37.724 INFO:teuthology.orchestra.run.smithi063.stdout:48 files removed 2024-04-23T19:05:37.739 INFO:teuthology.orchestra.run.smithi016.stdout:48 files removed 2024-04-23T19:05:37.751 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw 2024-04-23T19:05:37.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw 2024-04-23T19:05:38.506 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-04-23T19:05:38.686 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 126 kB/s | 83 kB 00:00 2024-04-23T19:05:38.899 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-23T19:05:39.011 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-04-23T19:05:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-23T19:05:39.387 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-23T19:05:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-23T19:05:41.125 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-04-23T19:05:43.026 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-23T19:05:43.488 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 8.5 MB/s | 19 MB 00:02 2024-04-23T19:05:47.118 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-23T19:05:48.389 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.2 MB 00:01 2024-04-23T19:05:48.813 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-23T19:05:50.123 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-23T19:05:50.556 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-23T19:05:55.771 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-23T19:05:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:05:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 11 M 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 3.4 M 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 3.1 M 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 5.4 M 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 21 M 2024-04-23T19:05:58.019 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 25 k 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 746 k 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T19:05:58.020 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 501 k 2024-04-23T19:05:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T19:05:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 5.4 M 2024-04-23T19:05:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T19:05:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T19:05:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T19:05:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 45 k 2024-04-23T19:05:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 135 k 2024-04-23T19:05:58.022 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 162 k 2024-04-23T19:05:58.022 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T19:05:58.022 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 320 k 2024-04-23T19:05:58.022 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 298 k 2024-04-23T19:05:58.022 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 99 k 2024-04-23T19:05:58.022 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T19:05:58.023 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T19:05:58.023 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T19:05:58.023 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T19:05:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-04-23T19:05:58.023 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T19:05:58.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:05:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:05:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:05:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:Install 30 Packages 2024-04-23T19:05:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 2 Packages 2024-04-23T19:05:58.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:05:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 60 M 2024-04-23T19:05:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:05:58.764 INFO:teuthology.orchestra.run.smithi016.stdout:(1/32): ceph-base-19.0.0-3101.g5bf3b3a9.el9.x86 10 MB/s | 5.4 MB 00:00 2024-04-23T19:05:58.805 INFO:teuthology.orchestra.run.smithi016.stdout:(2/32): ceph-selinux-19.0.0-3101.g5bf3b3a9.el9. 607 kB/s | 25 kB 00:00 2024-04-23T19:05:58.872 INFO:teuthology.orchestra.run.smithi016.stdout:(3/32): libcephfs2-19.0.0-3101.g5bf3b3a9.el9.x8 11 MB/s | 746 kB 00:00 2024-04-23T19:05:58.939 INFO:teuthology.orchestra.run.smithi016.stdout:(4/32): libradosstriper1-19.0.0-3101.g5bf3b3a9. 7.3 MB/s | 501 kB 00:00 2024-04-23T19:05:59.240 INFO:teuthology.orchestra.run.smithi016.stdout:(5/32): librgw2-19.0.0-3101.g5bf3b3a9.el9.x86_6 18 MB/s | 5.4 MB 00:00 2024-04-23T19:05:59.290 INFO:teuthology.orchestra.run.smithi016.stdout:(6/32): python3-ceph-argparse-19.0.0-3101.g5bf3 907 kB/s | 45 kB 00:00 2024-04-23T19:05:59.365 INFO:teuthology.orchestra.run.smithi016.stdout:(7/32): python3-ceph-common-19.0.0-3101.g5bf3b3 2.7 MB/s | 135 kB 00:00 2024-04-23T19:05:59.415 INFO:teuthology.orchestra.run.smithi016.stdout:(8/32): python3-cephfs-19.0.0-3101.g5bf3b3a9.el 3.2 MB/s | 162 kB 00:00 2024-04-23T19:05:59.474 INFO:teuthology.orchestra.run.smithi016.stdout:(9/32): python3-rados-19.0.0-3101.g5bf3b3a9.el9 5.4 MB/s | 320 kB 00:00 2024-04-23T19:05:59.532 INFO:teuthology.orchestra.run.smithi016.stdout:(10/32): python3-rbd-19.0.0-3101.g5bf3b3a9.el9. 5.0 MB/s | 298 kB 00:00 2024-04-23T19:05:59.583 INFO:teuthology.orchestra.run.smithi016.stdout:(11/32): python3-rgw-19.0.0-3101.g5bf3b3a9.el9. 1.9 MB/s | 99 kB 00:00 2024-04-23T19:05:59.725 INFO:teuthology.orchestra.run.smithi016.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 756 kB/s | 107 kB 00:00 2024-04-23T19:05:59.884 INFO:teuthology.orchestra.run.smithi016.stdout:(13/32): ceph-radosgw-19.0.0-3101.g5bf3b3a9.el9 6.5 MB/s | 11 MB 00:01 2024-04-23T19:05:59.909 INFO:teuthology.orchestra.run.smithi016.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 872 kB/s | 160 kB 00:00 2024-04-23T19:06:00.026 INFO:teuthology.orchestra.run.smithi016.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-04-23T19:06:00.084 INFO:teuthology.orchestra.run.smithi016.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-04-23T19:06:00.134 INFO:teuthology.orchestra.run.smithi016.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-23T19:06:00.160 INFO:teuthology.orchestra.run.smithi016.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 560 kB/s | 42 kB 00:00 2024-04-23T19:06:00.210 INFO:teuthology.orchestra.run.smithi016.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 722 kB/s | 36 kB 00:00 2024-04-23T19:06:00.252 INFO:teuthology.orchestra.run.smithi016.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-04-23T19:06:00.277 INFO:teuthology.orchestra.run.smithi016.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-04-23T19:06:00.912 INFO:teuthology.orchestra.run.smithi016.stdout:(22/32): ceph-common-19.0.0-3101.g5bf3b3a9.el9. 7.9 MB/s | 21 MB 00:02 2024-04-23T19:06:09.953 INFO:teuthology.orchestra.run.smithi016.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 2.9 kB/s | 26 kB 00:09 2024-04-23T19:06:11.636 INFO:teuthology.orchestra.run.smithi016.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 29 kB/s | 49 kB 00:01 2024-04-23T19:06:11.670 INFO:teuthology.orchestra.run.smithi016.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 27 kB/s | 309 kB 00:11 2024-04-23T19:06:11.903 INFO:teuthology.orchestra.run.smithi016.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 252 kB/s | 67 kB 00:00 2024-04-23T19:06:12.504 INFO:teuthology.orchestra.run.smithi016.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.0 MB/s | 838 kB 00:00 2024-04-23T19:06:12.904 INFO:teuthology.orchestra.run.smithi016.stdout:(28/32): libarrow-9.0.0-10.el9.x86_64.rpm 358 kB/s | 4.4 MB 00:12 2024-04-23T19:06:12.929 INFO:teuthology.orchestra.run.smithi016.stdout:(29/32): re2-20211101-3.el9.x86_64.rpm 187 kB/s | 192 kB 00:01 2024-04-23T19:06:13.196 INFO:teuthology.orchestra.run.smithi016.stdout:(30/32): librados2-19.0.0-3101.g5bf3b3a9.el9.x8 12 MB/s | 3.4 MB 00:00 2024-04-23T19:06:13.404 INFO:teuthology.orchestra.run.smithi016.stdout:(31/32): librbd1-19.0.0-3101.g5bf3b3a9.el9.x86_ 6.6 MB/s | 3.1 MB 00:00 2024-04-23T19:06:14.038 INFO:teuthology.orchestra.run.smithi016.stdout:(32/32): thrift-0.15.0-2.el9.x86_64.rpm 1.0 MB/s | 1.6 MB 00:01 2024-04-23T19:06:14.044 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:06:14.044 INFO:teuthology.orchestra.run.smithi016.stdout:Total 3.8 MB/s | 60 MB 00:16 2024-04-23T19:06:14.145 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:06:14.185 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:06:14.186 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:06:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:06:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:06:16.056 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 830 kB/s | 21 MB 00:25 2024-04-23T19:06:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:06:16.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T19:06:16.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-23T19:06:16.310 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-23T19:06:16.341 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T19:06:16.364 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T19:06:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T19:06:16.596 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T19:06:17.029 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-23T19:06:17.095 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 7/34 2024-04-23T19:06:17.188 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 7/34 2024-04-23T19:06:17.277 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 8/34 2024-04-23T19:06:17.314 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 9/34 2024-04-23T19:06:17.378 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 9/34 2024-04-23T19:06:17.530 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-23T19:06:17.574 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g5bf3b3a9.el9. 11/34 2024-04-23T19:06:17.644 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 12/34 2024-04-23T19:06:17.665 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_6 13/34 2024-04-23T19:06:17.730 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_6 13/34 2024-04-23T19:06:17.764 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-23T19:06:17.875 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-23T19:06:17.906 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-23T19:06:17.947 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-23T19:06:17.974 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-23T19:06:18.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-23T19:06:18.036 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-23T19:06:18.365 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-23T19:06:18.415 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 22/34 2024-04-23T19:06:18.506 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 22/34 2024-04-23T19:06:18.536 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 23/34 2024-04-23T19:06:18.573 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T19:06:19.078 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T19:06:19.110 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 26/34 2024-04-23T19:06:19.172 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 26/34 2024-04-23T19:06:19.253 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 27/34 2024-04-23T19:06:19.403 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x8 28/34 2024-04-23T19:06:21.214 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-23T19:06:21.789 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 29/34 2024-04-23T19:06:22.008 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 29/34 2024-04-23T19:06:22.580 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 29/34 2024-04-23T19:06:22.620 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 30/34 2024-04-23T19:06:22.711 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 30/34 2024-04-23T19:06:22.712 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T19:06:22.712 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T19:06:22.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:22.738 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 31/34 2024-04-23T19:06:23.449 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 11 M 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 3.4 M 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 3.1 M 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:06:23.452 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 5.4 M 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 21 M 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 25 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 746 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 501 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T19:06:23.453 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 5.4 M 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 45 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 135 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 162 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 320 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 298 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 99 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T19:06:23.454 INFO:teuthology.orchestra.run.smithi063.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout:Install 30 Packages 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 2 Packages 2024-04-23T19:06:23.455 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:23.456 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 60 M 2024-04-23T19:06:23.456 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:06:24.340 INFO:teuthology.orchestra.run.smithi063.stdout:(1/32): ceph-base-19.0.0-3101.g5bf3b3a9.el9.x86 10 MB/s | 5.4 MB 00:00 2024-04-23T19:06:24.382 INFO:teuthology.orchestra.run.smithi063.stdout:(2/32): ceph-selinux-19.0.0-3101.g5bf3b3a9.el9. 607 kB/s | 25 kB 00:00 2024-04-23T19:06:24.457 INFO:teuthology.orchestra.run.smithi063.stdout:(3/32): libcephfs2-19.0.0-3101.g5bf3b3a9.el9.x8 9.7 MB/s | 746 kB 00:00 2024-04-23T19:06:24.516 INFO:teuthology.orchestra.run.smithi063.stdout:(4/32): libradosstriper1-19.0.0-3101.g5bf3b3a9. 8.4 MB/s | 501 kB 00:00 2024-04-23T19:06:24.791 INFO:teuthology.orchestra.run.smithi063.stdout:(5/32): librgw2-19.0.0-3101.g5bf3b3a9.el9.x86_6 20 MB/s | 5.4 MB 00:00 2024-04-23T19:06:24.825 INFO:teuthology.orchestra.run.smithi063.stdout:(6/32): python3-ceph-argparse-19.0.0-3101.g5bf3 1.3 MB/s | 45 kB 00:00 2024-04-23T19:06:24.877 INFO:teuthology.orchestra.run.smithi063.stdout:(7/32): python3-ceph-common-19.0.0-3101.g5bf3b3 2.5 MB/s | 135 kB 00:00 2024-04-23T19:06:24.919 INFO:teuthology.orchestra.run.smithi063.stdout:(8/32): python3-cephfs-19.0.0-3101.g5bf3b3a9.el 3.8 MB/s | 162 kB 00:00 2024-04-23T19:06:24.969 INFO:teuthology.orchestra.run.smithi063.stdout:(9/32): python3-rados-19.0.0-3101.g5bf3b3a9.el9 6.2 MB/s | 320 kB 00:00 2024-04-23T19:06:25.020 INFO:teuthology.orchestra.run.smithi063.stdout:(10/32): python3-rbd-19.0.0-3101.g5bf3b3a9.el9. 5.9 MB/s | 298 kB 00:00 2024-04-23T19:06:25.061 INFO:teuthology.orchestra.run.smithi063.stdout:(11/32): python3-rgw-19.0.0-3101.g5bf3b3a9.el9. 2.3 MB/s | 99 kB 00:00 2024-04-23T19:06:25.170 INFO:teuthology.orchestra.run.smithi063.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 986 kB/s | 107 kB 00:00 2024-04-23T19:06:25.229 INFO:teuthology.orchestra.run.smithi063.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-23T19:06:25.270 INFO:teuthology.orchestra.run.smithi063.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-23T19:06:25.354 INFO:teuthology.orchestra.run.smithi063.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-23T19:06:25.404 INFO:teuthology.orchestra.run.smithi063.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-23T19:06:25.446 INFO:teuthology.orchestra.run.smithi063.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-04-23T19:06:25.488 INFO:teuthology.orchestra.run.smithi063.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-23T19:06:25.530 INFO:teuthology.orchestra.run.smithi063.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 861 kB/s | 36 kB 00:00 2024-04-23T19:06:25.572 INFO:teuthology.orchestra.run.smithi063.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-23T19:06:26.222 INFO:teuthology.orchestra.run.smithi063.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 475 kB/s | 309 kB 00:00 2024-04-23T19:06:27.664 INFO:teuthology.orchestra.run.smithi063.stdout:(22/32): ceph-radosgw-19.0.0-3101.g5bf3b3a9.el9 2.7 MB/s | 11 MB 00:03 2024-04-23T19:06:27.881 INFO:teuthology.orchestra.run.smithi063.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 120 kB/s | 26 kB 00:00 2024-04-23T19:06:28.098 INFO:teuthology.orchestra.run.smithi063.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 224 kB/s | 49 kB 00:00 2024-04-23T19:06:28.398 INFO:teuthology.orchestra.run.smithi063.stdout:(25/32): ceph-common-19.0.0-3101.g5bf3b3a9.el9. 4.6 MB/s | 21 MB 00:04 2024-04-23T19:06:28.423 INFO:teuthology.orchestra.run.smithi063.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 207 kB/s | 67 kB 00:00 2024-04-23T19:06:28.724 INFO:teuthology.orchestra.run.smithi063.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 640 kB/s | 192 kB 00:00 2024-04-23T19:06:28.840 INFO:teuthology.orchestra.run.smithi063.stdout:(28/32): libarrow-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-04-23T19:06:29.124 INFO:teuthology.orchestra.run.smithi063.stdout:(29/32): librados2-19.0.0-3101.g5bf3b3a9.el9.x8 12 MB/s | 3.4 MB 00:00 2024-04-23T19:06:29.299 INFO:teuthology.orchestra.run.smithi063.stdout:(30/32): librbd1-19.0.0-3101.g5bf3b3a9.el9.x86_ 18 MB/s | 3.1 MB 00:00 2024-04-23T19:06:29.641 INFO:teuthology.orchestra.run.smithi063.stdout:(31/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 675 kB/s | 838 kB 00:01 2024-04-23T19:06:30.283 INFO:teuthology.orchestra.run.smithi063.stdout:(32/32): thrift-0.15.0-2.el9.x86_64.rpm 1.0 MB/s | 1.6 MB 00:01 2024-04-23T19:06:30.287 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:06:30.287 INFO:teuthology.orchestra.run.smithi063.stdout:Total 8.8 MB/s | 60 MB 00:06 2024-04-23T19:06:30.389 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:06:30.430 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:06:30.430 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:06:30.922 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:06:30.923 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:06:32.418 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 31/34 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-04-23T19:06:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-04-23T19:06:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T19:06:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-23T19:06:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-23T19:06:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T19:06:32.605 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T19:06:32.804 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T19:06:32.859 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T19:06:33.203 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-23T19:06:33.306 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 7/34 2024-04-23T19:06:33.398 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 7/34 2024-04-23T19:06:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/34 2024-04-23T19:06:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/34 2024-04-23T19:06:33.456 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:06:33.456 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T19:06:33.457 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T19:06:33.457 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T19:06:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T19:06:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 8/34 2024-04-23T19:06:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 9/34 2024-04-23T19:06:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T19:06:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T19:06:33.570 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 9/34 2024-04-23T19:06:33.599 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-23T19:06:33.643 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g5bf3b3a9.el9. 11/34 2024-04-23T19:06:33.721 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 12/34 2024-04-23T19:06:33.743 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_6 13/34 2024-04-23T19:06:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_6 13/34 2024-04-23T19:06:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-23T19:06:34.063 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-23T19:06:34.100 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-23T19:06:34.141 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-23T19:06:34.174 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-23T19:06:34.203 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-23T19:06:34.236 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-23T19:06:34.565 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-23T19:06:34.607 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 22/34 2024-04-23T19:06:34.706 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 22/34 2024-04-23T19:06:34.728 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 23/34 2024-04-23T19:06:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T19:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T19:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/34 2024-04-23T19:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/34 2024-04-23T19:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/34 2024-04-23T19:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/34 2024-04-23T19:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 5/34 2024-04-23T19:06:34.876 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_6 6/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 7/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g5bf3b3a9.el9. 8/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x8 9/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 10/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 11/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 12/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 13/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T19:06:34.877 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T19:06:34.878 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T19:06:34.879 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T19:06:34.879 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T19:06:34.879 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 31/34 2024-04-23T19:06:34.879 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T19:06:34.879 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 33/34 2024-04-23T19:06:35.270 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T19:06:35.301 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 26/34 2024-04-23T19:06:35.372 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 26/34 2024-04-23T19:06:35.553 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 27/34 2024-04-23T19:06:35.586 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x8 28/34 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.697 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.698 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T19:06:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T19:06:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T19:06:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:35.700 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:06:35.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-test 2024-04-23T19:06:36.492 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:41 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:06:36.615 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 49 M 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 163 k 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:06:36.616 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:06:36.617 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-04-23T19:06:36.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:36.617 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 50 M 2024-04-23T19:06:36.617 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 211 M 2024-04-23T19:06:36.617 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:06:37.126 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-04-23T19:06:37.184 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-23T19:06:37.345 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): libcephsqlite-19.0.0-3101.g5bf3b3a9.el9. 384 kB/s | 163 kB 00:00 2024-04-23T19:06:37.926 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 29/34 2024-04-23T19:06:37.958 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 29/34 2024-04-23T19:06:38.529 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 29/34 2024-04-23T19:06:38.562 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 30/34 2024-04-23T19:06:38.654 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 30/34 2024-04-23T19:06:38.654 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T19:06:38.654 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T19:06:38.654 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:38.703 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 31/34 2024-04-23T19:06:39.730 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): ceph-test-19.0.0-3101.g5bf3b3a9.el9.x86_ 17 MB/s | 49 MB 00:02 2024-04-23T19:06:39.730 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:06:39.730 INFO:teuthology.orchestra.run.smithi016.stdout:Total 16 MB/s | 50 MB 00:03 2024-04-23T19:06:39.757 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:06:39.780 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:06:39.780 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:06:40.171 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:06:40.172 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:06:40.722 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:06:40.788 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-23T19:06:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-23T19:06:40.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/4 2024-04-23T19:06:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/4 2024-04-23T19:06:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/4 2024-04-23T19:06:48.288 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/4 2024-04-23T19:06:48.289 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/4 2024-04-23T19:06:48.289 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/4 2024-04-23T19:06:48.289 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:48.547 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 31/34 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-04-23T19:06:48.577 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:48.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph 2024-04-23T19:06:49.308 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:54 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:06:49.435 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:06:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:06:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:06:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 6.6 k 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 2.4 M 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 1.5 M 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 249 k 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 4.7 M 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 17 M 2024-04-23T19:06:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T19:06:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T19:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout:Install 36 Packages 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:06:49.441 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 30 M 2024-04-23T19:06:49.442 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 107 M 2024-04-23T19:06:49.442 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:06:49.567 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/34 2024-04-23T19:06:49.593 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/34 2024-04-23T19:06:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:06:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T19:06:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T19:06:49.594 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T19:06:49.594 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:49.595 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T19:06:49.678 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T19:06:49.679 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T19:06:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T19:06:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/34 2024-04-23T19:06:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/34 2024-04-23T19:06:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/34 2024-04-23T19:06:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 5/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_6 6/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 7/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g5bf3b3a9.el9. 8/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x8 9/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 10/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 11/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 12/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 13/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T19:06:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T19:06:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T19:06:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T19:06:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T19:06:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 31/34 2024-04-23T19:06:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T19:06:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 33/34 2024-04-23T19:06:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T19:06:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:51.801 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-04-23T19:06:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:51.804 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:06:52.076 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-test 2024-04-23T19:06:52.646 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:31 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:06:52.768 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:06:52.769 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:06:52.769 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:06:52.769 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:06:52.769 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 49 M 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 163 k 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:06:52.770 INFO:teuthology.orchestra.run.smithi063.stdout:Install 4 Packages 2024-04-23T19:06:52.771 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:06:52.771 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 50 M 2024-04-23T19:06:52.771 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 211 M 2024-04-23T19:06:52.771 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:06:53.322 INFO:teuthology.orchestra.run.smithi063.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-23T19:06:53.373 INFO:teuthology.orchestra.run.smithi063.stdout:(2/4): libcephsqlite-19.0.0-3101.g5bf3b3a9.el9. 653 kB/s | 163 kB 00:00 2024-04-23T19:06:53.423 INFO:teuthology.orchestra.run.smithi063.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 632 kB/s | 64 kB 00:00 2024-04-23T19:06:55.498 INFO:teuthology.orchestra.run.smithi063.stdout:(4/4): ceph-test-19.0.0-3101.g5bf3b3a9.el9.x86_ 21 MB/s | 49 MB 00:02 2024-04-23T19:06:55.500 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:06:55.500 INFO:teuthology.orchestra.run.smithi063.stdout:Total 18 MB/s | 50 MB 00:02 2024-04-23T19:06:55.527 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:06:55.550 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:06:55.550 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:06:55.940 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:06:55.941 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:06:56.487 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:06:56.575 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-23T19:06:56.631 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-23T19:06:56.658 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/4 2024-04-23T19:07:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/4 2024-04-23T19:07:01.599 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/4 2024-04-23T19:07:04.039 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/4 2024-04-23T19:07:04.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/4 2024-04-23T19:07:04.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/4 2024-04-23T19:07:04.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-23T19:07:04.306 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:04.307 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:07:04.584 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph 2024-04-23T19:07:05.107 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:44 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:07:05.233 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 6.6 k 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 2.4 M 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 1.5 M 2024-04-23T19:07:05.236 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 249 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 4.7 M 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 17 M 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T19:07:05.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T19:07:05.238 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout:Install 36 Packages 2024-04-23T19:07:05.239 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:05.240 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 30 M 2024-04-23T19:07:05.240 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 107 M 2024-04-23T19:07:05.240 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:07:15.953 INFO:teuthology.orchestra.run.smithi016.stdout:(1/36): ceph-19.0.0-3101.g5bf3b3a9.el9.x86_64.r 28 kB/s | 6.6 kB 00:00 2024-04-23T19:07:16.111 INFO:teuthology.orchestra.run.smithi016.stdout:(2/36): ceph-mgr-19.0.0-3101.g5bf3b3a9.el9.x86_ 3.8 MB/s | 1.5 MB 00:00 2024-04-23T19:07:16.161 INFO:teuthology.orchestra.run.smithi016.stdout:(3/36): ceph-mds-19.0.0-3101.g5bf3b3a9.el9.x86_ 5.3 MB/s | 2.4 MB 00:00 2024-04-23T19:07:16.228 INFO:teuthology.orchestra.run.smithi016.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3101.g5bf3 3.6 MB/s | 249 kB 00:00 2024-04-23T19:07:16.428 INFO:teuthology.orchestra.run.smithi016.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-23T19:07:16.704 INFO:teuthology.orchestra.run.smithi016.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 4.5 MB/s | 1.2 MB 00:00 2024-04-23T19:07:16.754 INFO:teuthology.orchestra.run.smithi016.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-23T19:07:16.962 INFO:teuthology.orchestra.run.smithi016.stdout:(8/36): ceph-osd-19.0.0-3101.g5bf3b3a9.el9.x86_ 20 MB/s | 17 MB 00:00 2024-04-23T19:07:16.988 INFO:teuthology.orchestra.run.smithi016.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 579 kB/s | 135 kB 00:00 2024-04-23T19:07:17.072 INFO:teuthology.orchestra.run.smithi016.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-04-23T19:07:17.130 INFO:teuthology.orchestra.run.smithi016.stdout:(11/36): python3-requests-2.25.1-8.el9.noarch.r 749 kB/s | 125 kB 00:00 2024-04-23T19:07:17.214 INFO:teuthology.orchestra.run.smithi016.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-04-23T19:07:17.255 INFO:teuthology.orchestra.run.smithi016.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 834 kB/s | 35 kB 00:00 2024-04-23T19:07:17.281 INFO:teuthology.orchestra.run.smithi016.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-23T19:07:17.306 INFO:teuthology.orchestra.run.smithi016.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T19:07:17.331 INFO:teuthology.orchestra.run.smithi016.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-04-23T19:07:17.456 INFO:teuthology.orchestra.run.smithi016.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-23T19:07:17.482 INFO:teuthology.orchestra.run.smithi016.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-04-23T19:07:17.507 INFO:teuthology.orchestra.run.smithi016.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 865 kB/s | 43 kB 00:00 2024-04-23T19:07:17.657 INFO:teuthology.orchestra.run.smithi016.stdout:(20/36): ceph-mon-19.0.0-3101.g5bf3b3a9.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-04-23T19:07:17.682 INFO:teuthology.orchestra.run.smithi016.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 856 kB/s | 172 kB 00:00 2024-04-23T19:07:17.708 INFO:teuthology.orchestra.run.smithi016.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-04-23T19:07:17.750 INFO:teuthology.orchestra.run.smithi016.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 116 kB/s | 11 kB 00:00 2024-04-23T19:07:17.775 INFO:teuthology.orchestra.run.smithi016.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 265 kB/s | 18 kB 00:00 2024-04-23T19:07:17.800 INFO:teuthology.orchestra.run.smithi016.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 252 kB/s | 23 kB 00:00 2024-04-23T19:07:17.842 INFO:teuthology.orchestra.run.smithi016.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 469 kB/s | 19 kB 00:00 2024-04-23T19:07:17.867 INFO:teuthology.orchestra.run.smithi016.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 295 kB/s | 20 kB 00:00 2024-04-23T19:07:17.893 INFO:teuthology.orchestra.run.smithi016.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 505 kB/s | 46 kB 00:00 2024-04-23T19:07:17.918 INFO:teuthology.orchestra.run.smithi016.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-04-23T19:07:17.943 INFO:teuthology.orchestra.run.smithi016.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-04-23T19:07:17.977 INFO:teuthology.orchestra.run.smithi016.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-04-23T19:07:18.002 INFO:teuthology.orchestra.run.smithi016.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 610 kB/s | 36 kB 00:00 2024-04-23T19:07:18.027 INFO:teuthology.orchestra.run.smithi016.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-23T19:07:18.069 INFO:teuthology.orchestra.run.smithi016.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-23T19:07:18.094 INFO:teuthology.orchestra.run.smithi016.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2024-04-23T19:07:18.136 INFO:teuthology.orchestra.run.smithi016.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-04-23T19:07:18.138 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:18.138 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.1 MB/s | 30 MB 00:28 2024-04-23T19:07:18.277 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:07:18.321 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:07:18.321 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:07:18.633 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:07:18.634 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:07:19.932 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:07:19.989 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T19:07:20.022 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T19:07:20.050 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T19:07:20.116 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T19:07:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T19:07:20.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T19:07:20.294 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T19:07:20.371 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T19:07:20.418 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T19:07:20.453 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T19:07:20.483 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T19:07:20.509 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T19:07:20.542 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T19:07:20.583 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T19:07:20.627 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T19:07:20.660 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T19:07:20.699 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T19:07:20.775 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T19:07:20.926 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T19:07:21.009 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T19:07:21.061 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T19:07:21.099 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T19:07:21.157 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T19:07:21.220 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T19:07:21.416 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T19:07:21.461 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T19:07:21.541 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T19:07:21.632 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T19:07:21.669 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T19:07:21.790 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T19:07:21.965 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g5bf3b3a9.el9. 31/36 2024-04-23T19:07:22.004 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/36 2024-04-23T19:07:22.029 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/36 2024-04-23T19:07:22.029 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:22.029 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T19:07:22.029 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T19:07:22.029 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T19:07:22.029 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:23.546 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 33/36 2024-04-23T19:07:23.573 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 33/36 2024-04-23T19:07:23.573 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:23.573 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T19:07:23.574 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T19:07:23.574 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T19:07:23.574 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:24.038 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 34/36 2024-04-23T19:07:24.046 INFO:teuthology.orchestra.run.smithi063.stdout:(1/36): ceph-19.0.0-3101.g5bf3b3a9.el9.x86_64.r 32 kB/s | 6.6 kB 00:00 2024-04-23T19:07:24.062 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 34/36 2024-04-23T19:07:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T19:07:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T19:07:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T19:07:24.063 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:24.297 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 35/36 2024-04-23T19:07:24.305 INFO:teuthology.orchestra.run.smithi063.stdout:(2/36): ceph-mgr-19.0.0-3101.g5bf3b3a9.el9.x86_ 3.2 MB/s | 1.5 MB 00:00 2024-04-23T19:07:24.321 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 35/36 2024-04-23T19:07:24.321 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:24.321 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T19:07:24.322 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T19:07:24.322 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T19:07:24.322 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:24.387 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 36/36 2024-04-23T19:07:24.397 INFO:teuthology.orchestra.run.smithi063.stdout:(3/36): ceph-mds-19.0.0-3101.g5bf3b3a9.el9.x86_ 4.3 MB/s | 2.4 MB 00:00 2024-04-23T19:07:24.481 INFO:teuthology.orchestra.run.smithi063.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3101.g5bf3 2.9 MB/s | 249 kB 00:00 2024-04-23T19:07:24.614 INFO:teuthology.orchestra.run.smithi063.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-04-23T19:07:24.698 INFO:teuthology.orchestra.run.smithi063.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 15 MB/s | 1.2 MB 00:00 2024-04-23T19:07:24.739 INFO:teuthology.orchestra.run.smithi063.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-23T19:07:24.790 INFO:teuthology.orchestra.run.smithi063.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-04-23T19:07:24.832 INFO:teuthology.orchestra.run.smithi063.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-04-23T19:07:24.874 INFO:teuthology.orchestra.run.smithi063.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.1 MB/s | 215 kB 00:00 2024-04-23T19:07:25.041 INFO:teuthology.orchestra.run.smithi063.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-23T19:07:25.099 INFO:teuthology.orchestra.run.smithi063.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-23T19:07:25.158 INFO:teuthology.orchestra.run.smithi063.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 595 kB/s | 35 kB 00:00 2024-04-23T19:07:25.209 INFO:teuthology.orchestra.run.smithi063.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T19:07:25.258 INFO:teuthology.orchestra.run.smithi063.stdout:(15/36): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-04-23T19:07:25.384 INFO:teuthology.orchestra.run.smithi063.stdout:(16/36): ceph-mon-19.0.0-3101.g5bf3b3a9.el9.x86 3.5 MB/s | 4.7 MB 00:01 2024-04-23T19:07:25.409 INFO:teuthology.orchestra.run.smithi063.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-04-23T19:07:25.676 INFO:teuthology.orchestra.run.smithi063.stdout:(18/36): ceph-osd-19.0.0-3101.g5bf3b3a9.el9.x86 12 MB/s | 17 MB 00:01 2024-04-23T19:07:25.726 INFO:teuthology.orchestra.run.smithi063.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-04-23T19:07:25.918 INFO:teuthology.orchestra.run.smithi063.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 283 kB/s | 151 kB 00:00 2024-04-23T19:07:26.018 INFO:teuthology.orchestra.run.smithi063.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-04-23T19:07:26.085 INFO:teuthology.orchestra.run.smithi063.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-04-23T19:07:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 36/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 5/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g5bf3b3a9.el9. 6/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T19:07:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T19:07:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T19:07:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T19:07:26.127 INFO:teuthology.orchestra.run.smithi063.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-04-23T19:07:26.152 INFO:teuthology.orchestra.run.smithi063.stdout:(24/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 361 kB/s | 172 kB 00:00 2024-04-23T19:07:26.185 INFO:teuthology.orchestra.run.smithi063.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 230 kB/s | 23 kB 00:00 2024-04-23T19:07:26.236 INFO:teuthology.orchestra.run.smithi063.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-04-23T19:07:26.261 INFO:teuthology.orchestra.run.smithi063.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-04-23T19:07:26.286 INFO:teuthology.orchestra.run.smithi063.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 460 kB/s | 46 kB 00:00 2024-04-23T19:07:26.370 INFO:teuthology.orchestra.run.smithi063.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 589 kB/s | 79 kB 00:00 2024-04-23T19:07:26.387 INFO:teuthology.orchestra.run.smithi063.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-04-23T19:07:26.454 INFO:teuthology.orchestra.run.smithi063.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-23T19:07:26.488 INFO:teuthology.orchestra.run.smithi063.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 356 kB/s | 36 kB 00:00 2024-04-23T19:07:26.638 INFO:teuthology.orchestra.run.smithi063.stdout:(33/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 336 kB/s | 90 kB 00:00 2024-04-23T19:07:26.663 INFO:teuthology.orchestra.run.smithi063.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-04-23T19:07:26.696 INFO:teuthology.orchestra.run.smithi063.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.0 MB/s | 427 kB 00:00 2024-04-23T19:07:26.738 INFO:teuthology.orchestra.run.smithi063.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 199 kB/s | 20 kB 00:00 2024-04-23T19:07:26.740 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:26.741 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.4 MB/s | 30 MB 00:21 2024-04-23T19:07:26.879 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:07:26.923 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:07:26.923 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:07:27.092 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T19:07:27.093 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T19:07:27.094 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T19:07:27.095 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:27.096 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:07:27.235 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:07:27.235 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:07:27.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-base 2024-04-23T19:07:27.961 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:32 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:07:28.041 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:07:28.084 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:07:28.085 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:07:28.085 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:07:28.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install cephadm 2024-04-23T19:07:28.526 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:07:28.583 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T19:07:28.625 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T19:07:28.652 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T19:07:28.663 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:33 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:07:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T19:07:28.768 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T19:07:28.790 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:07:28.791 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:28.791 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:07:28.791 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:28.791 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 762 k 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 762 k 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 769 k 2024-04-23T19:07:28.792 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:07:28.800 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T19:07:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T19:07:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T19:07:29.037 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T19:07:29.071 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T19:07:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T19:07:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T19:07:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T19:07:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T19:07:29.230 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm-19.0.0-3101.g5bf3b3a9.el9.noarch.rpm 1.7 MB/s | 762 kB 00:00 2024-04-23T19:07:29.230 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:29.230 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.7 MB/s | 762 kB 00:00 2024-04-23T19:07:29.231 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:07:29.236 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:07:29.236 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:07:29.242 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:07:29.243 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:07:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T19:07:29.295 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T19:07:29.319 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:07:29.334 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T19:07:29.411 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T19:07:29.561 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T19:07:29.644 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T19:07:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T19:07:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T19:07:29.792 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:29.799 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T19:07:29.863 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T19:07:29.877 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:30.058 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T19:07:30.112 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T19:07:30.193 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T19:07:30.281 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T19:07:30.319 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T19:07:30.332 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:30.441 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T19:07:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:30.556 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:07:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:30.556 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:07:30.616 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g5bf3b3a9.el9. 31/36 2024-04-23T19:07:30.645 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/36 2024-04-23T19:07:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 32/36 2024-04-23T19:07:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T19:07:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T19:07:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T19:07:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:30.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T19:07:31.296 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:36 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:07:31.420 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:07:31.420 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 145 k 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 145 k 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 435 k 2024-04-23T19:07:31.421 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:07:31.636 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-immutable-object-cache-19.0.0-3101.g5bf3b3 675 kB/s | 145 kB 00:00 2024-04-23T19:07:31.637 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:31.637 INFO:teuthology.orchestra.run.smithi016.stdout:Total 668 kB/s | 145 kB 00:00 2024-04-23T19:07:31.638 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:07:31.646 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:07:31.646 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:07:31.707 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:07:31.708 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:07:31.893 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:07:31.979 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9. 1/1 2024-04-23T19:07:32.005 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9. 1/1 2024-04-23T19:07:32.005 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:32.005 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T19:07:32.005 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:32.205 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 33/36 2024-04-23T19:07:32.234 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 33/36 2024-04-23T19:07:32.235 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:32.235 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T19:07:32.235 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T19:07:32.235 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T19:07:32.235 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:32.697 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 34/36 2024-04-23T19:07:32.722 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 34/36 2024-04-23T19:07:32.722 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:32.722 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T19:07:32.722 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T19:07:32.722 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T19:07:32.722 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:32.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9. 1/1 2024-04-23T19:07:32.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:07:32.872 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:32.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:07:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 35/36 2024-04-23T19:07:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 35/36 2024-04-23T19:07:32.981 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:32.981 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T19:07:32.981 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T19:07:32.981 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T19:07:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 36/36 2024-04-23T19:07:33.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr 2024-04-23T19:07:33.617 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:38 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:07:33.697 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:07:33.743 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:07:33.744 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:07:33.744 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:07:33.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T19:07:34.328 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:39 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:07:34.452 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:07:34.453 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:34.453 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 3.5 M 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 34 k 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 16 k 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T19:07:34.454 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout:Install 12 Packages 2024-04-23T19:07:34.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:34.456 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 8.6 M 2024-04-23T19:07:34.456 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 104 M 2024-04-23T19:07:34.456 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:07:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 36/36 2024-04-23T19:07:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/36 2024-04-23T19:07:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/36 2024-04-23T19:07:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 3/36 2024-04-23T19:07:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 4/36 2024-04-23T19:07:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 5/36 2024-04-23T19:07:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g5bf3b3a9.el9. 6/36 2024-04-23T19:07:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T19:07:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T19:07:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T19:07:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T19:07:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T19:07:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T19:07:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T19:07:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T19:07:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T19:07:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T19:07:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T19:07:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T19:07:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T19:07:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T19:07:35.724 INFO:teuthology.orchestra.run.smithi016.stdout:(1/12): ceph-grafana-dashboards-19.0.0-3101.g5b 78 kB/s | 34 kB 00:00 2024-04-23T19:07:35.738 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T19:07:35.738 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:35.738 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:07:35.739 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:35.739 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:35.739 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:35.739 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:35.739 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:35.739 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:35.740 INFO:teuthology.orchestra.run.smithi063.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T19:07:35.740 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T19:07:35.740 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T19:07:35.740 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T19:07:35.741 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T19:07:35.741 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T19:07:35.741 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T19:07:35.741 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T19:07:35.741 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T19:07:35.741 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T19:07:35.742 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:35.743 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:07:35.790 INFO:teuthology.orchestra.run.smithi016.stdout:(2/12): ceph-prometheus-alerts-19.0.0-3101.g5bf 33 kB/s | 16 kB 00:00 2024-04-23T19:07:35.916 INFO:teuthology.orchestra.run.smithi016.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-23T19:07:35.942 INFO:teuthology.orchestra.run.smithi016.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.7 MB/s | 269 kB 00:00 2024-04-23T19:07:36.073 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-base 2024-04-23T19:07:36.342 INFO:teuthology.orchestra.run.smithi016.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.3 MB/s | 548 kB 00:00 2024-04-23T19:07:36.426 INFO:teuthology.orchestra.run.smithi016.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-04-23T19:07:36.604 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:15 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:07:36.684 INFO:teuthology.orchestra.run.smithi016.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 7.9 MB/s | 2.0 MB 00:00 2024-04-23T19:07:36.685 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-base-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:07:36.728 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:07:36.730 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:07:36.730 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:07:36.734 INFO:teuthology.orchestra.run.smithi016.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-04-23T19:07:36.776 INFO:teuthology.orchestra.run.smithi016.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 741 kB/s | 31 kB 00:00 2024-04-23T19:07:36.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install cephadm 2024-04-23T19:07:36.826 INFO:teuthology.orchestra.run.smithi016.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-04-23T19:07:37.316 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:16 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:07:37.440 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:07:37.441 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:37.441 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 762 k 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:37.442 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 762 k 2024-04-23T19:07:37.443 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 769 k 2024-04-23T19:07:37.443 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:07:37.677 INFO:teuthology.orchestra.run.smithi016.stdout:(11/12): ceph-mgr-dashboard-19.0.0-3101.g5bf3b3 1.5 MB/s | 3.5 MB 00:02 2024-04-23T19:07:37.797 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm-19.0.0-3101.g5bf3b3a9.el9.noarch.rpm 2.1 MB/s | 762 kB 00:00 2024-04-23T19:07:37.797 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:37.798 INFO:teuthology.orchestra.run.smithi063.stdout:Total 2.1 MB/s | 762 kB 00:00 2024-04-23T19:07:37.798 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:07:37.803 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:07:37.803 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:07:37.809 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:07:37.810 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:07:37.881 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:07:37.935 INFO:teuthology.orchestra.run.smithi016.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 428 kB/s | 863 kB 00:02 2024-04-23T19:07:37.939 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:37.939 INFO:teuthology.orchestra.run.smithi016.stdout:Total 2.5 MB/s | 8.6 MB 00:03 2024-04-23T19:07:37.992 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:07:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:07:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:07:38.180 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:07:38.180 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:07:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:38.435 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:38.717 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:07:38.760 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:38.795 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-23T19:07:38.838 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-23T19:07:38.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-23T19:07:38.923 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-23T19:07:38.964 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/1 2024-04-23T19:07:38.964 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:38.964 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:07:38.964 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:38.964 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:38.964 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:07:39.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-23T19:07:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-23T19:07:39.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T19:07:39.239 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-23T19:07:39.296 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-23T19:07:39.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-23T19:07:39.356 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g5bf3b3a9.el9 10/12 2024-04-23T19:07:39.700 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:18 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:07:39.824 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 145 k 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-23T19:07:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:39.826 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 145 k 2024-04-23T19:07:39.826 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 435 k 2024-04-23T19:07:39.826 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:07:40.052 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-immutable-object-cache-19.0.0-3101.g5bf3b3 641 kB/s | 145 kB 00:00 2024-04-23T19:07:40.053 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:40.053 INFO:teuthology.orchestra.run.smithi063.stdout:Total 636 kB/s | 145 kB 00:00 2024-04-23T19:07:40.053 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:07:40.062 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:07:40.062 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:07:40.122 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:07:40.122 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:07:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:07:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9. 1/1 2024-04-23T19:07:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9. 1/1 2024-04-23T19:07:40.370 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:07:40.370 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T19:07:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:40.460 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g5bf3b3a9.el 11/12 2024-04-23T19:07:40.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noa 12/12 2024-04-23T19:07:41.276 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9. 1/1 2024-04-23T19:07:41.277 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:41.277 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:07:41.277 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:07:41.277 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:41.277 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:07:41.471 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr 2024-04-23T19:07:41.842 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noa 12/12 2024-04-23T19:07:41.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g5bf3b3a9.el 1/12 2024-04-23T19:07:41.842 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noa 2/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g5bf3b3a9.el9 3/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-23T19:07:41.843 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-23T19:07:42.005 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:21 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:07:42.108 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-mgr-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T19:07:42.142 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:42.143 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:07:42.152 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:07:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:07:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:07:42.212 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T19:07:42.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T19:07:42.738 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:21 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:07:42.863 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:07:42.864 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:42.865 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:07:42.865 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:42.865 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:07:42.865 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 3.5 M 2024-04-23T19:07:42.865 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:07:42.866 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T19:07:42.866 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 34 k 2024-04-23T19:07:42.866 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 16 k 2024-04-23T19:07:42.866 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T19:07:42.866 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T19:07:42.867 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T19:07:42.867 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T19:07:42.867 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T19:07:42.867 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T19:07:42.867 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T19:07:42.868 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T19:07:42.868 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:42.868 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:07:42.868 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:07:42.868 INFO:teuthology.orchestra.run.smithi063.stdout:Install 12 Packages 2024-04-23T19:07:42.868 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:42.870 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 8.6 M 2024-04-23T19:07:42.870 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 104 M 2024-04-23T19:07:42.870 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:07:42.906 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:47 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:07:43.033 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:07:43.034 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-23T19:07:43.034 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:07:43.034 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 7.4 M 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T19:07:43.035 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T19:07:43.036 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T19:07:43.036 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:43.036 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:07:43.036 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-23T19:07:43.036 INFO:teuthology.orchestra.run.smithi016.stdout:Install 12 Packages 2024-04-23T19:07:43.036 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:43.036 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 38 M 2024-04-23T19:07:43.037 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 209 M 2024-04-23T19:07:43.037 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:07:43.797 INFO:teuthology.orchestra.run.smithi063.stdout:(1/12): ceph-prometheus-alerts-19.0.0-3101.g5bf 40 kB/s | 16 kB 00:00 2024-04-23T19:07:43.861 INFO:teuthology.orchestra.run.smithi016.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 439 kB/s | 195 kB 00:00 2024-04-23T19:07:43.952 INFO:teuthology.orchestra.run.smithi016.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 358 kB/s | 33 kB 00:00 2024-04-23T19:07:43.981 INFO:teuthology.orchestra.run.smithi063.stdout:(2/12): ceph-grafana-dashboards-19.0.0-3101.g5b 57 kB/s | 34 kB 00:00 2024-04-23T19:07:43.986 INFO:teuthology.orchestra.run.smithi016.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-23T19:07:44.120 INFO:teuthology.orchestra.run.smithi016.stdout:(4/12): ceph-mgr-diskprediction-local-19.0.0-31 11 MB/s | 7.4 MB 00:00 2024-04-23T19:07:44.145 INFO:teuthology.orchestra.run.smithi016.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 115 kB/s | 18 kB 00:00 2024-04-23T19:07:44.212 INFO:teuthology.orchestra.run.smithi016.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 385 kB/s | 35 kB 00:00 2024-04-23T19:07:44.271 INFO:teuthology.orchestra.run.smithi016.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.4 MB/s | 3.0 MB 00:00 2024-04-23T19:07:44.337 INFO:teuthology.orchestra.run.smithi016.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-04-23T19:07:44.404 INFO:teuthology.orchestra.run.smithi016.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.7 MB/s | 250 kB 00:00 2024-04-23T19:07:44.555 INFO:teuthology.orchestra.run.smithi016.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-04-23T19:07:44.621 INFO:teuthology.orchestra.run.smithi016.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 10 MB/s | 4.7 MB 00:00 2024-04-23T19:07:44.731 INFO:teuthology.orchestra.run.smithi063.stdout:(3/12): ceph-mgr-dashboard-19.0.0-3101.g5bf3b3a 2.6 MB/s | 3.5 MB 00:01 2024-04-23T19:07:44.907 INFO:teuthology.orchestra.run.smithi063.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.8 MB/s | 863 kB 00:00 2024-04-23T19:07:45.115 INFO:teuthology.orchestra.run.smithi063.stdout:(5/12): python3-protobuf-3.14.0-13.el9.noarch.r 237 kB/s | 269 kB 00:01 2024-04-23T19:07:45.241 INFO:teuthology.orchestra.run.smithi063.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.6 MB/s | 548 kB 00:00 2024-04-23T19:07:45.374 INFO:teuthology.orchestra.run.smithi063.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-23T19:07:45.466 INFO:teuthology.orchestra.run.smithi063.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 9.1 MB/s | 2.0 MB 00:00 2024-04-23T19:07:45.691 INFO:teuthology.orchestra.run.smithi063.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 137 kB/s | 31 kB 00:00 2024-04-23T19:07:45.722 INFO:teuthology.orchestra.run.smithi016.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-23T19:07:45.723 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:45.723 INFO:teuthology.orchestra.run.smithi016.stdout:Total 14 MB/s | 38 MB 00:02 2024-04-23T19:07:45.800 INFO:teuthology.orchestra.run.smithi063.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 340 kB/s | 144 kB 00:00 2024-04-23T19:07:45.958 INFO:teuthology.orchestra.run.smithi063.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 707 kB/s | 188 kB 00:00 2024-04-23T19:07:46.090 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:07:46.108 INFO:teuthology.orchestra.run.smithi063.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 447 kB/s | 1.0 MB 00:02 2024-04-23T19:07:46.112 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:46.112 INFO:teuthology.orchestra.run.smithi063.stdout:Total 2.7 MB/s | 8.6 MB 00:03 2024-04-23T19:07:46.118 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:07:46.118 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:07:46.165 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:07:46.185 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:07:46.185 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:07:46.352 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:07:46.353 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:07:46.553 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:07:46.553 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:07:46.897 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:07:46.968 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-23T19:07:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-23T19:07:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-23T19:07:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-23T19:07:47.160 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-23T19:07:47.295 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-23T19:07:47.387 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-23T19:07:47.414 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:07:47.460 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-23T19:07:47.476 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T19:07:47.488 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-23T19:07:47.520 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g5bf3b3a9.el9 10/12 2024-04-23T19:07:47.520 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T19:07:47.625 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T19:07:47.664 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T19:07:47.874 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T19:07:47.904 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T19:07:47.983 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T19:07:48.627 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g5bf3b3a9.el 11/12 2024-04-23T19:07:48.661 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noa 12/12 2024-04-23T19:07:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T19:07:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T19:07:49.977 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noa 12/12 2024-04-23T19:07:49.977 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g5bf3b3a9.el 1/12 2024-04-23T19:07:49.977 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noa 2/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g5bf3b3a9.el9 3/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-23T19:07:49.978 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-23T19:07:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T19:07:50.267 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T19:07:50.268 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T19:07:50.268 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T19:07:50.268 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T19:07:50.268 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T19:07:50.268 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T19:07:50.268 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:50.268 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:07:50.528 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T19:07:51.063 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:30 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:07:51.170 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T19:07:51.189 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:07:51.190 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 7.4 M 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T19:07:51.191 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout:Install 12 Packages 2024-04-23T19:07:51.192 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:07:51.193 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 38 M 2024-04-23T19:07:51.193 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 209 M 2024-04-23T19:07:51.193 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:07:51.197 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b 12/12 2024-04-23T19:07:52.073 INFO:teuthology.orchestra.run.smithi063.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-04-23T19:07:52.115 INFO:teuthology.orchestra.run.smithi063.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-23T19:07:52.757 INFO:teuthology.orchestra.run.smithi063.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 48 kB/s | 33 kB 00:00 2024-04-23T19:07:53.349 INFO:teuthology.orchestra.run.smithi063.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 31 kB/s | 18 kB 00:00 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b 12/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b 1/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T19:07:53.875 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T19:07:53.983 INFO:teuthology.orchestra.run.smithi063.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 56 kB/s | 35 kB 00:00 2024-04-23T19:07:54.288 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T19:07:54.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T19:07:54.290 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T19:07:54.290 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T19:07:54.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:54.290 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:07:54.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-rook 2024-04-23T19:07:55.177 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:00 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:07:55.304 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 49 k 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T19:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout:Install 13 Packages 2024-04-23T19:07:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:07:55.307 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 2.2 M 2024-04-23T19:07:55.307 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 26 M 2024-04-23T19:07:55.307 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:07:55.779 INFO:teuthology.orchestra.run.smithi016.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 224 kB/s | 26 kB 00:00 2024-04-23T19:07:55.796 INFO:teuthology.orchestra.run.smithi016.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 146 kB/s | 19 kB 00:00 2024-04-23T19:07:55.888 INFO:teuthology.orchestra.run.smithi016.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.4 MB/s | 222 kB 00:00 2024-04-23T19:07:55.914 INFO:teuthology.orchestra.run.smithi016.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-23T19:07:55.972 INFO:teuthology.orchestra.run.smithi016.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 919 kB/s | 54 kB 00:00 2024-04-23T19:07:55.997 INFO:teuthology.orchestra.run.smithi016.stdout:(6/13): ceph-mgr-rook-19.0.0-3101.g5bf3b3a9.el9 148 kB/s | 49 kB 00:00 2024-04-23T19:07:56.064 INFO:teuthology.orchestra.run.smithi016.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-04-23T19:07:56.282 INFO:teuthology.orchestra.run.smithi016.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2024-04-23T19:07:56.298 INFO:teuthology.orchestra.run.smithi016.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 50 kB/s | 15 kB 00:00 2024-04-23T19:07:56.440 INFO:teuthology.orchestra.run.smithi016.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 417 kB/s | 59 kB 00:00 2024-04-23T19:07:56.515 INFO:teuthology.orchestra.run.smithi016.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 512 kB/s | 206 kB 00:00 2024-04-23T19:07:56.566 INFO:teuthology.orchestra.run.smithi016.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 716 kB/s | 90 kB 00:00 2024-04-23T19:07:56.616 INFO:teuthology.orchestra.run.smithi016.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.1 MB/s | 1.0 MB 00:00 2024-04-23T19:07:56.620 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:07:56.620 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-04-23T19:07:56.684 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:07:56.700 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:07:56.700 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:07:56.734 INFO:teuthology.orchestra.run.smithi063.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 663 kB/s | 3.0 MB 00:04 2024-04-23T19:07:56.794 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:07:56.795 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:07:57.310 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:07:57.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T19:07:57.553 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T19:07:57.609 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T19:07:57.633 INFO:teuthology.orchestra.run.smithi063.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 271 kB/s | 244 kB 00:00 2024-04-23T19:07:57.644 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T19:07:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T19:07:57.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T19:07:57.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-23T19:07:57.971 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T19:07:58.533 INFO:teuthology.orchestra.run.smithi063.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-31 1.1 MB/s | 7.4 MB 00:06 2024-04-23T19:07:58.738 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T19:07:58.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T19:07:58.818 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T19:07:58.868 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T19:07:58.895 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 13/13 2024-04-23T19:07:59.449 INFO:teuthology.orchestra.run.smithi063.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 273 kB/s | 250 kB 00:00 2024-04-23T19:07:59.674 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 13/13 2024-04-23T19:07:59.674 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/13 2024-04-23T19:07:59.674 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T19:07:59.674 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T19:07:59.674 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T19:08:00.005 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T19:08:00.006 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T19:08:00.007 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T19:08:00.007 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T19:08:00.007 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T19:08:00.007 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:00.007 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:00.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T19:08:00.775 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:05 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:00.900 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:00.901 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:00.901 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:08:00.901 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:00.902 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:00.902 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 142 k 2024-04-23T19:08:00.902 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:08:00.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T19:08:00.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T19:08:00.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T19:08:00.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T19:08:00.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T19:08:00.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:00.903 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:00.903 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:00.903 INFO:teuthology.orchestra.run.smithi016.stdout:Install 6 Packages 2024-04-23T19:08:00.904 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:00.904 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 7.0 M 2024-04-23T19:08:00.905 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 33 M 2024-04-23T19:08:00.905 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:01.125 INFO:teuthology.orchestra.run.smithi063.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 679 kB/s | 4.7 MB 00:07 2024-04-23T19:08:05.450 INFO:teuthology.orchestra.run.smithi063.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 668 kB/s | 5.1 MB 00:07 2024-04-23T19:08:16.009 INFO:teuthology.orchestra.run.smithi016.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 894 kB/s | 248 kB 00:00 2024-04-23T19:08:16.042 INFO:teuthology.orchestra.run.smithi016.stdout:(2/6): ceph-mgr-cephadm-19.0.0-3101.g5bf3b3a9.e 457 kB/s | 142 kB 00:00 2024-04-23T19:08:16.293 INFO:teuthology.orchestra.run.smithi016.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-23T19:08:16.643 INFO:teuthology.orchestra.run.smithi016.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 96 kB/s | 58 kB 00:00 2024-04-23T19:08:16.834 INFO:teuthology.orchestra.run.smithi016.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 95 kB/s | 51 kB 00:00 2024-04-23T19:08:17.293 INFO:teuthology.orchestra.run.smithi016.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 427 kB/s | 548 kB 00:01 2024-04-23T19:08:17.297 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:17.297 INFO:teuthology.orchestra.run.smithi016.stdout:Total 436 kB/s | 7.0 MB 00:16 2024-04-23T19:08:17.372 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:17.385 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:17.385 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:17.490 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:17.490 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:17.826 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:18.008 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T19:08:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T19:08:18.624 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T19:08:18.719 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T19:08:18.790 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T19:08:18.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 6/6 2024-04-23T19:08:19.561 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 6/6 2024-04-23T19:08:19.562 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/6 2024-04-23T19:08:19.562 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T19:08:19.562 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T19:08:19.562 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T19:08:19.562 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T19:08:19.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T19:08:19.846 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:19.846 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:20.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-fuse 2024-04-23T19:08:20.599 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:25 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:20.723 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 891 k 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:Install 2 Packages 2024-04-23T19:08:20.724 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:20.725 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 971 k 2024-04-23T19:08:20.725 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 2.8 M 2024-04-23T19:08:20.725 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:21.039 INFO:teuthology.orchestra.run.smithi016.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 329 kB/s | 80 kB 00:00 2024-04-23T19:08:21.098 INFO:teuthology.orchestra.run.smithi016.stdout:(2/2): ceph-fuse-19.0.0-3101.g5bf3b3a9.el9.x86_ 2.9 MB/s | 891 kB 00:00 2024-04-23T19:08:21.099 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:21.099 INFO:teuthology.orchestra.run.smithi016.stdout:Total 2.5 MB/s | 971 kB 00:00 2024-04-23T19:08:21.110 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:21.132 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:21.132 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:21.216 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:21.399 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:21.545 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T19:08:21.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/2 2024-04-23T19:08:21.998 INFO:teuthology.orchestra.run.smithi063.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 738 kB/s | 16 MB 00:22 2024-04-23T19:08:22.000 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:22.000 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.2 MB/s | 38 MB 00:30 2024-04-23T19:08:22.194 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/2 2024-04-23T19:08:22.194 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/2 2024-04-23T19:08:22.361 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:08:22.390 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:08:22.390 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:08:22.454 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T19:08:22.454 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:22.454 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:22.454 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T19:08:22.454 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:22.454 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:22.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-volume 2024-04-23T19:08:22.826 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:08:22.827 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:08:23.256 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:28 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:23.381 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:23.381 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:23.381 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-23T19:08:23.381 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 273 k 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout:Install 2 Packages 2024-04-23T19:08:23.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:23.383 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 350 k 2024-04-23T19:08:23.383 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 1.5 M 2024-04-23T19:08:23.384 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:23.669 INFO:teuthology.orchestra.run.smithi016.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 482 kB/s | 77 kB 00:00 2024-04-23T19:08:23.688 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:08:23.754 INFO:teuthology.orchestra.run.smithi016.stdout:(2/2): ceph-volume-19.0.0-3101.g5bf3b3a9.el9.no 1.1 MB/s | 273 kB 00:00 2024-04-23T19:08:23.754 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:23.754 INFO:teuthology.orchestra.run.smithi016.stdout:Total 944 kB/s | 350 kB 00:00 2024-04-23T19:08:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:23.769 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T19:08:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:23.813 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T19:08:23.918 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T19:08:23.949 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:23.957 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T19:08:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T19:08:24.142 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T19:08:24.171 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T19:08:24.188 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2/2 2024-04-23T19:08:24.204 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2/2 2024-04-23T19:08:24.204 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:08:24.205 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T19:08:24.205 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:24.250 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T19:08:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T19:08:24.979 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/2 2024-04-23T19:08:25.055 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T19:08:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T19:08:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:25.279 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:08:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T19:08:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:25.279 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:25.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados-devel 2024-04-23T19:08:26.117 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:31 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:26.242 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:26.243 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 127 k 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 127 k 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 456 k 2024-04-23T19:08:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:26.357 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T19:08:26.450 INFO:teuthology.orchestra.run.smithi016.stdout:librados-devel-19.0.0-3101.g5bf3b3a9.el9.x86_64 613 kB/s | 127 kB 00:00 2024-04-23T19:08:26.451 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:26.451 INFO:teuthology.orchestra.run.smithi016.stdout:Total 607 kB/s | 127 kB 00:00 2024-04-23T19:08:26.452 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:26.458 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:26.458 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:26.528 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:26.528 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:26.807 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:27.257 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:27.432 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T19:08:27.462 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b 12/12 2024-04-23T19:08:27.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:27.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:27.494 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:27.494 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:08:27.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:27.494 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:27.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs2 2024-04-23T19:08:28.293 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:33 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:28.375 INFO:teuthology.orchestra.run.smithi016.stdout:Package libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:08:28.419 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:28.420 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:08:28.420 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:28.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs-devel 2024-04-23T19:08:29.026 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:34 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:29.150 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:29.152 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:29.152 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-23T19:08:29.152 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:29.152 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:29.153 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 32 k 2024-04-23T19:08:29.153 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:29.153 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:29.153 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:29.153 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-23T19:08:29.153 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:29.153 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 32 k 2024-04-23T19:08:29.154 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 142 k 2024-04-23T19:08:29.154 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:29.321 INFO:teuthology.orchestra.run.smithi016.stdout:libcephfs-devel-19.0.0-3101.g5bf3b3a9.el9.x86_6 188 kB/s | 32 kB 00:00 2024-04-23T19:08:29.322 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:29.322 INFO:teuthology.orchestra.run.smithi016.stdout:Total 186 kB/s | 32 kB 00:00 2024-04-23T19:08:29.322 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:29.326 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:29.326 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:29.328 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:29.329 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:29.410 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:29.884 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:30.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:30.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:30.085 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:30.085 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:08:30.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:30.086 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b 12/12 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b 1/12 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T19:08:30.143 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T19:08:30.144 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T19:08:30.144 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T19:08:30.144 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T19:08:30.144 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T19:08:30.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados2 2024-04-23T19:08:30.515 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T19:08:30.515 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:30.515 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:08:30.515 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:08:30.515 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T19:08:30.515 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:30.516 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:08:30.792 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:35 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:30.873 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:08:30.894 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-rook 2024-04-23T19:08:30.917 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:30.918 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:08:30.919 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:30.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librbd1 2024-04-23T19:08:31.428 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:10 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:08:31.529 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:36 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:31.552 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 49 k 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T19:08:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout:Install 13 Packages 2024-04-23T19:08:31.555 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:31.556 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 2.2 M 2024-04-23T19:08:31.556 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 26 M 2024-04-23T19:08:31.556 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:08:31.611 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:08:31.654 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:31.655 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:08:31.656 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:31.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rados 2024-04-23T19:08:32.094 INFO:teuthology.orchestra.run.smithi063.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 241 kB/s | 26 kB 00:00 2024-04-23T19:08:32.120 INFO:teuthology.orchestra.run.smithi063.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 146 kB/s | 19 kB 00:00 2024-04-23T19:08:32.179 INFO:teuthology.orchestra.run.smithi063.stdout:(3/13): ceph-mgr-rook-19.0.0-3101.g5bf3b3a9.el9 259 kB/s | 49 kB 00:00 2024-04-23T19:08:32.245 INFO:teuthology.orchestra.run.smithi063.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-04-23T19:08:32.255 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:37 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:32.270 INFO:teuthology.orchestra.run.smithi063.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-23T19:08:32.304 INFO:teuthology.orchestra.run.smithi063.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-04-23T19:08:32.335 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:08:32.378 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:32.379 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:08:32.379 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:32.421 INFO:teuthology.orchestra.run.smithi063.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-04-23T19:08:32.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rgw 2024-04-23T19:08:32.488 INFO:teuthology.orchestra.run.smithi063.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 82 kB/s | 15 kB 00:00 2024-04-23T19:08:32.513 INFO:teuthology.orchestra.run.smithi063.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 133 kB/s | 32 kB 00:00 2024-04-23T19:08:32.580 INFO:teuthology.orchestra.run.smithi063.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 887 kB/s | 59 kB 00:00 2024-04-23T19:08:32.656 INFO:teuthology.orchestra.run.smithi063.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-04-23T19:08:32.697 INFO:teuthology.orchestra.run.smithi063.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 748 kB/s | 206 kB 00:00 2024-04-23T19:08:32.823 INFO:teuthology.orchestra.run.smithi063.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.1 MB/s | 1.0 MB 00:00 2024-04-23T19:08:32.825 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:32.825 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-04-23T19:08:32.891 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:08:32.907 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:08:32.907 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:08:32.972 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:37 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:32.998 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:08:33.000 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:08:33.053 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:08:33.097 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:33.098 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:08:33.098 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:33.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-cephfs 2024-04-23T19:08:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:08:33.694 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T19:08:33.703 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:38 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:33.758 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T19:08:33.784 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:08:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T19:08:33.828 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:33.829 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:08:33.829 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T19:08:33.900 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T19:08:33.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rbd 2024-04-23T19:08:34.107 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T19:08:34.222 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-23T19:08:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T19:08:34.438 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:39 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:34.520 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:08:34.563 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:34.564 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-23T19:08:34.564 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:34.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-fuse 2024-04-23T19:08:35.043 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T19:08:35.092 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T19:08:35.126 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T19:08:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:40 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:35.175 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T19:08:35.204 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 13/13 2024-04-23T19:08:35.299 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 85 k 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 85 k 2024-04-23T19:08:35.301 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 231 k 2024-04-23T19:08:35.301 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:35.524 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-fuse-19.0.0-3101.g5bf3b3a9.el9.x86_64.rpm 381 kB/s | 85 kB 00:00 2024-04-23T19:08:35.525 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:35.525 INFO:teuthology.orchestra.run.smithi016.stdout:Total 379 kB/s | 85 kB 00:00 2024-04-23T19:08:35.525 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:35.533 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:35.533 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:35.599 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:35.600 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:35.768 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:35.881 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:36.121 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 13/13 2024-04-23T19:08:36.121 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/13 2024-04-23T19:08:36.121 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T19:08:36.121 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T19:08:36.121 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T19:08:36.122 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T19:08:36.217 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:36.434 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T19:08:36.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T19:08:36.436 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T19:08:36.436 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T19:08:36.436 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:36.436 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:08:36.453 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:36.453 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:36.453 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:36.453 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:08:36.453 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:36.453 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:36.665 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T19:08:36.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-mirror 2024-04-23T19:08:37.196 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:16 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:08:37.287 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:42 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:37.320 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 142 k 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T19:08:37.321 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T19:08:37.322 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T19:08:37.322 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T19:08:37.322 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:37.322 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:08:37.322 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:37.322 INFO:teuthology.orchestra.run.smithi063.stdout:Install 6 Packages 2024-04-23T19:08:37.322 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:37.323 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 7.0 M 2024-04-23T19:08:37.323 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 33 M 2024-04-23T19:08:37.323 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:08:37.418 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:37.419 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:37.419 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-23T19:08:37.419 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:37.419 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:37.419 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 3.1 M 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 3.1 M 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 13 M 2024-04-23T19:08:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:37.795 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-mirror-19.0.0-3101.g5bf3b3a9.el9.x86_64.rpm 8.2 MB/s | 3.1 MB 00:00 2024-04-23T19:08:37.796 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:Total 8.2 MB/s | 3.1 MB 00:00 2024-04-23T19:08:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:37.891 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:37.892 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:38.060 INFO:teuthology.orchestra.run.smithi063.stdout:(1/6): ceph-mgr-cephadm-19.0.0-3101.g5bf3b3a9.e 456 kB/s | 142 kB 00:00 2024-04-23T19:08:38.268 INFO:teuthology.orchestra.run.smithi063.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.6 MB/s | 548 kB 00:00 2024-04-23T19:08:38.310 INFO:teuthology.orchestra.run.smithi063.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.3 MB/s | 58 kB 00:00 2024-04-23T19:08:38.335 INFO:teuthology.orchestra.run.smithi063.stdout:(4/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 422 kB/s | 248 kB 00:00 2024-04-23T19:08:38.342 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:38.360 INFO:teuthology.orchestra.run.smithi063.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-04-23T19:08:38.396 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:38.421 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:38.421 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:08:38.421 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T19:08:38.421 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T19:08:38.421 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T19:08:38.422 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:39.294 INFO:teuthology.orchestra.run.smithi063.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-04-23T19:08:39.296 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:39.296 INFO:teuthology.orchestra.run.smithi063.stdout:Total 3.5 MB/s | 7.0 MB 00:01 2024-04-23T19:08:39.370 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:08:39.382 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:08:39.382 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:08:39.414 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:39.414 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:39.415 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:08:39.415 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:39.486 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:08:39.486 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:08:39.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-nbd 2024-04-23T19:08:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:08:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T19:08:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T19:08:40.264 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:45 ago on Tue 23 Apr 2024 07:05:55 PM UTC. 2024-04-23T19:08:40.388 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-23T19:08:40.389 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:40.389 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-23T19:08:40.389 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:40.389 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 170 k 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 170 k 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 494 k 2024-04-23T19:08:40.390 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-23T19:08:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T19:08:40.606 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-nbd-19.0.0-3101.g5bf3b3a9.el9.x86_64.rpm 789 kB/s | 170 kB 00:00 2024-04-23T19:08:40.607 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:40.607 INFO:teuthology.orchestra.run.smithi016.stdout:Total 782 kB/s | 170 kB 00:00 2024-04-23T19:08:40.608 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-23T19:08:40.616 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-23T19:08:40.616 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-23T19:08:40.677 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T19:08:40.683 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-23T19:08:40.683 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-23T19:08:40.748 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T19:08:40.778 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 6/6 2024-04-23T19:08:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-23T19:08:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:41.325 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:41.498 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 6/6 2024-04-23T19:08:41.499 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/6 2024-04-23T19:08:41.499 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T19:08:41.499 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T19:08:41.499 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T19:08:41.499 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T19:08:41.550 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:08:41.550 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:41.550 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-23T19:08:41.550 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:08:41.551 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:08:41.551 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T19:08:41.753 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T19:08:41.754 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T19:08:41.754 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:41.754 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:08:41.813 DEBUG:teuthology.parallel:result is None 2024-04-23T19:08:41.996 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-fuse 2024-04-23T19:08:42.532 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:21 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:08:42.659 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:08:42.659 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:42.659 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 891 k 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout:Install 2 Packages 2024-04-23T19:08:42.660 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:08:42.661 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 971 k 2024-04-23T19:08:42.661 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 2.8 M 2024-04-23T19:08:42.661 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:08:59.071 INFO:teuthology.orchestra.run.smithi063.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 85 kB/s | 80 kB 00:00 2024-04-23T19:08:59.103 INFO:teuthology.orchestra.run.smithi063.stdout:(2/2): ceph-fuse-19.0.0-3101.g5bf3b3a9.el9.x86_ 907 kB/s | 891 kB 00:00 2024-04-23T19:08:59.104 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:08:59.105 INFO:teuthology.orchestra.run.smithi063.stdout:Total 59 kB/s | 971 kB 00:16 2024-04-23T19:08:59.119 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:08:59.146 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:08:59.146 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:08:59.234 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:08:59.234 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:08:59.425 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:08:59.577 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T19:08:59.638 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/2 2024-04-23T19:09:00.321 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2/2 2024-04-23T19:09:00.321 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/2 2024-04-23T19:09:00.584 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T19:09:00.584 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:00.585 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:09:00.585 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T19:09:00.585 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:00.585 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:00.843 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-volume 2024-04-23T19:09:01.379 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:40 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:01.508 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:01.509 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume noarch 2:19.0.0-3101.g5bf3b3a9.el9 ceph-noarch 273 k 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:Install 2 Packages 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 350 k 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 1.5 M 2024-04-23T19:09:01.510 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:09:01.759 INFO:teuthology.orchestra.run.smithi063.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 698 kB/s | 77 kB 00:00 2024-04-23T19:09:03.210 INFO:teuthology.orchestra.run.smithi063.stdout:(2/2): ceph-volume-19.0.0-3101.g5bf3b3a9.el9.no 175 kB/s | 273 kB 00:01 2024-04-23T19:09:03.210 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:09:03.210 INFO:teuthology.orchestra.run.smithi063.stdout:Total 206 kB/s | 350 kB 00:01 2024-04-23T19:09:03.222 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:09:03.233 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:09:03.233 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:09:03.261 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:09:03.262 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:09:03.409 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:09:03.607 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T19:09:03.654 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2/2 2024-04-23T19:09:03.671 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2/2 2024-04-23T19:09:03.671 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:09:03.671 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T19:09:03.671 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:04.586 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 1/2 2024-04-23T19:09:04.899 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T19:09:04.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:04.900 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:09:04.900 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume-2:19.0.0-3101.g5bf3b3a9.el9.noarch 2024-04-23T19:09:04.900 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T19:09:04.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:04.900 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:05.193 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados-devel 2024-04-23T19:09:05.730 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:44 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:05.855 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 127 k 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 127 k 2024-04-23T19:09:05.857 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 456 k 2024-04-23T19:09:05.857 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:09:06.082 INFO:teuthology.orchestra.run.smithi063.stdout:librados-devel-19.0.0-3101.g5bf3b3a9.el9.x86_64 561 kB/s | 127 kB 00:00 2024-04-23T19:09:06.083 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:09:06.083 INFO:teuthology.orchestra.run.smithi063.stdout:Total 558 kB/s | 127 kB 00:00 2024-04-23T19:09:06.083 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:09:06.089 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:09:06.089 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:09:06.156 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:09:06.156 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:09:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:09:06.414 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:06.769 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:06.998 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:06.999 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:06.999 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:09:06.999 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:09:06.999 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:06.999 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:07.335 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs2 2024-04-23T19:09:07.874 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:46 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:07.955 INFO:teuthology.orchestra.run.smithi063.stdout:Package libcephfs2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:09:08.000 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:08.001 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:09:08.001 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:08.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs-devel 2024-04-23T19:09:08.593 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:47 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:08.717 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 32 k 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-23T19:09:08.718 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:08.719 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 32 k 2024-04-23T19:09:08.719 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 142 k 2024-04-23T19:09:08.719 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:09:09.062 INFO:teuthology.orchestra.run.smithi063.stdout:libcephfs-devel-19.0.0-3101.g5bf3b3a9.el9.x86_6 93 kB/s | 32 kB 00:00 2024-04-23T19:09:09.062 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:09:09.062 INFO:teuthology.orchestra.run.smithi063.stdout:Total 92 kB/s | 32 kB 00:00 2024-04-23T19:09:09.062 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:09:09.066 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:09:09.066 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:09:09.068 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:09:09.069 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:09:09.161 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:09:09.268 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:09.746 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:09.947 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:09:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:09:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:09.947 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:10.127 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados2 2024-04-23T19:09:10.663 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:49 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:10.744 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:09:10.788 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:10.789 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:09:10.789 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:10.852 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librbd1 2024-04-23T19:09:11.387 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:50 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:11.469 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:09:11.513 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:11.514 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:09:11.514 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:11.580 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rados 2024-04-23T19:09:12.112 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:51 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:12.194 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rados-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:09:12.237 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:12.238 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:09:12.239 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:12.301 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rgw 2024-04-23T19:09:12.839 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:51 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:12.919 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rgw-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:09:12.962 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:12.963 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:09:12.963 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:13.027 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-cephfs 2024-04-23T19:09:13.557 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:52 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:13.640 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-cephfs-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:09:13.683 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:13.684 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:09:13.685 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:13.743 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rbd 2024-04-23T19:09:14.278 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:53 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:14.359 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 is already installed. 2024-04-23T19:09:14.402 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:14.403 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-04-23T19:09:14.403 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:14.465 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-fuse 2024-04-23T19:09:15.002 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:54 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:15.128 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:15.128 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:15.128 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 85 k 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-23T19:09:15.129 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:15.130 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 85 k 2024-04-23T19:09:15.130 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 231 k 2024-04-23T19:09:15.130 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:09:15.481 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-fuse-19.0.0-3101.g5bf3b3a9.el9.x86_64.rpm 243 kB/s | 85 kB 00:00 2024-04-23T19:09:15.482 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:09:15.482 INFO:teuthology.orchestra.run.smithi063.stdout:Total 242 kB/s | 85 kB 00:00 2024-04-23T19:09:15.482 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:09:15.490 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:09:15.490 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:09:15.557 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:09:15.558 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:09:15.703 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:09:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:16.261 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:16.498 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:09:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:09:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:16.499 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:16.761 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-mirror 2024-04-23T19:09:17.305 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:56 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:17.431 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:17.432 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:17.432 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-04-23T19:09:17.432 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:17.432 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:09:17.432 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 3.1 M 2024-04-23T19:09:17.432 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:17.433 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:09:17.433 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:17.433 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-23T19:09:17.433 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:17.433 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 3.1 M 2024-04-23T19:09:17.433 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 13 M 2024-04-23T19:09:17.433 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:09:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-mirror-19.0.0-3101.g5bf3b3a9.el9.x86_64.rpm 4.3 MB/s | 3.1 MB 00:00 2024-04-23T19:09:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:09:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.2 MB/s | 3.1 MB 00:00 2024-04-23T19:09:18.161 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:09:18.171 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:09:18.171 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:09:18.257 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:09:18.257 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:09:18.709 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:09:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:18.828 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T19:09:18.828 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T19:09:18.829 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T19:09:18.829 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T19:09:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:19.932 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:19.932 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:19.932 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:09:19.932 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:09:19.932 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:19.933 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:20.183 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-nbd 2024-04-23T19:09:20.727 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:02:59 ago on Tue 23 Apr 2024 07:06:21 PM UTC. 2024-04-23T19:09:20.852 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-04-23T19:09:20.853 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-04-23T19:09:20.854 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:20.854 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-04-23T19:09:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:19.0.0-3101.g5bf3b3a9.el9 ceph 170 k 2024-04-23T19:09:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:20.855 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-04-23T19:09:20.855 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-04-23T19:09:20.855 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-04-23T19:09:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:20.855 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 170 k 2024-04-23T19:09:20.856 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 494 k 2024-04-23T19:09:20.856 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-04-23T19:09:21.480 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-nbd-19.0.0-3101.g5bf3b3a9.el9.x86_64.rpm 272 kB/s | 170 kB 00:00 2024-04-23T19:09:21.480 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-04-23T19:09:21.481 INFO:teuthology.orchestra.run.smithi063.stdout:Total 271 kB/s | 170 kB 00:00 2024-04-23T19:09:21.481 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-04-23T19:09:21.491 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-04-23T19:09:21.491 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-04-23T19:09:21.560 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-04-23T19:09:21.561 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-04-23T19:09:21.723 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-04-23T19:09:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:22.218 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:22.465 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 1/1 2024-04-23T19:09:22.465 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:22.465 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-04-23T19:09:22.466 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:19.0.0-3101.g5bf3b3a9.el9.x86_64 2024-04-23T19:09:22.466 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:22.466 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-04-23T19:09:22.701 DEBUG:teuthology.parallel:result is None 2024-04-23T19:09:22.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5bf3b3a9b161d70b9aa7806055bdae202e03111c 2024-04-23T19:09:22.845 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T19:09:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-3101.g5bf3b3a9.el9 2024-04-23T19:09:22.872 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g5bf3b3a9.el9 2024-04-23T19:09:22.872 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g5bf3b3a9 is installed. 2024-04-23T19:09:22.874 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5bf3b3a9b161d70b9aa7806055bdae202e03111c 2024-04-23T19:09:23.169 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T19:09:23.195 INFO:teuthology.orchestra.run.smithi063.stdout:19.0.0-3101.g5bf3b3a9.el9 2024-04-23T19:09:23.195 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g5bf3b3a9.el9 2024-04-23T19:09:23.195 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g5bf3b3a9 is installed. 2024-04-23T19:09:23.197 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T19:09:23.197 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:23.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T19:09:23.228 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:23.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T19:09:23.263 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T19:09:23.263 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:23.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T19:09:23.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T19:09:23.365 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:23.365 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T19:09:23.393 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T19:09:23.488 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T19:09:23.488 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:23.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T19:09:23.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T19:09:23.587 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:23.588 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T19:09:23.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T19:09:23.683 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T19:09:23.683 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:23.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T19:09:23.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T19:09:23.773 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:23.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T19:09:23.800 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T19:09:23.867 INFO:teuthology.run_tasks:Running task ceph... 2024-04-23T19:09:23.958 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-23T19:09:23.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2024-04-23T19:09:23.961 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /var/log/ceph 2024-04-23T19:09:23.987 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-23T19:09:23.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T19:09:24.025 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T19:09:24.053 INFO:tasks.ceph:Creating extra log directories... 2024-04-23T19:09:24.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T19:09:24.089 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T19:09:24.127 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-23T19:09:24.127 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-23T19:09:24.127 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982', 'branch': 'rgw-update-boost-redis', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}}', 'email': None, 'first_in_suite': False, 'job_id': '7669982', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rgw-update-boost-redis'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c'}}, 'rgw': {'ec-data-pool': True, 'frontend': 'beast'}, 's3tests': {'slow_backend': True}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rgw-update-boost-redis', 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c'}}, 'owner': 'scheduled_suriarte@teuthology', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'rgw-update-boost-redis', 'suite_path': '/home/teuthworker/src/github.com_samarahu_ceph_5bf3b3a9b161d70b9aa7806055bdae202e03111c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/samarahu/ceph.git', 'suite_sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c', 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM5lFx6tyKbqLCCgBvx+a5yQJqhW6016rUZ7u/MxHiLyg08MFSLxgKNABTMQppRdYvMCQ3nCA9rXfxmUtkW9BVM=', 'smithi063.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHI5yAFXtJ66L36Lk3+SgnZX0DQj4WS1jlVZP19Redn2QimOalffOT6U6/o6+b9Zv36tqndX4zx3eXzrI0n2Y/k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '5bf3b3a9b161d70b9aa7806055bdae202e03111c', 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_15:04:03', 'tube': 'smithi', 'user': 'suriarte', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-23T19:09:24.128 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T19:09:24.153 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T19:09:24.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T19:09:24.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T19:09:24.242 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:24.242 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T19:09:24.287 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T19:09:24.288 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-23 19:08:38.662392117 +0000 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-23 19:05:01.560259130 +0000 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-23 19:05:01.560259130 +0000 2024-04-23T19:09:24.343 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-23 19:05:01.560259130 +0000 2024-04-23T19:09:24.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T19:09:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-23T19:09:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-23T19:09:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.9252e-05 s, 5.2 MB/s 2024-04-23T19:09:24.407 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T19:09:24.463 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-23 19:08:38.662392117 +0000 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-23 19:05:01.908253586 +0000 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-23 19:05:01.908253586 +0000 2024-04-23T19:09:24.518 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-23 19:05:01.908253586 +0000 2024-04-23T19:09:24.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T19:09:24.581 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-23T19:09:24.581 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-23T19:09:24.581 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.7618e-05 s, 5.2 MB/s 2024-04-23T19:09:24.582 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T19:09:24.637 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-04-23T19:09:24.693 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T19:09:24.693 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:24.693 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-23T19:09:24.693 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:24.693 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:24.693 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-23 19:08:38.663392101 +0000 2024-04-23T19:09:24.693 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-23 19:05:02.258248010 +0000 2024-04-23T19:09:24.694 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-23 19:05:02.258248010 +0000 2024-04-23T19:09:24.694 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-23 19:05:02.258248010 +0000 2024-04-23T19:09:24.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T19:09:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-23T19:09:24.756 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-23T19:09:24.756 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000104169 s, 4.9 MB/s 2024-04-23T19:09:24.757 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T19:09:24.814 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-04-23T19:09:24.869 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T19:09:24.869 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:24.869 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-23T19:09:24.870 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:24.870 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:24.870 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-23 19:08:38.663392101 +0000 2024-04-23T19:09:24.870 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-23 19:05:02.589242736 +0000 2024-04-23T19:09:24.870 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-23 19:05:02.589242736 +0000 2024-04-23T19:09:24.870 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-23 19:05:02.589242736 +0000 2024-04-23T19:09:24.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T19:09:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-23T19:09:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-23T19:09:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 8.7877e-05 s, 5.8 MB/s 2024-04-23T19:09:24.934 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T19:09:24.990 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-23T19:09:24.991 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:24.991 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T19:09:25.010 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T19:09:25.010 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-04-23T19:09:25.069 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T19:09:25.069 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:25.069 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-23T19:09:25.069 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:25.070 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:25.070 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-23 19:09:19.102875599 +0000 2024-04-23T19:09:25.070 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-23 19:05:17.626299660 +0000 2024-04-23T19:09:25.070 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-23 19:05:17.626299660 +0000 2024-04-23T19:09:25.070 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-23 19:05:17.626299660 +0000 2024-04-23T19:09:25.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T19:09:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-23T19:09:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-23T19:09:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000216419 s, 2.4 MB/s 2024-04-23T19:09:25.138 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T19:09:25.198 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-23 19:09:19.103875590 +0000 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-23 19:05:17.979296295 +0000 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-23 19:05:17.979296295 +0000 2024-04-23T19:09:25.256 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-23 19:05:17.979296295 +0000 2024-04-23T19:09:25.257 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T19:09:25.322 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-23T19:09:25.322 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-23T19:09:25.322 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000263609 s, 1.9 MB/s 2024-04-23T19:09:25.324 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T19:09:25.382 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-04-23T19:09:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T19:09:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:25.439 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T19:09:25.439 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:25.439 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:25.440 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-23 19:09:19.103875590 +0000 2024-04-23T19:09:25.440 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-23 19:05:18.339292863 +0000 2024-04-23T19:09:25.440 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-23 19:05:18.339292863 +0000 2024-04-23T19:09:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-23 19:05:18.339292863 +0000 2024-04-23T19:09:25.440 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T19:09:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-23T19:09:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-23T19:09:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000193502 s, 2.6 MB/s 2024-04-23T19:09:25.507 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T19:09:25.566 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-04-23T19:09:25.623 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T19:09:25.623 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T19:09:25.624 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-23T19:09:25.624 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T19:09:25.624 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T19:09:25.624 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-04-23 19:09:19.104875580 +0000 2024-04-23T19:09:25.624 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-04-23 19:05:18.709289336 +0000 2024-04-23T19:09:25.624 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-04-23 19:05:18.709289336 +0000 2024-04-23T19:09:25.624 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-04-23 19:05:18.709289336 +0000 2024-04-23T19:09:25.624 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T19:09:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-04-23T19:09:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-04-23T19:09:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000188954 s, 2.7 MB/s 2024-04-23T19:09:25.690 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T19:09:25.749 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-23T19:09:25.749 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-23T19:09:25.749 INFO:tasks.ceph:Generating config... 2024-04-23T19:09:25.751 INFO:tasks.ceph:[client] debug ms = 1 2024-04-23T19:09:25.752 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-04-23T19:09:25.752 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-23T19:09:25.752 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-23T19:09:25.752 INFO:tasks.ceph:[client] setuser = ceph 2024-04-23T19:09:25.752 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-23T19:09:25.752 INFO:tasks.ceph:[osd] debug objclass = 20 2024-04-23T19:09:25.752 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-23T19:09:25.752 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-23T19:09:25.753 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-23T19:09:25.754 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-23T19:09:25.754 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-23T19:09:25.754 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-23T19:09:25.754 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-23T19:09:25.754 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-23T19:09:25.754 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-23T19:09:25.754 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-23T19:09:25.754 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-23T19:09:25.755 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-23T19:09:25.755 INFO:tasks.ceph:Setting up mon.a... 2024-04-23T19:09:25.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-23T19:09:25.807 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2024-04-23T19:09:25.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-23T19:09:25.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T19:09:25.881 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16')] 2024-04-23T19:09:25.882 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-04-23T19:09:25.883 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:25.883 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T19:09:25.939 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:09:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:09:26.020 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:26.020 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid f80ae34c-e526-409e-90c3-45f2fcaa83f6 2024-04-23T19:09:26.020 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:fsid f80ae34c-e526-409e-90c3-45f2fcaa83f6 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2024-04-23T19:09:26.020679+0000 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-23T19:09:26.020679+0000 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 17 (quincy) 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-04-23T19:09:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-04-23T19:09:26.022 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T19:09:26.078 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f80ae34c-e526-409e-90c3-45f2fcaa83f6... 2024-04-23T19:09:26.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T19:09:26.121 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T19:09:26.162 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-04-23T19:09:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-04-23T19:09:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T19:09:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-04-23T19:09:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-04-23T19:09:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-04-23T19:09:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-04-23T19:09:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = 172.21.15.16 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = f80ae34c-e526-409e-90c3-45f2fcaa83f6 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-04-23T19:09:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug objclass = 20 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-04-23T19:09:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.169 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.170 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-04-23T19:09:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-23T19:09:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T19:09:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T19:09:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T19:09:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-04-23T19:09:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T19:09:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T19:09:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-23T19:09:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: rgw gc obj min wait = 15 2024-04-23T19:09:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: debug rgw = 20 2024-04-23T19:09:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: setgroup = ceph 2024-04-23T19:09:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: setuser = ceph 2024-04-23T19:09:26.174 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-04-23T19:09:26.175 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-04-23T19:09:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.177 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = 172.21.15.16 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = f80ae34c-e526-409e-90c3-45f2fcaa83f6 2024-04-23T19:09:26.178 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-04-23T19:09:26.179 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.180 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: debug objclass = 20 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: bdev async discard = True 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: bdev enable discard = True 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore allocator = bitmap 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 1/20 2024-04-23T19:09:26.181 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 1/20 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 4/10 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-04-23T19:09:26.182 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-04-23T19:09:26.183 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.184 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-04-23T19:09:26.185 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout: rgw gc obj min wait = 15 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout: debug rgw = 20 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout: setgroup = ceph 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout: setuser = ceph 2024-04-23T19:09:26.186 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-04-23T19:09:26.188 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-23T19:09:26.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-23T19:09:26.282 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-23T19:09:26.282 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:26.282 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-23T19:09:26.297 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:26.297 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-23T19:09:26.352 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2024-04-23T19:09:26.353 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:26.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T19:09:26.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T19:09:26.426 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:26.426 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:26.482 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:09:26.482 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:26.482 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T19:09:26.482 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T19:09:26.522 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:26.523 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:26.580 INFO:tasks.ceph:Setting up mon nodes... 2024-04-23T19:09:26.580 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-23T19:09:26.580 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T19:09:26.690 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T19:09:26.693 INFO:tasks.ceph:Setting up mds nodes... 2024-04-23T19:09:26.693 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-23T19:09:26.693 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T19:09:26.734 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-23T19:09:26.747 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-23T19:09:26.747 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-23T19:09:26.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-23T19:09:26.773 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-23T19:09:26.773 INFO:tasks.ceph:role: osd.0 2024-04-23T19:09:26.773 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-23T19:09:26.773 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T19:09:26.838 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T19:09:26.839 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T19:09:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-04-23T19:09:26.873 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-04-23T19:09:26.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-23T19:09:26.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-23T19:09:27.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:27.109+0000 7f2210e25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-23T19:09:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:27.109+0000 7f2210e25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-23T19:09:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:27.109+0000 7f2210e25640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T19:09:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:27.109+0000 7f2210e25640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T19:09:27.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:27.109+0000 7f2210e25640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T19:09:27.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:27.109+0000 7f2210e25640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-23T19:09:29.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-23T19:09:29.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-23T19:09:30.025 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-23T19:09:30.025 INFO:tasks.ceph:role: osd.1 2024-04-23T19:09:30.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:09:30.025 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T19:09:30.101 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T19:09:30.101 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T19:09:30.102 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T19:09:30.620 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-04-23T19:09:30.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-04-23T19:09:30.623 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-23T19:09:30.741 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-23T19:09:30.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-23T19:09:30.837 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-23T19:09:30.837 INFO:tasks.ceph:role: osd.2 2024-04-23T19:09:30.837 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:09:30.837 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T19:09:30.909 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T19:09:30.910 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T19:09:31.367 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-04-23T19:09:31.369 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-04-23T19:09:31.369 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-04-23T19:09:31.456 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-23T19:09:31.485 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-23T19:09:31.550 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-23T19:09:31.550 INFO:tasks.ceph:role: osd.3 2024-04-23T19:09:31.551 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:09:31.551 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T19:09:31.622 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T19:09:31.623 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T19:09:31.623 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T19:09:32.110 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-04-23T19:09:32.112 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-04-23T19:09:32.112 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-04-23T19:09:32.224 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-23T19:09:32.254 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:32.349+0000 7f9e1d36a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-23T19:09:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:32.352+0000 7f9e1d36a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-23T19:09:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:32.352+0000 7f9e1d36a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T19:09:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:32.352+0000 7f9e1d36a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T19:09:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:32.352+0000 7f9e1d36a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T19:09:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:32.356+0000 7f9e1d36a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-23T19:09:35.203 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-23T19:09:35.232 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:35.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:35.323+0000 7fad55643640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-23T19:09:35.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:35.323+0000 7fad55643640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-23T19:09:35.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:35.323+0000 7fad55643640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T19:09:35.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:35.323+0000 7fad55643640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T19:09:35.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:35.323+0000 7fad55643640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T19:09:35.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:35.324+0000 7fad55643640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-23T19:09:38.183 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-23T19:09:38.213 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:38.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:38.301+0000 7f3921f61640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-23T19:09:38.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:38.302+0000 7f3921f61640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-23T19:09:38.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:38.302+0000 7f3921f61640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T19:09:38.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:38.302+0000 7f3921f61640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T19:09:38.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:38.302+0000 7f3921f61640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T19:09:38.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:38.302+0000 7f3921f61640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-23T19:09:41.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-23T19:09:41.194 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-23T19:09:41.194 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:41.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-23T19:09:41.227 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:41.227 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-23T19:09:41.261 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:41.261 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-23T19:09:41.329 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:41.329 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-23T19:09:41.395 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:41.395 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-23T19:09:41.461 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:41.461 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-23T19:09:41.518 INFO:tasks.ceph:Adding keys to all mons... 2024-04-23T19:09:41.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T19:09:41.545 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBnByhm1AqSBhAAcWmAp+ruh2Sw6wEOPiiLhg== 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBmByhmF8EvKRAAhAsWS05tLvLBkcMDVqj9Ig== 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBsByhmU6zpFBAAy7UC24zy9KHplO2dKZQ5wg== 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBvByhmoTVZExAA4hbQJhUGXzAKqAule2lN0g== 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQByByhmxXASEhAAG3rmPBl7QKXYw/jZmtV1Ew== 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-23T19:09:41.546 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBmByhmyuTQKxAAGcZ0BxGwj05AOUbyoFiZVg== 2024-04-23T19:09:41.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T19:09:41.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T19:09:41.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T19:09:41.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T19:09:41.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T19:09:41.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T19:09:41.908 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-23T19:09:41.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-23T19:09:41.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T19:09:42.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-23T19:09:42.052 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T19:09:42.107 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-23T19:09:42.108 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-23T19:09:42.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-23T19:09:42.149 INFO:tasks.ceph.mon.a:Started 2024-04-23T19:09:42.150 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-23T19:09:42.150 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-23T19:09:42.150 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-23T19:09:42.153 INFO:tasks.ceph.mgr.x:Started 2024-04-23T19:09:42.153 INFO:tasks.ceph:Setting crush tunables to default 2024-04-23T19:09:42.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2024-04-23T19:09:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.252+0000 7fcc77ef3640 1 Processor -- start 2024-04-23T19:09:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.252+0000 7fcc77ef3640 1 -- start start 2024-04-23T19:09:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.252+0000 7fcc77ef3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc700997a0 0x7fcc70099b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.252+0000 7fcc77ef3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7009e670 con 0x7fcc7009a0c0 2024-04-23T19:09:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.252+0000 7fcc77ef3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7009b0f0 con 0x7fcc700997a0 2024-04-23T19:09:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.253+0000 7fcc75c8f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc700997a0 0x7fcc70099b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.253+0000 7fcc7548e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc7009a0c0 0x7fcc70097470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49048/0 (socket says 172.21.15.16:49048) 2024-04-23T19:09:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.253+0000 7fcc7548e640 1 -- 172.21.15.16:0/3194154615 learned_addr learned my addr 172.21.15.16:0/3194154615 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.253+0000 7fcc75c8f640 1 -- 172.21.15.16:0/3194154615 >> v1:172.21.15.16:6789/0 conn(0x7fcc7009a0c0 legacy=0x7fcc70097470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.253+0000 7fcc75c8f640 1 -- 172.21.15.16:0/3194154615 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7009b400 con 0x7fcc700997a0 2024-04-23T19:09:42.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.254+0000 7fcc75c8f640 1 --2- 172.21.15.16:0/3194154615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc70099b80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009720 tx=0x7fcc6c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3421c8725b81916 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc74c8d640 1 -- 172.21.15.16:0/3194154615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc6c0316d0 con 0x7fcc700997a0 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc74c8d640 1 -- 172.21.15.16:0/3194154615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c037070 con 0x7fcc700997a0 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc74c8d640 1 -- 172.21.15.16:0/3194154615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc6c0319c0 con 0x7fcc700997a0 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc77ef3640 1 -- 172.21.15.16:0/3194154615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 msgr2=0x7fcc70099b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc77ef3640 1 --2- 172.21.15.16:0/3194154615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc70099b80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009720 tx=0x7fcc6c02b840 comp rx=0 tx=0).stop 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc77ef3640 1 -- 172.21.15.16:0/3194154615 shutdown_connections 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc77ef3640 1 --2- 172.21.15.16:0/3194154615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc70099b80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc77ef3640 1 -- 172.21.15.16:0/3194154615 >> 172.21.15.16:0/3194154615 conn(0x7fcc700a1850 msgr2=0x7fcc7009dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc77ef3640 1 -- 172.21.15.16:0/3194154615 shutdown_connections 2024-04-23T19:09:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.255+0000 7fcc77ef3640 1 -- 172.21.15.16:0/3194154615 wait complete. 2024-04-23T19:09:42.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.256+0000 7fcc77ef3640 1 Processor -- start 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.256+0000 7fcc77ef3640 1 -- start start 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc77ef3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc77ef3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7009a9e0 con 0x7fcc700997a0 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc75c8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc75c8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc7006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42722/0 (socket says 172.21.15.16:42722) 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc75c8f640 1 -- 172.21.15.16:0/1543160406 learned_addr learned my addr 172.21.15.16:0/1543160406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc75c8f640 1 -- 172.21.15.16:0/1543160406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc700711b0 con 0x7fcc700997a0 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc75c8f640 1 --2- 172.21.15.16:0/1543160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc7006edb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009720 tx=0x7fcc6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc667fc640 1 -- 172.21.15.16:0/1543160406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc6c032770 con 0x7fcc700997a0 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc667fc640 1 -- 172.21.15.16:0/1543160406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c032910 con 0x7fcc700997a0 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc70070540 con 0x7fcc700997a0 2024-04-23T19:09:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc667fc640 1 -- 172.21.15.16:0/1543160406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc6c032bf0 con 0x7fcc700997a0 2024-04-23T19:09:42.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.257+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc700718b0 con 0x7fcc700997a0 2024-04-23T19:09:42.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.258+0000 7fcc667fc640 1 -- 172.21.15.16:0/1543160406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc6c04c060 con 0x7fcc700997a0 2024-04-23T19:09:42.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.258+0000 7fcc667fc640 1 -- 172.21.15.16:0/1543160406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc6c02f050 con 0x7fcc700997a0 2024-04-23T19:09:42.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.258+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc70063660 con 0x7fcc700997a0 2024-04-23T19:09:42.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.260+0000 7fcc667fc640 1 -- 172.21.15.16:0/1543160406 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcc700711b0 con 0x7fcc700997a0 2024-04-23T19:09:42.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.327+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcc7009b670 con 0x7fcc700997a0 2024-04-23T19:09:42.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.327+0000 7fcc667fc640 1 -- 172.21.15.16:0/1543160406 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fcc6c053090 con 0x7fcc700997a0 2024-04-23T19:09:42.329 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-04-23T19:09:42.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.328+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 msgr2=0x7fcc7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.328+0000 7fcc77ef3640 1 --2- 172.21.15.16:0/1543160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc7006edb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009720 tx=0x7fcc6c0046c0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.328+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 shutdown_connections 2024-04-23T19:09:42.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.328+0000 7fcc77ef3640 1 --2- 172.21.15.16:0/1543160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc700997a0 0x7fcc7006edb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.328+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 >> 172.21.15.16:0/1543160406 conn(0x7fcc700a1850 msgr2=0x7fcc7012e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:42.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.328+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 shutdown_connections 2024-04-23T19:09:42.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:42.328+0000 7fcc77ef3640 1 -- 172.21.15.16:0/1543160406 wait complete. 2024-04-23T19:09:42.339 INFO:tasks.ceph:check_enable_crimson: False 2024-04-23T19:09:42.339 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-23T19:09:42.339 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-23T19:09:42.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-23T19:09:42.363 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:42.364 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-23T19:09:42.389 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:42.389 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-23T19:09:42.453 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-04-23T19:09:42.453 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-23T19:09:42.518 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new c8925ca3-b81e-4e90-9554-7a26a6b9ee13 0 2024-04-23T19:09:42.538 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:42.537+0000 7fa616842040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T19:09:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.637+0000 7f1d3517e640 1 Processor -- start 2024-04-23T19:09:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.637+0000 7f1d3517e640 1 -- start start 2024-04-23T19:09:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.637+0000 7f1d3517e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d30119000 0x7f1d3011b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.637+0000 7f1d3517e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d3011b9c0 con 0x7f1d30116270 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.637+0000 7f1d3517e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3011cba0 con 0x7f1d30119000 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.638+0000 7f1d2e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d30119000 0x7f1d3011b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.638+0000 7f1d2e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d30119000 0x7f1d3011b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48852/0 (socket says 172.21.15.63:48852) 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.638+0000 7f1d2e59c640 1 -- 172.21.15.63:0/3255435820 learned_addr learned my addr 172.21.15.63:0/3255435820 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.638+0000 7f1d2ed9d640 1 --1- 172.21.15.63:0/3255435820 >> v1:172.21.15.16:6789/0 conn(0x7f1d30116270 0x7f1d30116650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:54298/0 (socket says 172.21.15.63:54298) 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.638+0000 7f1d2e59c640 1 -- 172.21.15.63:0/3255435820 >> v1:172.21.15.16:6789/0 conn(0x7f1d30116270 legacy=0x7f1d30116650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.638+0000 7f1d2e59c640 1 -- 172.21.15.63:0/3255435820 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3011cd20 con 0x7f1d30119000 2024-04-23T19:09:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.638+0000 7f1d2dd9b640 1 -- 172.21.15.63:0/3255435820 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023918875 0 0) 0x7f1d3011b9c0 con 0x7f1d30116270 2024-04-23T19:09:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.639+0000 7f1d2e59c640 1 --2- 172.21.15.63:0/3255435820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30119000 0x7f1d3011b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1d200097f0 tx=0x7f1d2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b936891971ee790b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.639+0000 7f1d2dd9b640 1 -- 172.21.15.63:0/3255435820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d20035020 con 0x7f1d30119000 2024-04-23T19:09:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.639+0000 7f1d2dd9b640 1 -- 172.21.15.63:0/3255435820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d200317a0 con 0x7f1d30119000 2024-04-23T19:09:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.639+0000 7f1d2dd9b640 1 -- 172.21.15.63:0/3255435820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d20031aa0 con 0x7f1d30119000 2024-04-23T19:09:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.640+0000 7f1d3517e640 1 -- 172.21.15.63:0/3255435820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30119000 msgr2=0x7f1d3011b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.640+0000 7f1d3517e640 1 --2- 172.21.15.63:0/3255435820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30119000 0x7f1d3011b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1d200097f0 tx=0x7f1d2002b910 comp rx=0 tx=0).stop 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.640+0000 7f1d3517e640 1 -- 172.21.15.63:0/3255435820 shutdown_connections 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.640+0000 7f1d3517e640 1 --2- 172.21.15.63:0/3255435820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30119000 0x7f1d3011b3f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.640+0000 7f1d3517e640 1 -- 172.21.15.63:0/3255435820 >> 172.21.15.63:0/3255435820 conn(0x7f1d30070ed0 msgr2=0x7f1d300732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.640+0000 7f1d3517e640 1 -- 172.21.15.63:0/3255435820 shutdown_connections 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.640+0000 7f1d3517e640 1 -- 172.21.15.63:0/3255435820 wait complete. 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d3517e640 1 Processor -- start 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d3517e640 1 -- start start 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d3517e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30116270 0x7f1d301a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d3517e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3011c0b0 con 0x7f1d30116270 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d2ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30116270 0x7f1d301a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d2ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30116270 0x7f1d301a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48866/0 (socket says 172.21.15.63:48866) 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d2ed9d640 1 -- 172.21.15.63:0/1313206903 learned_addr learned my addr 172.21.15.63:0/1313206903 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.641+0000 7f1d2ed9d640 1 -- 172.21.15.63:0/1313206903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d301a52a0 con 0x7f1d30116270 2024-04-23T19:09:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d2ed9d640 1 --2- 172.21.15.63:0/1313206903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30116270 0x7f1d301a4d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400c420 tx=0x7f1d2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d177fe640 1 -- 172.21.15.63:0/1313206903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d24018070 con 0x7f1d30116270 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d177fe640 1 -- 172.21.15.63:0/1313206903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d240074d0 con 0x7f1d30116270 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d177fe640 1 -- 172.21.15.63:0/1313206903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d24004950 con 0x7f1d30116270 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d301a5590 con 0x7f1d30116270 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a9580 con 0x7f1d30116270 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d177fe640 1 -- 172.21.15.63:0/1313206903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1d2401b070 con 0x7f1d30116270 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.642+0000 7f1d177fe640 1 -- 172.21.15.63:0/1313206903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f1d2401c3b0 con 0x7f1d30116270 2024-04-23T19:09:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.643+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d301a57c0 con 0x7f1d30116270 2024-04-23T19:09:42.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.645+0000 7f1d177fe640 1 -- 172.21.15.63:0/1313206903 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1d24022020 con 0x7f1d30116270 2024-04-23T19:09:42.692 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:42.691+0000 7fa616842040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T19:09:42.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.719+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c8925ca3-b81e-4e90-9554-7a26a6b9ee13", "id": 0} v 0) -- 0x7f1d300505e0 con 0x7f1d30116270 2024-04-23T19:09:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.720+0000 7f1d177fe640 1 -- 172.21.15.63:0/1313206903 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8925ca3-b81e-4e90-9554-7a26a6b9ee13", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f1d24004af0 con 0x7f1d30116270 2024-04-23T19:09:42.721 INFO:teuthology.orchestra.run.smithi063.stdout:0 2024-04-23T19:09:42.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.721+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30116270 msgr2=0x7f1d301a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.721+0000 7f1d3517e640 1 --2- 172.21.15.63:0/1313206903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30116270 0x7f1d301a4d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400c420 tx=0x7f1d2400c8e0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.721+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 shutdown_connections 2024-04-23T19:09:42.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.721+0000 7f1d3517e640 1 --2- 172.21.15.63:0/1313206903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d30116270 0x7f1d301a4d60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.721+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 >> 172.21.15.63:0/1313206903 conn(0x7f1d30070ed0 msgr2=0x7f1d300732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.721+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 shutdown_connections 2024-04-23T19:09:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.721+0000 7f1d3517e640 1 -- 172.21.15.63:0/1313206903 wait complete. 2024-04-23T19:09:42.733 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 6bf01767-14ca-4771-86ec-2369b6eefd4d 1 2024-04-23T19:09:42.749 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:42.748+0000 7fa616842040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T19:09:42.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f10521a7640 1 Processor -- start 2024-04-23T19:09:42.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f10521a7640 1 -- start start 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f10521a7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f104c118460 0x7f104c118840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f10521a7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f104c118ed0 con 0x7f104c1121b0 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f10521a7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f104c112d70 con 0x7f104c118460 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f104b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f104c118460 0x7f104c118840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f104b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f104c118460 0x7f104c118840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48876/0 (socket says 172.21.15.63:48876) 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f104affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f104c1121b0 0x7f104c112660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:54308/0 (socket says 172.21.15.63:54308) 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.832+0000 7f104b7fe640 1 -- 172.21.15.63:0/652899471 learned_addr learned my addr 172.21.15.63:0/652899471 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.833+0000 7f104b7fe640 1 -- 172.21.15.63:0/652899471 >> v1:172.21.15.16:6789/0 conn(0x7f104c1121b0 legacy=0x7f104c112660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.833+0000 7f104a7fc640 1 -- 172.21.15.63:0/652899471 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3720442659 0 0) 0x7f104c118ed0 con 0x7f104c1121b0 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.833+0000 7f104b7fe640 1 -- 172.21.15.63:0/652899471 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c112f20 con 0x7f104c118460 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.833+0000 7f104b7fe640 1 --2- 172.21.15.63:0/652899471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c118460 0x7f104c118840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f103c0097f0 tx=0x7f103c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b095114152ee43c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f104a7fc640 1 -- 172.21.15.63:0/652899471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c035020 con 0x7f104c118460 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f104a7fc640 1 -- 172.21.15.63:0/652899471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f103c0317a0 con 0x7f104c118460 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f104a7fc640 1 -- 172.21.15.63:0/652899471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c031aa0 con 0x7f104c118460 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f10521a7640 1 -- 172.21.15.63:0/652899471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c118460 msgr2=0x7f104c118840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f10521a7640 1 --2- 172.21.15.63:0/652899471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c118460 0x7f104c118840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f103c0097f0 tx=0x7f103c02b910 comp rx=0 tx=0).stop 2024-04-23T19:09:42.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f10521a7640 1 -- 172.21.15.63:0/652899471 shutdown_connections 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f10521a7640 1 --2- 172.21.15.63:0/652899471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c118460 0x7f104c118840 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f10521a7640 1 -- 172.21.15.63:0/652899471 >> 172.21.15.63:0/652899471 conn(0x7f104c06d8c0 msgr2=0x7f104c06dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f10521a7640 1 -- 172.21.15.63:0/652899471 shutdown_connections 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.834+0000 7f10521a7640 1 -- 172.21.15.63:0/652899471 wait complete. 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.835+0000 7f10521a7640 1 Processor -- start 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.835+0000 7f10521a7640 1 -- start start 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.835+0000 7f10521a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c1121b0 0x7f104c1a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:42.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.835+0000 7f10521a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f104c11e6a0 con 0x7f104c1121b0 2024-04-23T19:09:42.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.836+0000 7f104b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c1121b0 0x7f104c1a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:42.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.836+0000 7f104b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c1121b0 0x7f104c1a6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48888/0 (socket says 172.21.15.63:48888) 2024-04-23T19:09:42.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.836+0000 7f104b7fe640 1 -- 172.21.15.63:0/2889383107 learned_addr learned my addr 172.21.15.63:0/2889383107 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:09:42.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.836+0000 7f104b7fe640 1 -- 172.21.15.63:0/2889383107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c1a7320 con 0x7f104c1121b0 2024-04-23T19:09:42.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.836+0000 7f104b7fe640 1 --2- 172.21.15.63:0/2889383107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c1121b0 0x7f104c1a6de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f103c000c00 tx=0x7f103c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.837+0000 7f1033fff640 1 -- 172.21.15.63:0/2889383107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c03ad00 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.837+0000 7f1033fff640 1 -- 172.21.15.63:0/2889383107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f103c0443f0 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.837+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c1a75b0 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.837+0000 7f1033fff640 1 -- 172.21.15.63:0/2889383107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c044710 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.837+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1ab450 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.837+0000 7f1033fff640 1 -- 172.21.15.63:0/2889383107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f103c04b060 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.837+0000 7f1033fff640 1 -- 172.21.15.63:0/2889383107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f103c04dcf0 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.838+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c0505e0 con 0x7f104c1121b0 2024-04-23T19:09:42.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.840+0000 7f1033fff640 1 -- 172.21.15.63:0/2889383107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f103c052020 con 0x7f104c1121b0 2024-04-23T19:09:42.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.915+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6bf01767-14ca-4771-86ec-2369b6eefd4d", "id": 1} v 0) -- 0x7f104c118460 con 0x7f104c1121b0 2024-04-23T19:09:42.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.916+0000 7f1033fff640 1 -- 172.21.15.63:0/2889383107 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6bf01767-14ca-4771-86ec-2369b6eefd4d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f103c03f070 con 0x7f104c1121b0 2024-04-23T19:09:42.917 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-04-23T19:09:42.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.916+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c1121b0 msgr2=0x7f104c1a6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:42.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.916+0000 7f10521a7640 1 --2- 172.21.15.63:0/2889383107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c1121b0 0x7f104c1a6de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f103c000c00 tx=0x7f103c0046c0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.916+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 shutdown_connections 2024-04-23T19:09:42.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.916+0000 7f10521a7640 1 --2- 172.21.15.63:0/2889383107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c1121b0 0x7f104c1a6de0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:42.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.916+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 >> 172.21.15.63:0/2889383107 conn(0x7f104c06d8c0 msgr2=0x7f104c114280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:42.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.917+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 shutdown_connections 2024-04-23T19:09:42.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:42.917+0000 7f10521a7640 1 -- 172.21.15.63:0/2889383107 wait complete. 2024-04-23T19:09:42.929 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 3a1dcf6d-e482-4751-babc-9b0c896a3b00 2 2024-04-23T19:09:43.034 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:43.031+0000 7fa616842040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T19:09:43.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.067+0000 7f00bed54640 1 Processor -- start 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.067+0000 7f00bed54640 1 -- start start 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00bed54640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00b811b1f0 0x7f00b811d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00bed54640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b811dbb0 con 0x7f00b8118460 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00bed54640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f00b811ed90 con 0x7f00b811b1f0 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00b811b1f0 0x7f00b811d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00bcaf0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f00b8118460 0x7f00b8118840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:54324/0 (socket says 172.21.15.63:54324) 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00bcaf0640 1 -- 172.21.15.63:0/3596286803 learned_addr learned my addr 172.21.15.63:0/3596286803 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00b77fe640 1 -- 172.21.15.63:0/3596286803 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175330387 0 0) 0x7f00b811dbb0 con 0x7f00b8118460 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00b77fe640 1 -- 172.21.15.63:0/3596286803 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a8003520 con 0x7f00b8118460 2024-04-23T19:09:43.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00b7fff640 1 -- 172.21.15.63:0/3596286803 >> v1:172.21.15.16:6789/0 conn(0x7f00b8118460 legacy=0x7f00b8118840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.068+0000 7f00b7fff640 1 -- 172.21.15.63:0/3596286803 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b811ef10 con 0x7f00b811b1f0 2024-04-23T19:09:43.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.070+0000 7f00b7fff640 1 --2- 172.21.15.63:0/3596286803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b811b1f0 0x7f00b811d5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009720 tx=0x7f00a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf04f227ff8eb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:43.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.070+0000 7f00b77fe640 1 -- 172.21.15.63:0/3596286803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a40316d0 con 0x7f00b811b1f0 2024-04-23T19:09:43.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.070+0000 7f00b77fe640 1 -- 172.21.15.63:0/3596286803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4037070 con 0x7f00b811b1f0 2024-04-23T19:09:43.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.070+0000 7f00b77fe640 1 -- 172.21.15.63:0/3596286803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a40319c0 con 0x7f00b811b1f0 2024-04-23T19:09:43.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.070+0000 7f00bed54640 1 -- 172.21.15.63:0/3596286803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b811b1f0 msgr2=0x7f00b811d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.070+0000 7f00bed54640 1 --2- 172.21.15.63:0/3596286803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b811b1f0 0x7f00b811d5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009720 tx=0x7f00a402b840 comp rx=0 tx=0).stop 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.071+0000 7f00bed54640 1 -- 172.21.15.63:0/3596286803 shutdown_connections 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.071+0000 7f00bed54640 1 --2- 172.21.15.63:0/3596286803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b811b1f0 0x7f00b811d5e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.071+0000 7f00bed54640 1 -- 172.21.15.63:0/3596286803 >> 172.21.15.63:0/3596286803 conn(0x7f00b806d8c0 msgr2=0x7f00b806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.071+0000 7f00bed54640 1 -- 172.21.15.63:0/3596286803 shutdown_connections 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.071+0000 7f00bed54640 1 -- 172.21.15.63:0/3596286803 wait complete. 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.071+0000 7f00bed54640 1 Processor -- start 2024-04-23T19:09:43.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.071+0000 7f00bed54640 1 -- start start 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.072+0000 7f00bed54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b8118460 0x7f00b81a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.072+0000 7f00bed54640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00b811e2a0 con 0x7f00b8118460 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.072+0000 7f00bcaf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b8118460 0x7f00b81a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.072+0000 7f00bcaf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b8118460 0x7f00b81a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48914/0 (socket says 172.21.15.63:48914) 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.072+0000 7f00bcaf0640 1 -- 172.21.15.63:0/970396852 learned_addr learned my addr 172.21.15.63:0/970396852 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.072+0000 7f00bcaf0640 1 -- 172.21.15.63:0/970396852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b81a7310 con 0x7f00b8118460 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00bcaf0640 1 --2- 172.21.15.63:0/970396852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b8118460 0x7f00b81a6dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c480 tx=0x7f00ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:43.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00b57fa640 1 -- 172.21.15.63:0/970396852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00ac017070 con 0x7f00b8118460 2024-04-23T19:09:43.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00b57fa640 1 -- 172.21.15.63:0/970396852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac005c10 con 0x7f00b8118460 2024-04-23T19:09:43.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b81a7600 con 0x7f00b8118460 2024-04-23T19:09:43.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00b57fa640 1 -- 172.21.15.63:0/970396852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00ac007070 con 0x7f00b8118460 2024-04-23T19:09:43.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81ab6a0 con 0x7f00b8118460 2024-04-23T19:09:43.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00b57fa640 1 -- 172.21.15.63:0/970396852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f00ac01a070 con 0x7f00b8118460 2024-04-23T19:09:43.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.073+0000 7f00b57fa640 1 -- 172.21.15.63:0/970396852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f00ac012820 con 0x7f00b8118460 2024-04-23T19:09:43.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.074+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b80505e0 con 0x7f00b8118460 2024-04-23T19:09:43.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.076+0000 7f00b57fa640 1 -- 172.21.15.63:0/970396852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f00ac021070 con 0x7f00b8118460 2024-04-23T19:09:43.081 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:43.080+0000 7fa616842040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T19:09:43.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.150+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3a1dcf6d-e482-4751-babc-9b0c896a3b00", "id": 2} v 0) -- 0x7f00b81a77d0 con 0x7f00b8118460 2024-04-23T19:09:43.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.151+0000 7f00b57fa640 1 -- 172.21.15.63:0/970396852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3a1dcf6d-e482-4751-babc-9b0c896a3b00", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f00ac012510 con 0x7f00b8118460 2024-04-23T19:09:43.152 INFO:teuthology.orchestra.run.smithi063.stdout:2 2024-04-23T19:09:43.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.152+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b8118460 msgr2=0x7f00b81a6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.152+0000 7f00bed54640 1 --2- 172.21.15.63:0/970396852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b8118460 0x7f00b81a6dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c480 tx=0x7f00ac00c940 comp rx=0 tx=0).stop 2024-04-23T19:09:43.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.152+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 shutdown_connections 2024-04-23T19:09:43.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.152+0000 7f00bed54640 1 --2- 172.21.15.63:0/970396852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b8118460 0x7f00b81a6dd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:43.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.152+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 >> 172.21.15.63:0/970396852 conn(0x7f00b806d8c0 msgr2=0x7f00b8110db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:43.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.152+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 shutdown_connections 2024-04-23T19:09:43.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.152+0000 7f00bed54640 1 -- 172.21.15.63:0/970396852 wait complete. 2024-04-23T19:09:43.164 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 534b421f-9079-4429-ac83-791f6222a368 3 2024-04-23T19:09:43.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.262+0000 7f54d3773640 1 Processor -- start 2024-04-23T19:09:43.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.262+0000 7f54d3773640 1 -- start start 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.262+0000 7f54d3773640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54cc11b410 0x7f54cc11d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.262+0000 7f54d3773640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54cc10fbc0 con 0x7f54cc10f010 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.262+0000 7f54d3773640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f54cc11ed50 con 0x7f54cc11b410 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.263+0000 7f54d0d0e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54cc11b410 0x7f54cc11d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.263+0000 7f54d0d0e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54cc11b410 0x7f54cc11d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48930/0 (socket says 172.21.15.63:48930) 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.263+0000 7f54d0d0e640 1 -- 172.21.15.63:0/3519183401 learned_addr learned my addr 172.21.15.63:0/3519183401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.263+0000 7f54d150f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f54cc10f010 0x7f54cc10f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:54330/0 (socket says 172.21.15.63:54330) 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.263+0000 7f54d0d0e640 1 -- 172.21.15.63:0/3519183401 >> v1:172.21.15.16:6789/0 conn(0x7f54cc10f010 legacy=0x7f54cc10f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.263+0000 7f54d0d0e640 1 -- 172.21.15.63:0/3519183401 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc11ef00 con 0x7f54cc11b410 2024-04-23T19:09:43.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.263+0000 7f54c3fff640 1 -- 172.21.15.63:0/3519183401 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413942602 0 0) 0x7f54cc10fbc0 con 0x7f54cc10f010 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.264+0000 7f54d0d0e640 1 --2- 172.21.15.63:0/3519183401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc11b410 0x7f54cc11d800 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0097f0 tx=0x7f54bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64081f9b12b4867 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.264+0000 7f54c3fff640 1 -- 172.21.15.63:0/3519183401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc035020 con 0x7f54cc11b410 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.264+0000 7f54c3fff640 1 -- 172.21.15.63:0/3519183401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54bc0317a0 con 0x7f54cc11b410 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.264+0000 7f54c3fff640 1 -- 172.21.15.63:0/3519183401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc031aa0 con 0x7f54cc11b410 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 -- 172.21.15.63:0/3519183401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc11b410 msgr2=0x7f54cc11d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 --2- 172.21.15.63:0/3519183401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc11b410 0x7f54cc11d800 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0097f0 tx=0x7f54bc02b910 comp rx=0 tx=0).stop 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 -- 172.21.15.63:0/3519183401 shutdown_connections 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 --2- 172.21.15.63:0/3519183401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc11b410 0x7f54cc11d800 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 -- 172.21.15.63:0/3519183401 >> 172.21.15.63:0/3519183401 conn(0x7f54cc0700e0 msgr2=0x7f54cc072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 -- 172.21.15.63:0/3519183401 shutdown_connections 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 -- 172.21.15.63:0/3519183401 wait complete. 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 Processor -- start 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.265+0000 7f54d3773640 1 -- start start 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.266+0000 7f54d3773640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc10f010 0x7f54cc1a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.266+0000 7f54d3773640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc11e260 con 0x7f54cc10f010 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.266+0000 7f54d150f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc10f010 0x7f54cc1a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.266+0000 7f54d150f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc10f010 0x7f54cc1a6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48938/0 (socket says 172.21.15.63:48938) 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.266+0000 7f54d150f640 1 -- 172.21.15.63:0/1188839504 learned_addr learned my addr 172.21.15.63:0/1188839504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.266+0000 7f54d150f640 1 -- 172.21.15.63:0/1188839504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc1a73f0 con 0x7f54cc10f010 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.266+0000 7f54d150f640 1 --2- 172.21.15.63:0/1188839504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc10f010 0x7f54cc1a6eb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c420 tx=0x7f54c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.267+0000 7f54c1ffb640 1 -- 172.21.15.63:0/1188839504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c8018070 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.267+0000 7f54c1ffb640 1 -- 172.21.15.63:0/1188839504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c80074d0 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.267+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54cc1a76e0 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.267+0000 7f54c1ffb640 1 -- 172.21.15.63:0/1188839504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c8004950 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.267+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54cc1ab790 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.267+0000 7f54c1ffb640 1 -- 172.21.15.63:0/1188839504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54c801b070 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.267+0000 7f54c1ffb640 1 -- 172.21.15.63:0/1188839504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f54c801c840 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.268+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54cc0505e0 con 0x7f54cc10f010 2024-04-23T19:09:43.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.270+0000 7f54c1ffb640 1 -- 172.21.15.63:0/1188839504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f54c8022070 con 0x7f54cc10f010 2024-04-23T19:09:43.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.344+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "534b421f-9079-4429-ac83-791f6222a368", "id": 3} v 0) -- 0x7f54cc1a7910 con 0x7f54cc10f010 2024-04-23T19:09:43.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.345+0000 7f54c1ffb640 1 -- 172.21.15.63:0/1188839504 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "534b421f-9079-4429-ac83-791f6222a368", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f54c801c4e0 con 0x7f54cc10f010 2024-04-23T19:09:43.346 INFO:teuthology.orchestra.run.smithi063.stdout:3 2024-04-23T19:09:43.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.346+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc10f010 msgr2=0x7f54cc1a6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.346+0000 7f54d3773640 1 --2- 172.21.15.63:0/1188839504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc10f010 0x7f54cc1a6eb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c420 tx=0x7f54c800c8e0 comp rx=0 tx=0).stop 2024-04-23T19:09:43.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.346+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 shutdown_connections 2024-04-23T19:09:43.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.346+0000 7f54d3773640 1 --2- 172.21.15.63:0/1188839504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54cc10f010 0x7f54cc1a6eb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:43.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.346+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 >> 172.21.15.63:0/1188839504 conn(0x7f54cc0700e0 msgr2=0x7f54cc070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:43.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.346+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 shutdown_connections 2024-04-23T19:09:43.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:09:43.346+0000 7f54d3773640 1 -- 172.21.15.63:0/1188839504 wait complete. 2024-04-23T19:09:43.357 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-23T19:09:43.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-23T19:09:43.360 INFO:tasks.ceph.osd.0:Started 2024-04-23T19:09:43.360 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-23T19:09:43.360 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-23T19:09:43.362 INFO:tasks.ceph.osd.1:Started 2024-04-23T19:09:43.362 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-23T19:09:43.363 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-23T19:09:43.364 INFO:tasks.ceph.osd.2:Started 2024-04-23T19:09:43.365 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-23T19:09:43.365 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-23T19:09:43.367 INFO:tasks.ceph.osd.3:Started 2024-04-23T19:09:43.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T19:09:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.479+0000 7f7dc4785640 1 Processor -- start 2024-04-23T19:09:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.479+0000 7f7dc4785640 1 -- start start 2024-04-23T19:09:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.479+0000 7f7dc4785640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7dbc12ec80 0x7f7dbc131070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.479+0000 7f7dc4785640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dbc128410 con 0x7f7dbc1288d0 2024-04-23T19:09:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.479+0000 7f7dc4785640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7dbc1286b0 con 0x7f7dbc12ec80 2024-04-23T19:09:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.480+0000 7f7dc1d20640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7dbc12ec80 0x7f7dbc131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.480+0000 7f7dc2521640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7dbc1288d0 0x7f7dbc128cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49060/0 (socket says 172.21.15.16:49060) 2024-04-23T19:09:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.480+0000 7f7dc2521640 1 -- 172.21.15.16:0/475902488 learned_addr learned my addr 172.21.15.16:0/475902488 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.480+0000 7f7dc1d20640 1 -- 172.21.15.16:0/475902488 >> v1:172.21.15.16:6789/0 conn(0x7f7dbc1288d0 legacy=0x7f7dbc128cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.480+0000 7f7dc1d20640 1 -- 172.21.15.16:0/475902488 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc06de40 con 0x7f7dbc12ec80 2024-04-23T19:09:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc1d20640 1 --2- 172.21.15.16:0/475902488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc12ec80 0x7f7dbc131070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b9c0 tx=0x7f7dac02e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a840935aae14a16f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc151f640 1 -- 172.21.15.16:0/475902488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dac03b070 con 0x7f7dbc12ec80 2024-04-23T19:09:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc151f640 1 -- 172.21.15.16:0/475902488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac02ecf0 con 0x7f7dbc12ec80 2024-04-23T19:09:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc151f640 1 -- 172.21.15.16:0/475902488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dac036440 con 0x7f7dbc12ec80 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc4785640 1 -- 172.21.15.16:0/475902488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc12ec80 msgr2=0x7f7dbc131070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc4785640 1 --2- 172.21.15.16:0/475902488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc12ec80 0x7f7dbc131070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b9c0 tx=0x7f7dac02e010 comp rx=0 tx=0).stop 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc4785640 1 -- 172.21.15.16:0/475902488 shutdown_connections 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc4785640 1 --2- 172.21.15.16:0/475902488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc12ec80 0x7f7dbc131070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.481+0000 7f7dc4785640 1 -- 172.21.15.16:0/475902488 >> 172.21.15.16:0/475902488 conn(0x7f7dbc09a200 msgr2=0x7f7dbc097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.482+0000 7f7dc4785640 1 -- 172.21.15.16:0/475902488 shutdown_connections 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.482+0000 7f7dc4785640 1 -- 172.21.15.16:0/475902488 wait complete. 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.482+0000 7f7dc4785640 1 Processor -- start 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.482+0000 7f7dc4785640 1 -- start start 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7dc4785640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1288d0 0x7f7dbc1bf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7dc4785640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc0945f0 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7dc2521640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1288d0 0x7f7dbc1bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7dc2521640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1288d0 0x7f7dbc1bf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42738/0 (socket says 172.21.15.16:42738) 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7dc2521640 1 -- 172.21.15.16:0/2577374286 learned_addr learned my addr 172.21.15.16:0/2577374286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7dc2521640 1 -- 172.21.15.16:0/2577374286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc072df0 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7dc2521640 1 --2- 172.21.15.16:0/2577374286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1288d0 0x7f7dbc1bf420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c4a0 tx=0x7f7db800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7db2ffd640 1 -- 172.21.15.16:0/2577374286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db8017070 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.483+0000 7f7db2ffd640 1 -- 172.21.15.16:0/2577374286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db8005c10 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.484+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc12a1d0 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.484+0000 7f7db2ffd640 1 -- 172.21.15.16:0/2577374286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db8007070 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.484+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc12a940 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.484+0000 7f7db2ffd640 1 -- 172.21.15.16:0/2577374286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7db801a070 con 0x7f7dbc1288d0 2024-04-23T19:09:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.484+0000 7f7db2ffd640 1 -- 172.21.15.16:0/2577374286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7db801cba0 con 0x7f7dbc1288d0 2024-04-23T19:09:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.484+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc063660 con 0x7f7dbc1288d0 2024-04-23T19:09:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.486+0000 7f7db2ffd640 1 -- 172.21.15.16:0/2577374286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7db801c440 con 0x7f7dbc1288d0 2024-04-23T19:09:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.548+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7dbc067f20 con 0x7f7dbc1288d0 2024-04-23T19:09:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.549+0000 7f7db2ffd640 1 -- 172.21.15.16:0/2577374286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7f7db801c620 con 0x7f7dbc1288d0 2024-04-23T19:09:43.550 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:43.550 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"fsid":"f80ae34c-e526-409e-90c3-45f2fcaa83f6","created":"2024-04-23T19:09:42.236475+0000","modified":"2024-04-23T19:09:43.346338+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T19:09:43.346338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8925ca3-b81e-4e90-9554-7a26a6b9ee13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6bf01767-14ca-4771-86ec-2369b6eefd4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3a1dcf6d-e482-4751-babc-9b0c896a3b00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"534b421f-9079-4429-ac83-791f6222a368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:09:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.549+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1288d0 msgr2=0x7f7dbc1bf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.549+0000 7f7dc4785640 1 --2- 172.21.15.16:0/2577374286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1288d0 0x7f7dbc1bf420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c4a0 tx=0x7f7db800c960 comp rx=0 tx=0).stop 2024-04-23T19:09:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.549+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 shutdown_connections 2024-04-23T19:09:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.549+0000 7f7dc4785640 1 --2- 172.21.15.16:0/2577374286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1288d0 0x7f7dbc1bf420 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.549+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 >> 172.21.15.16:0/2577374286 conn(0x7f7dbc09a200 msgr2=0x7f7dbc094e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.550+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 shutdown_connections 2024-04-23T19:09:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:43.550+0000 7f7dc4785640 1 -- 172.21.15.16:0/2577374286 wait complete. 2024-04-23T19:09:43.560 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-23T19:09:43.561 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-23T19:09:43.815 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:43.814+0000 7fa616842040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T19:09:43.906 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:43.905+0000 7fa616842040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T19:09:43.950 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:43.949+0000 7fa616842040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T19:09:43.958 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-23T19:09:43.957+0000 7f6e1c213640 -1 Falling back to public interface 2024-04-23T19:09:43.962 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-04-23T19:09:43.961+0000 7f2b6ac43640 -1 Falling back to public interface 2024-04-23T19:09:43.963 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-04-23T19:09:43.962+0000 7fade2a0e640 -1 Falling back to public interface 2024-04-23T19:09:43.965 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T19:09:43.970 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-04-23T19:09:43.969+0000 7f2e1a4a0640 -1 Falling back to public interface 2024-04-23T19:09:44.059 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.058+0000 7fcd98021640 1 Processor -- start 2024-04-23T19:09:44.059 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.058+0000 7fcd98021640 1 -- start start 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd98021640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd9012f640 0x7fcd9006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd98021640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd900704f0 con 0x7fcd9012f260 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd98021640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcd900709a0 con 0x7fcd9012f640 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd955bc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd9012f640 0x7fcd9006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd95dbd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd9012f260 0x7fcd9006e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49074/0 (socket says 172.21.15.16:49074) 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd95dbd640 1 -- 172.21.15.16:0/3529952596 learned_addr learned my addr 172.21.15.16:0/3529952596 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd955bc640 1 -- 172.21.15.16:0/3529952596 >> v1:172.21.15.16:6789/0 conn(0x7fcd9012f260 legacy=0x7fcd9006e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:44.060 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.059+0000 7fcd955bc640 1 -- 172.21.15.16:0/3529952596 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd90071ed0 con 0x7fcd9012f640 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.060+0000 7fcd955bc640 1 --2- 172.21.15.16:0/3529952596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f640 0x7fcd9006ef60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009750 tx=0x7fcd8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2403df576468bc63 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.060+0000 7fcd94dbb640 1 -- 172.21.15.16:0/3529952596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd800316d0 con 0x7fcd9012f640 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.060+0000 7fcd94dbb640 1 -- 172.21.15.16:0/3529952596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd80037070 con 0x7fcd9012f640 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.060+0000 7fcd94dbb640 1 -- 172.21.15.16:0/3529952596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd800319c0 con 0x7fcd9012f640 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.061+0000 7fcd98021640 1 -- 172.21.15.16:0/3529952596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f640 msgr2=0x7fcd9006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.061+0000 7fcd98021640 1 --2- 172.21.15.16:0/3529952596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f640 0x7fcd9006ef60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009750 tx=0x7fcd8002b840 comp rx=0 tx=0).stop 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.061+0000 7fcd98021640 1 -- 172.21.15.16:0/3529952596 shutdown_connections 2024-04-23T19:09:44.062 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.061+0000 7fcd98021640 1 --2- 172.21.15.16:0/3529952596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f640 0x7fcd9006ef60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:44.063 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.061+0000 7fcd98021640 1 -- 172.21.15.16:0/3529952596 >> 172.21.15.16:0/3529952596 conn(0x7fcd9009fe40 msgr2=0x7fcd9006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:44.063 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.061+0000 7fcd98021640 1 -- 172.21.15.16:0/3529952596 shutdown_connections 2024-04-23T19:09:44.063 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.061+0000 7fcd98021640 1 -- 172.21.15.16:0/3529952596 wait complete. 2024-04-23T19:09:44.063 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd98021640 1 Processor -- start 2024-04-23T19:09:44.063 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd98021640 1 -- start start 2024-04-23T19:09:44.064 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd98021640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f260 0x7fcd9011aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:44.064 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd98021640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9012fab0 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd95dbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f260 0x7fcd9011aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd95dbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f260 0x7fcd9011aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42750/0 (socket says 172.21.15.16:42750) 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd95dbd640 1 -- 172.21.15.16:0/3315905466 learned_addr learned my addr 172.21.15.16:0/3315905466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.062+0000 7fcd95dbd640 1 -- 172.21.15.16:0/3315905466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9011d740 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd95dbd640 1 --2- 172.21.15.16:0/3315905466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f260 0x7fcd9011aae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c4a0 tx=0x7fcd8c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd867fc640 1 -- 172.21.15.16:0/3315905466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd8c017070 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd867fc640 1 -- 172.21.15.16:0/3315905466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c005c10 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9011b020 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd867fc640 1 -- 172.21.15.16:0/3315905466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd8c007070 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9011b4f0 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd867fc640 1 -- 172.21.15.16:0/3315905466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd8c01a070 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.063+0000 7fcd867fc640 1 -- 172.21.15.16:0/3315905466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fcd8c01cb50 con 0x7fcd9012f260 2024-04-23T19:09:44.065 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.064+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9006f450 con 0x7fcd9012f260 2024-04-23T19:09:44.066 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.066+0000 7fcd867fc640 1 -- 172.21.15.16:0/3315905466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcd8c01cda0 con 0x7fcd9012f260 2024-04-23T19:09:44.126 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:44.125+0000 7fa616842040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T19:09:44.128 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.127+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd90063660 con 0x7fcd9012f260 2024-04-23T19:09:44.128 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.128+0000 7fcd867fc640 1 -- 172.21.15.16:0/3315905466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7fcd8c01c440 con 0x7fcd9012f260 2024-04-23T19:09:44.129 INFO:teuthology.misc.health.smithi016.stdout: 2024-04-23T19:09:44.129 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":6,"fsid":"f80ae34c-e526-409e-90c3-45f2fcaa83f6","created":"2024-04-23T19:09:42.236475+0000","modified":"2024-04-23T19:09:43.346338+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T19:09:43.346338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8925ca3-b81e-4e90-9554-7a26a6b9ee13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6bf01767-14ca-4771-86ec-2369b6eefd4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3a1dcf6d-e482-4751-babc-9b0c896a3b00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"534b421f-9079-4429-ac83-791f6222a368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:09:44.130 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.128+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f260 msgr2=0x7fcd9011aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:44.130 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.128+0000 7fcd98021640 1 --2- 172.21.15.16:0/3315905466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f260 0x7fcd9011aae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c4a0 tx=0x7fcd8c00c960 comp rx=0 tx=0).stop 2024-04-23T19:09:44.130 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.129+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 shutdown_connections 2024-04-23T19:09:44.130 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.129+0000 7fcd98021640 1 --2- 172.21.15.16:0/3315905466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9012f260 0x7fcd9011aae0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:44.130 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.129+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 >> 172.21.15.16:0/3315905466 conn(0x7fcd9009fe40 msgr2=0x7fcd900a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:44.130 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.129+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 shutdown_connections 2024-04-23T19:09:44.130 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:44.129+0000 7fcd98021640 1 -- 172.21.15.16:0/3315905466 wait complete. 2024-04-23T19:09:44.138 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-23T19:09:44.175 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:44.174+0000 7fa616842040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T19:09:44.230 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:44.229+0000 7fa616842040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T19:09:44.347 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:44.346+0000 7fa616842040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T19:09:44.744 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:44.743+0000 7fa616842040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T19:09:45.014 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.013+0000 7fa616842040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T19:09:45.068 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.067+0000 7fa616842040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T19:09:45.190 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.189+0000 7fa616842040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T19:09:45.479 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.478+0000 7fa616842040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T19:09:45.528 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.527+0000 7fa616842040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T19:09:45.694 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.690+0000 7fa616842040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T19:09:45.835 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.834+0000 7fa616842040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T19:09:45.887 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.886+0000 7fa616842040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T19:09:45.960 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:45.959+0000 7fa616842040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T19:09:46.007 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:46.006+0000 7fa616842040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T19:09:46.510 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:46.508+0000 7fa616842040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T19:09:46.630 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:46.629+0000 7fa616842040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T19:09:47.380 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-23T19:09:47.379+0000 7f6e1c213640 -1 osd.0 0 log_to_monitors true 2024-04-23T19:09:47.449 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-04-23T19:09:47.448+0000 7fade2a0e640 -1 osd.2 0 log_to_monitors true 2024-04-23T19:09:47.469 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-04-23T19:09:47.468+0000 7f2b6ac43640 -1 osd.1 0 log_to_monitors true 2024-04-23T19:09:47.470 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-04-23T19:09:47.470+0000 7f2e1a4a0640 -1 osd.3 0 log_to_monitors true 2024-04-23T19:09:47.639 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-04-23T19:09:47.638+0000 7fa5b7454640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-23T19:09:49.426 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-23T19:09:49.425+0000 7f6e17bff640 -1 osd.0 0 waiting for initial osdmap 2024-04-23T19:09:49.436 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-23T19:09:49.436+0000 7f6e0fa1f640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T19:09:50.347 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-04-23T19:09:50.346+0000 7faddec65640 -1 osd.2 0 waiting for initial osdmap 2024-04-23T19:09:50.362 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-04-23T19:09:50.361+0000 7fadd6271640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T19:09:50.421 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-04-23T19:09:50.420+0000 7f2e16436640 -1 osd.3 0 waiting for initial osdmap 2024-04-23T19:09:50.432 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-04-23T19:09:50.431+0000 7f2e0e256640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T19:09:50.454 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-04-23T19:09:50.453+0000 7f2b66bd9640 -1 osd.1 0 waiting for initial osdmap 2024-04-23T19:09:50.468 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-04-23T19:09:50.467+0000 7f2b5e9f9640 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T19:09:50.541 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T19:09:50.645 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.644+0000 7ff047515640 1 Processor -- start 2024-04-23T19:09:50.645 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.644+0000 7ff047515640 1 -- start start 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff047515640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff04012ebb0 0x7ff040130fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff047515640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff04012b9c0 con 0x7ff04012bcc0 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff047515640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff04006cfb0 con 0x7ff04012ebb0 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff044ab0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff04012ebb0 0x7ff040130fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff0452b1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff04012bcc0 0x7ff04012c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57758/0 (socket says 172.21.15.16:57758) 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff0452b1640 1 -- 172.21.15.16:0/2908061467 learned_addr learned my addr 172.21.15.16:0/2908061467 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff044ab0640 1 -- 172.21.15.16:0/2908061467 >> v1:172.21.15.16:6789/0 conn(0x7ff04012bcc0 legacy=0x7ff04012c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:50.646 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.645+0000 7ff044ab0640 1 -- 172.21.15.16:0/2908061467 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04006ec40 con 0x7ff04012ebb0 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.646+0000 7ff044ab0640 1 --2- 172.21.15.16:0/2908061467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012ebb0 0x7ff040130fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff030009720 tx=0x7ff03002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535ac6de8c02b3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.646+0000 7ff037fff640 1 -- 172.21.15.16:0/2908061467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0300316d0 con 0x7ff04012ebb0 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.646+0000 7ff037fff640 1 -- 172.21.15.16:0/2908061467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff030037070 con 0x7ff04012ebb0 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.646+0000 7ff037fff640 1 -- 172.21.15.16:0/2908061467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0300319c0 con 0x7ff04012ebb0 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.647+0000 7ff047515640 1 -- 172.21.15.16:0/2908061467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012ebb0 msgr2=0x7ff040130fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.647+0000 7ff047515640 1 --2- 172.21.15.16:0/2908061467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012ebb0 0x7ff040130fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff030009720 tx=0x7ff03002b840 comp rx=0 tx=0).stop 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.647+0000 7ff047515640 1 -- 172.21.15.16:0/2908061467 shutdown_connections 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.647+0000 7ff047515640 1 --2- 172.21.15.16:0/2908061467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012ebb0 0x7ff040130fa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:50.648 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.647+0000 7ff047515640 1 -- 172.21.15.16:0/2908061467 >> 172.21.15.16:0/2908061467 conn(0x7ff04009b840 msgr2=0x7ff04006d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:50.649 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.647+0000 7ff047515640 1 -- 172.21.15.16:0/2908061467 shutdown_connections 2024-04-23T19:09:50.649 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.647+0000 7ff047515640 1 -- 172.21.15.16:0/2908061467 wait complete. 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.648+0000 7ff047515640 1 Processor -- start 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.648+0000 7ff047515640 1 -- start start 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.648+0000 7ff047515640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012bcc0 0x7ff040125f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.648+0000 7ff047515640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff040072380 con 0x7ff04012bcc0 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.649+0000 7ff0452b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012bcc0 0x7ff040125f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.649+0000 7ff0452b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012bcc0 0x7ff040125f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40994/0 (socket says 172.21.15.16:40994) 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.649+0000 7ff0452b1640 1 -- 172.21.15.16:0/1006482693 learned_addr learned my addr 172.21.15.16:0/1006482693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.649+0000 7ff0452b1640 1 -- 172.21.15.16:0/1006482693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff040127490 con 0x7ff04012bcc0 2024-04-23T19:09:50.650 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.649+0000 7ff0452b1640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012bcc0 0x7ff040125f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00c4a0 tx=0x7ff03c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:50.651 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.650+0000 7ff035ffb640 1 -- 172.21.15.16:0/1006482693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03c017070 con 0x7ff04012bcc0 2024-04-23T19:09:50.651 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.650+0000 7ff035ffb640 1 -- 172.21.15.16:0/1006482693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c005c10 con 0x7ff04012bcc0 2024-04-23T19:09:50.652 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.650+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff040127940 con 0x7ff04012bcc0 2024-04-23T19:09:50.652 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.650+0000 7ff035ffb640 1 -- 172.21.15.16:0/1006482693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03c007070 con 0x7ff04012bcc0 2024-04-23T19:09:50.652 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.650+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff040127e70 con 0x7ff04012bcc0 2024-04-23T19:09:50.652 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.651+0000 7ff035ffb640 1 -- 172.21.15.16:0/1006482693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 45778+0+0 (secure 0 0 0) 0x7ff03c01a070 con 0x7ff04012bcc0 2024-04-23T19:09:50.652 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.651+0000 7ff035ffb640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff024038530 0x7ff02403a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:50.652 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.651+0000 7ff035ffb640 1 -- 172.21.15.16:0/1006482693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 2183+0+0 (secure 0 0 0) 0x7ff03c04a310 con 0x7ff04012bcc0 2024-04-23T19:09:50.652 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.652+0000 7ff044ab0640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff024038530 0x7ff02403a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:50.653 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.652+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04006b550 con 0x7ff04012bcc0 2024-04-23T19:09:50.653 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.652+0000 7ff044ab0640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff024038530 0x7ff02403a9f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff030036000 tx=0x7ff030008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:50.657 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.657+0000 7ff035ffb640 1 -- 172.21.15.16:0/1006482693 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff03c0117a0 con 0x7ff04012bcc0 2024-04-23T19:09:50.843 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.842+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff040063660 con 0x7ff04012bcc0 2024-04-23T19:09:50.843 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.843+0000 7ff035ffb640 1 -- 172.21.15.16:0/1006482693 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) ==== 74+0+5110 (secure 0 0 0) 0x7ff03c012400 con 0x7ff04012bcc0 2024-04-23T19:09:50.844 INFO:teuthology.misc.health.smithi016.stdout: 2024-04-23T19:09:50.844 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":9,"fsid":"f80ae34c-e526-409e-90c3-45f2fcaa83f6","created":"2024-04-23T19:09:42.236475+0000","modified":"2024-04-23T19:09:50.427502+0000","last_up_change":"2024-04-23T19:09:50.427502+0000","last_in_change":"2024-04-23T19:09:43.346338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8925ca3-b81e-4e90-9554-7a26a6b9ee13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6801","nonce":2262294208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6803","nonce":2262294208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6807","nonce":2262294208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6805","nonce":2262294208}]},"public_addr":"172.21.15.16:6801/2262294208","cluster_addr":"172.21.15.16:6803/2262294208","heartbeat_back_addr":"172.21.15.16:6807/2262294208","heartbeat_front_addr":"172.21.15.16:6805/2262294208","state":["exists","up"]},{"osd":1,"uuid":"6bf01767-14ca-4771-86ec-2369b6eefd4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3a1dcf6d-e482-4751-babc-9b0c896a3b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6809","nonce":753864785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6811","nonce":753864785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6815","nonce":753864785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6813","nonce":753864785}]},"public_addr":"172.21.15.63:6809/753864785","cluster_addr":"172.21.15.63:6811/753864785","heartbeat_back_addr":"172.21.15.63:6815/753864785","heartbeat_front_addr":"172.21.15.63:6813/753864785","state":["exists","up"]},{"osd":3,"uuid":"534b421f-9079-4429-ac83-791f6222a368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff024038530 msgr2=0x7ff02403a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff024038530 0x7ff02403a9f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff030036000 tx=0x7ff030008030 comp rx=0 tx=0).stop 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012bcc0 msgr2=0x7ff040125f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012bcc0 0x7ff040125f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00c4a0 tx=0x7ff03c00c960 comp rx=0 tx=0).stop 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 shutdown_connections 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff024038530 0x7ff02403a9f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 --2- 172.21.15.16:0/1006482693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04012bcc0 0x7ff040125f90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 >> 172.21.15.16:0/1006482693 conn(0x7ff04009b840 msgr2=0x7ff04012a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 shutdown_connections 2024-04-23T19:09:50.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:50.845+0000 7ff047515640 1 -- 172.21.15.16:0/1006482693 wait complete. 2024-04-23T19:09:50.855 DEBUG:teuthology.misc:2 of 4 OSDs are up 2024-04-23T19:09:57.260 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T19:09:57.365 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.363+0000 7f4aa58c8640 1 Processor -- start 2024-04-23T19:09:57.365 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.363+0000 7f4aa58c8640 1 -- start start 2024-04-23T19:09:57.365 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4aa58c8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aa012f640 0x7f4aa006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:57.365 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4aa58c8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa00704f0 con 0x7f4aa012f260 2024-04-23T19:09:57.365 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4aa58c8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa00709a0 con 0x7f4aa012f640 2024-04-23T19:09:57.365 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4a9e7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aa012f640 0x7f4aa006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:57.366 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4a9effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4aa012f260 0x7f4aa006e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60448/0 (socket says 172.21.15.16:60448) 2024-04-23T19:09:57.366 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4a9e7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aa012f640 0x7f4aa006ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56644/0 (socket says 172.21.15.16:56644) 2024-04-23T19:09:57.366 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4a9effd640 1 -- 172.21.15.16:0/528736730 learned_addr learned my addr 172.21.15.16:0/528736730 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:57.366 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4a9e7fc640 1 -- 172.21.15.16:0/528736730 >> v1:172.21.15.16:6789/0 conn(0x7f4aa012f260 legacy=0x7f4aa006e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:57.366 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.364+0000 7f4a9e7fc640 1 -- 172.21.15.16:0/528736730 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa0071ed0 con 0x7f4aa012f640 2024-04-23T19:09:57.367 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.365+0000 7f4a9e7fc640 1 --2- 172.21.15.16:0/528736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f640 0x7f4aa006ef60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009750 tx=0x7f4a9402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe93ea1c47921902 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:57.367 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.365+0000 7f4a9dffb640 1 -- 172.21.15.16:0/528736730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a940316d0 con 0x7f4aa012f640 2024-04-23T19:09:57.367 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.365+0000 7f4a9dffb640 1 -- 172.21.15.16:0/528736730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a94037070 con 0x7f4aa012f640 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.365+0000 7f4a9dffb640 1 -- 172.21.15.16:0/528736730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a940319c0 con 0x7f4aa012f640 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.366+0000 7f4aa58c8640 1 -- 172.21.15.16:0/528736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f640 msgr2=0x7f4aa006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.366+0000 7f4aa58c8640 1 --2- 172.21.15.16:0/528736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f640 0x7f4aa006ef60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009750 tx=0x7f4a9402b840 comp rx=0 tx=0).stop 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.366+0000 7f4aa58c8640 1 -- 172.21.15.16:0/528736730 shutdown_connections 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.366+0000 7f4aa58c8640 1 --2- 172.21.15.16:0/528736730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f640 0x7f4aa006ef60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.366+0000 7f4aa58c8640 1 -- 172.21.15.16:0/528736730 >> 172.21.15.16:0/528736730 conn(0x7f4aa009fe40 msgr2=0x7f4aa006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.366+0000 7f4aa58c8640 1 -- 172.21.15.16:0/528736730 shutdown_connections 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.366+0000 7f4aa58c8640 1 -- 172.21.15.16:0/528736730 wait complete. 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.367+0000 7f4aa58c8640 1 Processor -- start 2024-04-23T19:09:57.368 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.367+0000 7f4aa58c8640 1 -- start start 2024-04-23T19:09:57.370 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.367+0000 7f4aa58c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f260 0x7f4aa011aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:57.370 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.367+0000 7f4aa58c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa012fab0 con 0x7f4aa012f260 2024-04-23T19:09:57.370 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f260 0x7f4aa011aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:57.370 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f260 0x7f4aa011aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56656/0 (socket says 172.21.15.16:56656) 2024-04-23T19:09:57.370 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a9effd640 1 -- 172.21.15.16:0/2502975697 learned_addr learned my addr 172.21.15.16:0/2502975697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:57.370 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a9effd640 1 -- 172.21.15.16:0/2502975697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa011d740 con 0x7f4aa012f260 2024-04-23T19:09:57.370 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a9effd640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f260 0x7f4aa011aae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000cb20 tx=0x7f4a9000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a87fff640 1 -- 172.21.15.16:0/2502975697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a90017070 con 0x7f4aa012f260 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a87fff640 1 -- 172.21.15.16:0/2502975697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9000abf0 con 0x7f4aa012f260 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4a87fff640 1 -- 172.21.15.16:0/2502975697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a90012400 con 0x7f4aa012f260 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa011b020 con 0x7f4aa012f260 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.368+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa011b4f0 con 0x7f4aa012f260 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.369+0000 7f4a87fff640 1 -- 172.21.15.16:0/2502975697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4a9001a070 con 0x7f4aa012f260 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.369+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa006f450 con 0x7f4aa012f260 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.370+0000 7f4a87fff640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a8c0385c0 0x7f4a8c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:57.371 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.370+0000 7f4a87fff640 1 -- 172.21.15.16:0/2502975697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3129+0+0 (secure 0 0 0) 0x7f4a9004b160 con 0x7f4aa012f260 2024-04-23T19:09:57.372 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.370+0000 7f4a9e7fc640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a8c0385c0 0x7f4a8c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:57.372 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.371+0000 7f4a9e7fc640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a8c0385c0 0x7f4a8c03aa80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a94036000 tx=0x7f4a94008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:57.375 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.375+0000 7f4a87fff640 1 -- 172.21.15.16:0/2502975697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a900113a0 con 0x7f4aa012f260 2024-04-23T19:09:57.565 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.565+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4aa0063660 con 0x7f4aa012f260 2024-04-23T19:09:57.566 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.566+0000 7f4a87fff640 1 -- 172.21.15.16:0/2502975697 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7941 (secure 0 0 0) 0x7f4a90011580 con 0x7f4aa012f260 2024-04-23T19:09:57.566 INFO:teuthology.misc.health.smithi016.stdout: 2024-04-23T19:09:57.566 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":12,"fsid":"f80ae34c-e526-409e-90c3-45f2fcaa83f6","created":"2024-04-23T19:09:42.236475+0000","modified":"2024-04-23T19:09:53.430424+0000","last_up_change":"2024-04-23T19:09:51.428209+0000","last_in_change":"2024-04-23T19:09:43.346338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T19:09:50.651043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8925ca3-b81e-4e90-9554-7a26a6b9ee13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6801","nonce":2262294208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6803","nonce":2262294208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6807","nonce":2262294208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6805","nonce":2262294208}]},"public_addr":"172.21.15.16:6801/2262294208","cluster_addr":"172.21.15.16:6803/2262294208","heartbeat_back_addr":"172.21.15.16:6807/2262294208","heartbeat_front_addr":"172.21.15.16:6805/2262294208","state":["exists","up"]},{"osd":1,"uuid":"6bf01767-14ca-4771-86ec-2369b6eefd4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6801","nonce":3216879695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6803","nonce":3216879695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6807","nonce":3216879695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6805","nonce":3216879695}]},"public_addr":"172.21.15.63:6801/3216879695","cluster_addr":"172.21.15.63:6803/3216879695","heartbeat_back_addr":"172.21.15.63:6807/3216879695","heartbeat_front_addr":"172.21.15.63:6805/3216879695","state":["exists","up"]},{"osd":2,"uuid":"3a1dcf6d-e482-4751-babc-9b0c896a3b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6809","nonce":753864785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6811","nonce":753864785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6815","nonce":753864785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6813","nonce":753864785}]},"public_addr":"172.21.15.63:6809/753864785","cluster_addr":"172.21.15.63:6811/753864785","heartbeat_back_addr":"172.21.15.63:6815/753864785","heartbeat_front_addr":"172.21.15.63:6813/753864785","state":["exists","up"]},{"osd":3,"uuid":"534b421f-9079-4429-ac83-791f6222a368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6817","nonce":1289558434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6819","nonce":1289558434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6823","nonce":1289558434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6821","nonce":1289558434}]},"public_addr":"172.21.15.63:6817/1289558434","cluster_addr":"172.21.15.63:6819/1289558434","heartbeat_back_addr":"172.21.15.63:6823/1289558434","heartbeat_front_addr":"172.21.15.63:6821/1289558434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.387629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.445168+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.449933+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.519415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:09:57.568 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a8c0385c0 msgr2=0x7f4a8c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a8c0385c0 0x7f4a8c03aa80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a94036000 tx=0x7f4a94008030 comp rx=0 tx=0).stop 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f260 msgr2=0x7f4aa011aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f260 0x7f4aa011aae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000cb20 tx=0x7f4a9000a060 comp rx=0 tx=0).stop 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 shutdown_connections 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a8c0385c0 0x7f4a8c03aa80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 --2- 172.21.15.16:0/2502975697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa012f260 0x7f4aa011aae0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 >> 172.21.15.16:0/2502975697 conn(0x7f4aa009fe40 msgr2=0x7f4aa00a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 shutdown_connections 2024-04-23T19:09:57.569 INFO:teuthology.misc.health.smithi016.stderr:2024-04-23T19:09:57.568+0000 7f4aa58c8640 1 -- 172.21.15.16:0/2502975697 wait complete. 2024-04-23T19:09:57.578 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-23T19:09:57.578 INFO:tasks.ceph:Creating RBD pool 2024-04-23T19:09:57.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.677+0000 7f7fbbb7c640 1 Processor -- start 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.677+0000 7f7fbbb7c640 1 -- start start 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fbbb7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7fb4131f80 0x7f7fb4134370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fbbb7c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fb4096880 con 0x7f7fb4128730 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fbbb7c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7fb4096cc0 con 0x7f7fb4131f80 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fb9918640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7fb4128730 0x7f7fb4128b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60450/0 (socket says 172.21.15.16:60450) 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fb9918640 1 -- 172.21.15.16:0/632124465 learned_addr learned my addr 172.21.15.16:0/632124465 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fb9117640 1 --2- 172.21.15.16:0/632124465 >> v2:172.21.15.16:3300/0 conn(0x7f7fb4131f80 0x7f7fb4134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fb8916640 1 -- 172.21.15.16:0/632124465 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1634346518 0 0) 0x7f7fb4096880 con 0x7f7fb4128730 2024-04-23T19:09:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fb8916640 1 -- 172.21.15.16:0/632124465 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fa0003520 con 0x7f7fb4128730 2024-04-23T19:09:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fb9117640 1 -- 172.21.15.16:0/632124465 >> v1:172.21.15.16:6789/0 conn(0x7f7fb4128730 legacy=0x7f7fb4128b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.678+0000 7f7fb9117640 1 -- 172.21.15.16:0/632124465 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb409a740 con 0x7f7fb4131f80 2024-04-23T19:09:57.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.679+0000 7f7fb9117640 1 --2- 172.21.15.16:0/632124465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4131f80 0x7f7fb4134370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009720 tx=0x7f7fa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ca7fa30ddbf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:57.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.679+0000 7f7fb8916640 1 -- 172.21.15.16:0/632124465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fa4035020 con 0x7f7fb4131f80 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.679+0000 7f7fb8916640 1 -- 172.21.15.16:0/632124465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa4031760 con 0x7f7fb4131f80 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.679+0000 7f7fb8916640 1 -- 172.21.15.16:0/632124465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fa4031a60 con 0x7f7fb4131f80 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.679+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/632124465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4131f80 msgr2=0x7f7fb4134370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.679+0000 7f7fbbb7c640 1 --2- 172.21.15.16:0/632124465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4131f80 0x7f7fb4134370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009720 tx=0x7f7fa402b8d0 comp rx=0 tx=0).stop 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.680+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/632124465 shutdown_connections 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.680+0000 7f7fbbb7c640 1 --2- 172.21.15.16:0/632124465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4131f80 0x7f7fb4134370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.680+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/632124465 >> 172.21.15.16:0/632124465 conn(0x7f7fb409d8f0 msgr2=0x7f7fb4099dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.680+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/632124465 shutdown_connections 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.680+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/632124465 wait complete. 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.680+0000 7f7fbbb7c640 1 Processor -- start 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fbbb7c640 1 -- start start 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fbbb7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4128730 0x7f7fb41c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fbbb7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb40970e0 con 0x7f7fb4128730 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fb9918640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4128730 0x7f7fb41c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fb9918640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4128730 0x7f7fb41c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56676/0 (socket says 172.21.15.16:56676) 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fb9918640 1 -- 172.21.15.16:0/2766136498 learned_addr learned my addr 172.21.15.16:0/2766136498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fb9918640 1 -- 172.21.15.16:0/2766136498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb4091660 con 0x7f7fb4128730 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.681+0000 7f7fb9918640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4128730 0x7f7fb41c3ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00c480 tx=0x7f7fac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.682+0000 7f7faa7fc640 1 -- 172.21.15.16:0/2766136498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fac017070 con 0x7f7fb4128730 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.682+0000 7f7faa7fc640 1 -- 172.21.15.16:0/2766136498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac005c10 con 0x7f7fb4128730 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.682+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb4092980 con 0x7f7fb4128730 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.682+0000 7f7faa7fc640 1 -- 172.21.15.16:0/2766136498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fac007070 con 0x7f7fb4128730 2024-04-23T19:09:57.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.682+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb4092120 con 0x7f7fb4128730 2024-04-23T19:09:57.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.682+0000 7f7faa7fc640 1 -- 172.21.15.16:0/2766136498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7fac01a070 con 0x7f7fb4128730 2024-04-23T19:09:57.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.682+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb4063660 con 0x7f7fb4128730 2024-04-23T19:09:57.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.683+0000 7f7faa7fc640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7f980385c0 0x7f7f9803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:57.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.683+0000 7f7faa7fc640 1 -- 172.21.15.16:0/2766136498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3129+0+0 (secure 0 0 0) 0x7f7fac04a7c0 con 0x7f7fb4128730 2024-04-23T19:09:57.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.683+0000 7f7fb9117640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7f980385c0 0x7f7f9803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:57.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.684+0000 7f7fb9117640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7f980385c0 0x7f7f9803aa80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4036000 tx=0x7f7fa4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:57.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.687+0000 7f7faa7fc640 1 -- 172.21.15.16:0/2766136498 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fb4091660 con 0x7f7fb4128730 2024-04-23T19:09:57.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:57.871+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f7fb4067f20 con 0x7f7fb4128730 2024-04-23T19:09:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.435+0000 7f7faa7fc640 1 -- 172.21.15.16:0/2766136498 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) ==== 109+0+0 (secure 0 0 0) 0x7f7fac01b730 con 0x7f7fb4128730 2024-04-23T19:09:58.436 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'rbd' created 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7f980385c0 msgr2=0x7f7f9803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7f980385c0 0x7f7f9803aa80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4036000 tx=0x7f7fa4008030 comp rx=0 tx=0).stop 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4128730 msgr2=0x7f7fb41c3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4128730 0x7f7fb41c3ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00c480 tx=0x7f7fac00c940 comp rx=0 tx=0).stop 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 shutdown_connections 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7f980385c0 0x7f7f9803aa80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 --2- 172.21.15.16:0/2766136498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb4128730 0x7f7fb41c3ad0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 >> 172.21.15.16:0/2766136498 conn(0x7f7fb409d8f0 msgr2=0x7f7fb4094750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 shutdown_connections 2024-04-23T19:09:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.438+0000 7f7fbbb7c640 1 -- 172.21.15.16:0/2766136498 wait complete. 2024-04-23T19:09:58.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-23T19:09:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.555+0000 7fb523160640 1 Processor -- start 2024-04-23T19:09:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.555+0000 7fb523160640 1 -- start start 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb523160640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb51c0709c0 0x7fb51c133e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb523160640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb51c06ee10 con 0x7fb51c06f2d0 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb523160640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb51c06bc80 con 0x7fb51c0709c0 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb51bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb51c0709c0 0x7fb51c133e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb520efc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb51c06f2d0 0x7fb51c06f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60454/0 (socket says 172.21.15.16:60454) 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb520efc640 1 -- 172.21.15.16:0/3658205813 learned_addr learned my addr 172.21.15.16:0/3658205813 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb51bfff640 1 -- 172.21.15.16:0/3658205813 >> v1:172.21.15.16:6789/0 conn(0x7fb51c06f2d0 legacy=0x7fb51c06f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb51bfff640 1 -- 172.21.15.16:0/3658205813 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c0710c0 con 0x7fb51c0709c0 2024-04-23T19:09:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.556+0000 7fb51b7fe640 1 -- 172.21.15.16:0/3658205813 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989222214 0 0) 0x7fb51c06ee10 con 0x7fb51c06f2d0 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb51bfff640 1 --2- 172.21.15.16:0/3658205813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c0709c0 0x7fb51c133e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb510009720 tx=0x7fb51002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72532139960bd1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb51b7fe640 1 -- 172.21.15.16:0/3658205813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5100316d0 con 0x7fb51c0709c0 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb51b7fe640 1 -- 172.21.15.16:0/3658205813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510037070 con 0x7fb51c0709c0 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb51b7fe640 1 -- 172.21.15.16:0/3658205813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5100319c0 con 0x7fb51c0709c0 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb523160640 1 -- 172.21.15.16:0/3658205813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c0709c0 msgr2=0x7fb51c133e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb523160640 1 --2- 172.21.15.16:0/3658205813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c0709c0 0x7fb51c133e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb510009720 tx=0x7fb51002b840 comp rx=0 tx=0).stop 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb523160640 1 -- 172.21.15.16:0/3658205813 shutdown_connections 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb523160640 1 --2- 172.21.15.16:0/3658205813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c0709c0 0x7fb51c133e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.557+0000 7fb523160640 1 -- 172.21.15.16:0/3658205813 >> 172.21.15.16:0/3658205813 conn(0x7fb51c09d640 msgr2=0x7fb51c09da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.558+0000 7fb523160640 1 -- 172.21.15.16:0/3658205813 shutdown_connections 2024-04-23T19:09:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.558+0000 7fb523160640 1 -- 172.21.15.16:0/3658205813 wait complete. 2024-04-23T19:09:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.558+0000 7fb523160640 1 Processor -- start 2024-04-23T19:09:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.558+0000 7fb523160640 1 -- start start 2024-04-23T19:09:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb523160640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c06f2d0 0x7fb51c118280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb523160640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c134560 con 0x7fb51c06f2d0 2024-04-23T19:09:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb520efc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c06f2d0 0x7fb51c118280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb520efc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c06f2d0 0x7fb51c118280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56704/0 (socket says 172.21.15.16:56704) 2024-04-23T19:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb520efc640 1 -- 172.21.15.16:0/1547890128 learned_addr learned my addr 172.21.15.16:0/1547890128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb520efc640 1 -- 172.21.15.16:0/1547890128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c11a910 con 0x7fb51c06f2d0 2024-04-23T19:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb520efc640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c06f2d0 0x7fb51c118280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c840 tx=0x7fb51400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb5197fa640 1 -- 172.21.15.16:0/1547890128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb514018070 con 0x7fb51c06f2d0 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb5197fa640 1 -- 172.21.15.16:0/1547890128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51400a660 con 0x7fb51c06f2d0 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.559+0000 7fb5197fa640 1 -- 172.21.15.16:0/1547890128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb51400a980 con 0x7fb51c06f2d0 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.560+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c11aba0 con 0x7fb51c06f2d0 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.560+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c1189a0 con 0x7fb51c06f2d0 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.561+0000 7fb5197fa640 1 -- 172.21.15.16:0/1547890128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb51401b070 con 0x7fb51c06f2d0 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.561+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51c063660 con 0x7fb51c06f2d0 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.561+0000 7fb5197fa640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb5080385c0 0x7fb50803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.561+0000 7fb5197fa640 1 -- 172.21.15.16:0/1547890128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3471+0+0 (secure 0 0 0) 0x7fb51404aca0 con 0x7fb51c06f2d0 2024-04-23T19:09:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.561+0000 7fb51bfff640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb5080385c0 0x7fb50803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.562+0000 7fb51bfff640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb5080385c0 0x7fb50803aa80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb510036000 tx=0x7fb510008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.566+0000 7fb5197fa640 1 -- 172.21.15.16:0/1547890128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb514011360 con 0x7fb51c06f2d0 2024-04-23T19:09:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:58.752+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7fb51c067f20 con 0x7fb51c06f2d0 2024-04-23T19:09:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.441+0000 7fb5197fa640 1 -- 172.21.15.16:0/1547890128 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) ==== 173+0+0 (secure 0 0 0) 0x7fb51400adb0 con 0x7fb51c06f2d0 2024-04-23T19:09:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb5080385c0 msgr2=0x7fb50803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb5080385c0 0x7fb50803aa80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb510036000 tx=0x7fb510008030 comp rx=0 tx=0).stop 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c06f2d0 msgr2=0x7fb51c118280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c06f2d0 0x7fb51c118280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c840 tx=0x7fb51400cd00 comp rx=0 tx=0).stop 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 shutdown_connections 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb5080385c0 0x7fb50803aa80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 --2- 172.21.15.16:0/1547890128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb51c06f2d0 0x7fb51c118280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 >> 172.21.15.16:0/1547890128 conn(0x7fb51c09d640 msgr2=0x7fb51c12c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 shutdown_connections 2024-04-23T19:09:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.445+0000 7fb523160640 1 -- 172.21.15.16:0/1547890128 wait complete. 2024-04-23T19:09:59.460 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-23T19:09:59.460 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T19:09:59.461 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-23T19:09:59.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-23T19:09:59.461 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-23T19:09:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.566+0000 7f1b8377e640 1 Processor -- start 2024-04-23T19:09:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.566+0000 7f1b8377e640 1 -- start start 2024-04-23T19:09:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b8377e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b7c1288d0 0x7f1b7c128cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b8377e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7c128410 con 0x7f1b7c121e40 2024-04-23T19:09:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b8377e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7c1286b0 con 0x7f1b7c1288d0 2024-04-23T19:09:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b8151a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b7c1288d0 0x7f1b7c128cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b80d19640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1b7c121e40 0x7f1b7c122280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60460/0 (socket says 172.21.15.16:60460) 2024-04-23T19:09:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b80d19640 1 -- 172.21.15.16:0/3920223712 learned_addr learned my addr 172.21.15.16:0/3920223712 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b8151a640 1 -- 172.21.15.16:0/3920223712 >> v1:172.21.15.16:6789/0 conn(0x7f1b7c121e40 legacy=0x7f1b7c122280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b8151a640 1 -- 172.21.15.16:0/3920223712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c06de40 con 0x7f1b7c1288d0 2024-04-23T19:09:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.567+0000 7f1b73fff640 1 -- 172.21.15.16:0/3920223712 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622356615 0 0) 0x7f1b7c128410 con 0x7f1b7c121e40 2024-04-23T19:09:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.568+0000 7f1b8151a640 1 --2- 172.21.15.16:0/3920223712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c1288d0 0x7f1b7c128cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009720 tx=0x7f1b7802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336aeed895228c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.568+0000 7f1b73fff640 1 -- 172.21.15.16:0/3920223712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b780316d0 con 0x7f1b7c1288d0 2024-04-23T19:09:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.568+0000 7f1b73fff640 1 -- 172.21.15.16:0/3920223712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b78037070 con 0x7f1b7c1288d0 2024-04-23T19:09:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.568+0000 7f1b73fff640 1 -- 172.21.15.16:0/3920223712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b780319c0 con 0x7f1b7c1288d0 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.569+0000 7f1b8377e640 1 -- 172.21.15.16:0/3920223712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c1288d0 msgr2=0x7f1b7c128cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.569+0000 7f1b8377e640 1 --2- 172.21.15.16:0/3920223712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c1288d0 0x7f1b7c128cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009720 tx=0x7f1b7802b840 comp rx=0 tx=0).stop 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.569+0000 7f1b8377e640 1 -- 172.21.15.16:0/3920223712 shutdown_connections 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.569+0000 7f1b8377e640 1 --2- 172.21.15.16:0/3920223712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c1288d0 0x7f1b7c128cb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.569+0000 7f1b8377e640 1 -- 172.21.15.16:0/3920223712 >> 172.21.15.16:0/3920223712 conn(0x7f1b7c09a200 msgr2=0x7f1b7c097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.569+0000 7f1b8377e640 1 -- 172.21.15.16:0/3920223712 shutdown_connections 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.569+0000 7f1b8377e640 1 -- 172.21.15.16:0/3920223712 wait complete. 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.570+0000 7f1b8377e640 1 Processor -- start 2024-04-23T19:09:59.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.570+0000 7f1b8377e640 1 -- start start 2024-04-23T19:09:59.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.570+0000 7f1b8377e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c121e40 0x7f1b7c1b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:59.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.570+0000 7f1b8377e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c0945f0 con 0x7f1b7c121e40 2024-04-23T19:09:59.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.570+0000 7f1b8151a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c121e40 0x7f1b7c1b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:59.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.570+0000 7f1b8151a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c121e40 0x7f1b7c1b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56720/0 (socket says 172.21.15.16:56720) 2024-04-23T19:09:59.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.570+0000 7f1b8151a640 1 -- 172.21.15.16:0/189860620 learned_addr learned my addr 172.21.15.16:0/189860620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:59.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.571+0000 7f1b8151a640 1 -- 172.21.15.16:0/189860620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c0730f0 con 0x7f1b7c121e40 2024-04-23T19:09:59.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.571+0000 7f1b8151a640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c121e40 0x7f1b7c1b0880 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009840 tx=0x7f1b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.571+0000 7f1b71ffb640 1 -- 172.21.15.16:0/189860620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b78032770 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.571+0000 7f1b71ffb640 1 -- 172.21.15.16:0/189860620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b78032910 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.571+0000 7f1b71ffb640 1 -- 172.21.15.16:0/189860620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b78032c30 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.571+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c06e790 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.571+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1b0fa0 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.572+0000 7f1b71ffb640 1 -- 172.21.15.16:0/189860620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1b7804c060 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.572+0000 7f1b71ffb640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1b600385c0 0x7f1b6003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.572+0000 7f1b71ffb640 1 -- 172.21.15.16:0/189860620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3482+0+0 (secure 0 0 0) 0x7f1b7802f050 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.572+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c063660 con 0x7f1b7c121e40 2024-04-23T19:09:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.573+0000 7f1b80d19640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1b600385c0 0x7f1b6003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:59.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.573+0000 7f1b80d19640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1b600385c0 0x7f1b6003aa80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c007910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.577+0000 7f1b71ffb640 1 -- 172.21.15.16:0/189860620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b78040070 con 0x7f1b7c121e40 2024-04-23T19:09:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.810+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1b7c06ef40 con 0x7f1b7c121e40 2024-04-23T19:09:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.812+0000 7f1b71ffb640 1 -- 172.21.15.16:0/189860620 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+87797 (secure 0 0 0) 0x7f1b78039310 con 0x7f1b7c121e40 2024-04-23T19:09:59.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1b600385c0 msgr2=0x7f1b6003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1b600385c0 0x7f1b6003aa80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c007910 comp rx=0 tx=0).stop 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c121e40 msgr2=0x7f1b7c1b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c121e40 0x7f1b7c1b0880 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009840 tx=0x7f1b780046c0 comp rx=0 tx=0).stop 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 shutdown_connections 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1b600385c0 0x7f1b6003aa80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 --2- 172.21.15.16:0/189860620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c121e40 0x7f1b7c1b0880 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 >> 172.21.15.16:0/189860620 conn(0x7f1b7c09a200 msgr2=0x7f1b7c097ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 shutdown_connections 2024-04-23T19:09:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.814+0000 7f1b8377e640 1 -- 172.21.15.16:0/189860620 wait complete. 2024-04-23T19:09:59.825 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4252129218},{"type":"v1","addr":"172.21.15.63:6825","nonce":4252129218}]},"active_addr":"172.21.15.63:6825/4252129218","active_change":"2024-04-23T19:09:46.632606+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":3242415666}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":231217141}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1599257317}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":905478923}]}]} 2024-04-23T19:09:59.827 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-23T19:09:59.827 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-23T19:09:59.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T19:09:59.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.926+0000 7f04d3a31640 1 Processor -- start 2024-04-23T19:09:59.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.926+0000 7f04d3a31640 1 -- start start 2024-04-23T19:09:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d3a31640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04cc12c290 0x7f04cc133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d3a31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04cc09aa00 con 0x7f04cc12b7a0 2024-04-23T19:09:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d3a31640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f04cc0a20d0 con 0x7f04cc12c290 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d0fcc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04cc12c290 0x7f04cc133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d17cd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f04cc12b7a0 0x7f04cc12bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60468/0 (socket says 172.21.15.16:60468) 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d17cd640 1 -- 172.21.15.16:0/3619817752 learned_addr learned my addr 172.21.15.16:0/3619817752 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d0fcc640 1 -- 172.21.15.16:0/3619817752 >> v1:172.21.15.16:6789/0 conn(0x7f04cc12b7a0 legacy=0x7f04cc12bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.927+0000 7f04d0fcc640 1 -- 172.21.15.16:0/3619817752 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc09af50 con 0x7f04cc12c290 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.928+0000 7f04d0fcc640 1 --2- 172.21.15.16:0/3619817752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12c290 0x7f04cc133fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009720 tx=0x7f04c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705b22621cf1d1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.928+0000 7f04cbfff640 1 -- 172.21.15.16:0/3619817752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c00316d0 con 0x7f04cc12c290 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.928+0000 7f04cbfff640 1 -- 172.21.15.16:0/3619817752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c0037070 con 0x7f04cc12c290 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.928+0000 7f04cbfff640 1 -- 172.21.15.16:0/3619817752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c00319c0 con 0x7f04cc12c290 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 -- 172.21.15.16:0/3619817752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12c290 msgr2=0x7f04cc133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:09:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 --2- 172.21.15.16:0/3619817752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12c290 0x7f04cc133fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009720 tx=0x7f04c002b840 comp rx=0 tx=0).stop 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 -- 172.21.15.16:0/3619817752 shutdown_connections 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 --2- 172.21.15.16:0/3619817752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12c290 0x7f04cc133fc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 -- 172.21.15.16:0/3619817752 >> 172.21.15.16:0/3619817752 conn(0x7f04cc0a15b0 msgr2=0x7f04cc0a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 -- 172.21.15.16:0/3619817752 shutdown_connections 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 -- 172.21.15.16:0/3619817752 wait complete. 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.929+0000 7f04d3a31640 1 Processor -- start 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d3a31640 1 -- start start 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d3a31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12b7a0 0x7f04cc1c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d3a31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc134500 con 0x7f04cc12b7a0 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d17cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12b7a0 0x7f04cc1c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d17cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12b7a0 0x7f04cc1c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56738/0 (socket says 172.21.15.16:56738) 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d17cd640 1 -- 172.21.15.16:0/3301755552 learned_addr learned my addr 172.21.15.16:0/3301755552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d17cd640 1 -- 172.21.15.16:0/3301755552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc06da40 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d17cd640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12b7a0 0x7f04cc1c3950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04c400c4a0 tx=0x7f04c400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04c9ffb640 1 -- 172.21.15.16:0/3301755552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c4017070 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04c9ffb640 1 -- 172.21.15.16:0/3301755552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c4005c10 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc133690 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.930+0000 7f04c9ffb640 1 -- 172.21.15.16:0/3301755552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c4007070 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.931+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc06e8b0 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.931+0000 7f04c9ffb640 1 -- 172.21.15.16:0/3301755552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f04c401a070 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.931+0000 7f04c9ffb640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f04b80385c0 0x7f04b803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.931+0000 7f04c9ffb640 1 -- 172.21.15.16:0/3301755552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3482+0+0 (secure 0 0 0) 0x7f04c404a880 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.932+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04cc06e6c0 con 0x7f04cc12b7a0 2024-04-23T19:09:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.932+0000 7f04d0fcc640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f04b80385c0 0x7f04b803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:09:59.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.932+0000 7f04d0fcc640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f04b80385c0 0x7f04b803aa80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04c0036000 tx=0x7f04c0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:09:59.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:09:59.937+0000 7f04c9ffb640 1 -- 172.21.15.16:0/3301755552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04c4007210 con 0x7f04cc12b7a0 2024-04-23T19:10:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.125+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04cc063660 con 0x7f04cc12b7a0 2024-04-23T19:10:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.126+0000 7f04c9ffb640 1 -- 172.21.15.16:0/3301755552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+9722 (secure 0 0 0) 0x7f04c4012400 con 0x7f04cc12b7a0 2024-04-23T19:10:00.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:10:00.127 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"fsid":"f80ae34c-e526-409e-90c3-45f2fcaa83f6","created":"2024-04-23T19:09:42.236475+0000","modified":"2024-04-23T19:09:59.439871+0000","last_up_change":"2024-04-23T19:09:51.428209+0000","last_in_change":"2024-04-23T19:09:43.346338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T19:09:50.651043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T19:09:57.875436+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8925ca3-b81e-4e90-9554-7a26a6b9ee13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6801","nonce":2262294208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6803","nonce":2262294208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6807","nonce":2262294208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6805","nonce":2262294208}]},"public_addr":"172.21.15.16:6801/2262294208","cluster_addr":"172.21.15.16:6803/2262294208","heartbeat_back_addr":"172.21.15.16:6807/2262294208","heartbeat_front_addr":"172.21.15.16:6805/2262294208","state":["exists","up"]},{"osd":1,"uuid":"6bf01767-14ca-4771-86ec-2369b6eefd4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6801","nonce":3216879695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6803","nonce":3216879695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6807","nonce":3216879695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6805","nonce":3216879695}]},"public_addr":"172.21.15.63:6801/3216879695","cluster_addr":"172.21.15.63:6803/3216879695","heartbeat_back_addr":"172.21.15.63:6807/3216879695","heartbeat_front_addr":"172.21.15.63:6805/3216879695","state":["exists","up"]},{"osd":2,"uuid":"3a1dcf6d-e482-4751-babc-9b0c896a3b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6809","nonce":753864785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6811","nonce":753864785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6815","nonce":753864785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6813","nonce":753864785}]},"public_addr":"172.21.15.63:6809/753864785","cluster_addr":"172.21.15.63:6811/753864785","heartbeat_back_addr":"172.21.15.63:6815/753864785","heartbeat_front_addr":"172.21.15.63:6813/753864785","state":["exists","up"]},{"osd":3,"uuid":"534b421f-9079-4429-ac83-791f6222a368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6817","nonce":1289558434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6819","nonce":1289558434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6823","nonce":1289558434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6821","nonce":1289558434}]},"public_addr":"172.21.15.63:6817/1289558434","cluster_addr":"172.21.15.63:6819/1289558434","heartbeat_back_addr":"172.21.15.63:6823/1289558434","heartbeat_front_addr":"172.21.15.63:6821/1289558434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.387629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.445168+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.449933+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.519415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f04b80385c0 msgr2=0x7f04b803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f04b80385c0 0x7f04b803aa80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04c0036000 tx=0x7f04c0008030 comp rx=0 tx=0).stop 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12b7a0 msgr2=0x7f04cc1c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12b7a0 0x7f04cc1c3950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04c400c4a0 tx=0x7f04c400c960 comp rx=0 tx=0).stop 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 shutdown_connections 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f04b80385c0 0x7f04b803aa80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 --2- 172.21.15.16:0/3301755552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc12b7a0 0x7f04cc1c3950 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 >> 172.21.15.16:0/3301755552 conn(0x7f04cc0a15b0 msgr2=0x7f04cc12a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 shutdown_connections 2024-04-23T19:10:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.128+0000 7f04d3a31640 1 -- 172.21.15.16:0/3301755552 wait complete. 2024-04-23T19:10:00.139 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-23T19:10:00.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T19:10:00.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.240+0000 7f54242d3640 1 Processor -- start 2024-04-23T19:10:00.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.240+0000 7f54242d3640 1 -- start start 2024-04-23T19:10:00.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.240+0000 7f54242d3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f541c096400 0x7f541c131d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.240+0000 7f54242d3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f541c095d90 con 0x7f541c12cda0 2024-04-23T19:10:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.240+0000 7f54242d3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f541c12c9e0 con 0x7f541c096400 2024-04-23T19:10:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.240+0000 7f542186e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f541c096400 0x7f541c131d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.240+0000 7f542206f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f541c12cda0 0x7f541c12d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60484/0 (socket says 172.21.15.16:60484) 2024-04-23T19:10:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.241+0000 7f542206f640 1 -- 172.21.15.16:0/3880022219 learned_addr learned my addr 172.21.15.16:0/3880022219 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.241+0000 7f542186e640 1 -- 172.21.15.16:0/3880022219 >> v1:172.21.15.16:6789/0 conn(0x7f541c12cda0 legacy=0x7f541c12d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.241+0000 7f542186e640 1 -- 172.21.15.16:0/3880022219 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c12d890 con 0x7f541c096400 2024-04-23T19:10:00.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.241+0000 7f542186e640 1 --2- 172.21.15.16:0/3880022219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c131d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f540c009720 tx=0x7f540c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99124d771cf0f39a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f542106d640 1 -- 172.21.15.16:0/3880022219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f540c0316d0 con 0x7f541c096400 2024-04-23T19:10:00.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f542106d640 1 -- 172.21.15.16:0/3880022219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f540c037070 con 0x7f541c096400 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f542106d640 1 -- 172.21.15.16:0/3880022219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f540c0319c0 con 0x7f541c096400 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f54242d3640 1 -- 172.21.15.16:0/3880022219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 msgr2=0x7f541c131d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f54242d3640 1 --2- 172.21.15.16:0/3880022219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c131d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f540c009720 tx=0x7f540c02b840 comp rx=0 tx=0).stop 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f54242d3640 1 -- 172.21.15.16:0/3880022219 shutdown_connections 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f54242d3640 1 --2- 172.21.15.16:0/3880022219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c131d60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f54242d3640 1 -- 172.21.15.16:0/3880022219 >> 172.21.15.16:0/3880022219 conn(0x7f541c09a200 msgr2=0x7f541c097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f54242d3640 1 -- 172.21.15.16:0/3880022219 shutdown_connections 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.242+0000 7f54242d3640 1 -- 172.21.15.16:0/3880022219 wait complete. 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.243+0000 7f54242d3640 1 Processor -- start 2024-04-23T19:10:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.243+0000 7f54242d3640 1 -- start start 2024-04-23T19:10:00.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.243+0000 7f54242d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c12dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.243+0000 7f54242d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f541c096cf0 con 0x7f541c096400 2024-04-23T19:10:00.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f542206f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c12dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f542206f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c12dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56758/0 (socket says 172.21.15.16:56758) 2024-04-23T19:10:00.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f542206f640 1 -- 172.21.15.16:0/971364286 learned_addr learned my addr 172.21.15.16:0/971364286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f542206f640 1 -- 172.21.15.16:0/971364286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c093b20 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f542206f640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c12dd00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5418005880 tx=0x7f541800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f5412ffd640 1 -- 172.21.15.16:0/971364286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5418017070 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f5412ffd640 1 -- 172.21.15.16:0/971364286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5418005c10 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c131260 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f5412ffd640 1 -- 172.21.15.16:0/971364286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5418007070 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.244+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c12e240 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.245+0000 7f5412ffd640 1 -- 172.21.15.16:0/971364286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f541801a070 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.245+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c063660 con 0x7f541c096400 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.245+0000 7f5412ffd640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f54000385c0 0x7f540003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.245+0000 7f5412ffd640 1 -- 172.21.15.16:0/971364286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3482+0+0 (secure 0 0 0) 0x7f541804a910 con 0x7f541c096400 2024-04-23T19:10:00.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.246+0000 7f542186e640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f54000385c0 0x7f540003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.246+0000 7f542186e640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f54000385c0 0x7f540003aa80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f540c036000 tx=0x7f540c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.250+0000 7f5412ffd640 1 -- 172.21.15.16:0/971364286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5418012400 con 0x7f541c096400 2024-04-23T19:10:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.434+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f541c067f20 con 0x7f541c096400 2024-04-23T19:10:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.435+0000 7f5412ffd640 1 -- 172.21.15.16:0/971364286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+9722 (secure 0 0 0) 0x7f541801a360 con 0x7f541c096400 2024-04-23T19:10:00.435 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:10:00.436 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"fsid":"f80ae34c-e526-409e-90c3-45f2fcaa83f6","created":"2024-04-23T19:09:42.236475+0000","modified":"2024-04-23T19:09:59.439871+0000","last_up_change":"2024-04-23T19:09:51.428209+0000","last_in_change":"2024-04-23T19:09:43.346338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T19:09:50.651043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T19:09:57.875436+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8925ca3-b81e-4e90-9554-7a26a6b9ee13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6801","nonce":2262294208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6803","nonce":2262294208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6807","nonce":2262294208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6805","nonce":2262294208}]},"public_addr":"172.21.15.16:6801/2262294208","cluster_addr":"172.21.15.16:6803/2262294208","heartbeat_back_addr":"172.21.15.16:6807/2262294208","heartbeat_front_addr":"172.21.15.16:6805/2262294208","state":["exists","up"]},{"osd":1,"uuid":"6bf01767-14ca-4771-86ec-2369b6eefd4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6801","nonce":3216879695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6803","nonce":3216879695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6807","nonce":3216879695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6805","nonce":3216879695}]},"public_addr":"172.21.15.63:6801/3216879695","cluster_addr":"172.21.15.63:6803/3216879695","heartbeat_back_addr":"172.21.15.63:6807/3216879695","heartbeat_front_addr":"172.21.15.63:6805/3216879695","state":["exists","up"]},{"osd":2,"uuid":"3a1dcf6d-e482-4751-babc-9b0c896a3b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6809","nonce":753864785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6811","nonce":753864785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6815","nonce":753864785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6813","nonce":753864785}]},"public_addr":"172.21.15.63:6809/753864785","cluster_addr":"172.21.15.63:6811/753864785","heartbeat_back_addr":"172.21.15.63:6815/753864785","heartbeat_front_addr":"172.21.15.63:6813/753864785","state":["exists","up"]},{"osd":3,"uuid":"534b421f-9079-4429-ac83-791f6222a368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6817","nonce":1289558434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6819","nonce":1289558434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6823","nonce":1289558434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6821","nonce":1289558434}]},"public_addr":"172.21.15.63:6817/1289558434","cluster_addr":"172.21.15.63:6819/1289558434","heartbeat_back_addr":"172.21.15.63:6823/1289558434","heartbeat_front_addr":"172.21.15.63:6821/1289558434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.387629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.445168+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.449933+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.519415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f54000385c0 msgr2=0x7f540003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f54000385c0 0x7f540003aa80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f540c036000 tx=0x7f540c008030 comp rx=0 tx=0).stop 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 msgr2=0x7f541c12dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c12dd00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5418005880 tx=0x7f541800c8e0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 shutdown_connections 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f54000385c0 0x7f540003aa80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 --2- 172.21.15.16:0/971364286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541c096400 0x7f541c12dd00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 >> 172.21.15.16:0/971364286 conn(0x7f541c09a200 msgr2=0x7f541c121e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 shutdown_connections 2024-04-23T19:10:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.437+0000 7f54242d3640 1 -- 172.21.15.16:0/971364286 wait complete. 2024-04-23T19:10:00.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-23T19:10:00.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-23T19:10:00.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-23T19:10:00.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-23T19:10:00.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7fb3ab4f2640 1 Processor -- start 2024-04-23T19:10:00.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7fc7e21c7640 1 Processor -- start 2024-04-23T19:10:00.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7f2196752640 1 Processor -- start 2024-04-23T19:10:00.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7fc7e21c7640 1 -- start start 2024-04-23T19:10:00.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7f2196752640 1 -- start start 2024-04-23T19:10:00.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7efdfee60640 1 Processor -- start 2024-04-23T19:10:00.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7fb3ab4f2640 1 -- start start 2024-04-23T19:10:00.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.556+0000 7efdfee60640 1 -- start start 2024-04-23T19:10:00.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fc7e21c7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7dc12c290 0x7fc7dc133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fc7e21c7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7dc09aa00 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fc7e21c7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc7dc0a20d0 con 0x7fc7dc12c290 2024-04-23T19:10:00.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7f2196752640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f219012cda0 0x7f219012d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fb3ab4f2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3a41287f0 0x7fb3a4128bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7efdfee60640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdf8093420 0x7efdf8093800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fb3ab4f2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a406de40 con 0x7fb3a4129110 2024-04-23T19:10:00.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7f2196752640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2190096c90 con 0x7f219012d6c0 2024-04-23T19:10:00.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7f2196752640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f219012c9e0 con 0x7f219012cda0 2024-04-23T19:10:00.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7efdfee60640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf8096620 con 0x7efdf812b150 2024-04-23T19:10:00.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7efdfee60640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7efdf806d2b0 con 0x7efdf8093420 2024-04-23T19:10:00.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7f218ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f219012cda0 0x7f219012d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7f218f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f219012d6c0 0x7f2190094280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60498/0 (socket says 172.21.15.16:60498) 2024-04-23T19:10:00.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7f218f7fe640 1 -- 172.21.15.16:0/1723932747 learned_addr learned my addr 172.21.15.16:0/1723932747 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:00.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fc7daffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7dc12c290 0x7fc7dc133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fc7daffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7dc12c290 0x7fc7dc133fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56762/0 (socket says 172.21.15.16:56762) 2024-04-23T19:10:00.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fc7daffd640 1 -- 172.21.15.16:0/1565386402 learned_addr learned my addr 172.21.15.16:0/1565386402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fb3ab4f2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a41285d0 con 0x7fb3a41287f0 2024-04-23T19:10:00.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fc7db7fe640 1 --1- 172.21.15.16:0/1565386402 >> v1:172.21.15.16:6789/0 conn(0x7fc7dc12b7a0 0x7fc7dc12bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60488/0 (socket says 172.21.15.16:60488) 2024-04-23T19:10:00.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fb3a928e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3a41287f0 0x7fb3a4128bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fb3a928e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3a41287f0 0x7fb3a4128bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56782/0 (socket says 172.21.15.16:56782) 2024-04-23T19:10:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7efdfcbfc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdf8093420 0x7efdf8093800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7efdfcbfc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdf8093420 0x7efdf8093800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56790/0 (socket says 172.21.15.16:56790) 2024-04-23T19:10:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7efdfcbfc640 1 -- 172.21.15.16:0/1655788728 learned_addr learned my addr 172.21.15.16:0/1655788728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7fb3a928e640 1 -- 172.21.15.16:0/104842621 learned_addr learned my addr 172.21.15.16:0/104842621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.557+0000 7f218ffff640 1 -- 172.21.15.16:0/1723932747 >> v1:172.21.15.16:6789/0 conn(0x7f219012d6c0 legacy=0x7f2190094280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7efdfcbfc640 1 -- 172.21.15.16:0/1655788728 >> v1:172.21.15.16:6789/0 conn(0x7efdf812b150 legacy=0x7efdf812b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T19:10:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7f218ffff640 1 -- 172.21.15.16:0/1723932747 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2190095d90 con 0x7f219012cda0 2024-04-23T19:10:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7fc7da7fc640 1 -- 172.21.15.16:0/1565386402 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907828662 0 0) 0x7fc7dc09aa00 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7fc7da7fc640 1 -- 172.21.15.16:0/1565386402 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7c4003520 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7efdfcbfc640 1 -- 172.21.15.16:0/1655788728 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf8072e50 con 0x7efdf8093420 2024-04-23T19:10:00.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7fb3a928e640 1 -- 172.21.15.16:0/104842621 >> v1:172.21.15.16:6789/0 conn(0x7fb3a4129110 legacy=0x7fb3a4121e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7fb3a928e640 1 -- 172.21.15.16:0/104842621 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a406d340 con 0x7fb3a41287f0 2024-04-23T19:10:00.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7fc7daffd640 1 -- 172.21.15.16:0/1565386402 >> v1:172.21.15.16:6789/0 conn(0x7fc7dc12b7a0 legacy=0x7fc7dc12bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7fc7daffd640 1 -- 172.21.15.16:0/1565386402 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc09af50 con 0x7fc7dc12c290 2024-04-23T19:10:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.558+0000 7efdfcbfc640 1 --2- 172.21.15.16:0/1655788728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf8093420 0x7efdf8093800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efdec0097f0 tx=0x7efdec02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a94ac12e3082578 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7efdf77fe640 1 -- 172.21.15.16:0/1655788728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec035020 con 0x7efdf8093420 2024-04-23T19:10:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7f218ffff640 1 --2- 172.21.15.16:0/1723932747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f219012d180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2180009720 tx=0x7f218002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f7d0cdc17e894a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7efdf77fe640 1 -- 172.21.15.16:0/1655788728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec0317c0 con 0x7efdf8093420 2024-04-23T19:10:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7efdf77fe640 1 -- 172.21.15.16:0/1655788728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec031ac0 con 0x7efdf8093420 2024-04-23T19:10:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fc7daffd640 1 --2- 172.21.15.16:0/1565386402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12c290 0x7fc7dc133fc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00097f0 tx=0x7fc7d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeec3e48f42d2350 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7f218effd640 1 -- 172.21.15.16:0/1723932747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21800316d0 con 0x7f219012cda0 2024-04-23T19:10:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7f218effd640 1 -- 172.21.15.16:0/1723932747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180037070 con 0x7f219012cda0 2024-04-23T19:10:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7f218effd640 1 -- 172.21.15.16:0/1723932747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21800319c0 con 0x7f219012cda0 2024-04-23T19:10:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fc7da7fc640 1 -- 172.21.15.16:0/1565386402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7d0035020 con 0x7fc7dc12c290 2024-04-23T19:10:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fc7da7fc640 1 -- 172.21.15.16:0/1565386402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d00317a0 con 0x7fc7dc12c290 2024-04-23T19:10:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fb3a928e640 1 --2- 172.21.15.16:0/104842621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a4128bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097f0 tx=0x7fb3a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b11a14f18a50f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fc7da7fc640 1 -- 172.21.15.16:0/1565386402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7d0031aa0 con 0x7fc7dc12c290 2024-04-23T19:10:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fb39bfff640 1 -- 172.21.15.16:0/104842621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3a0035020 con 0x7fb3a41287f0 2024-04-23T19:10:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fb39bfff640 1 -- 172.21.15.16:0/104842621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a00317a0 con 0x7fb3a41287f0 2024-04-23T19:10:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.559+0000 7fb39bfff640 1 -- 172.21.15.16:0/104842621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3a0031aa0 con 0x7fb3a41287f0 2024-04-23T19:10:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7efdfee60640 1 -- 172.21.15.16:0/1655788728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf8093420 msgr2=0x7efdf8093800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7efdfee60640 1 --2- 172.21.15.16:0/1655788728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf8093420 0x7efdf8093800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efdec0097f0 tx=0x7efdec02b930 comp rx=0 tx=0).stop 2024-04-23T19:10:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7efdfee60640 1 -- 172.21.15.16:0/1655788728 shutdown_connections 2024-04-23T19:10:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7efdfee60640 1 --2- 172.21.15.16:0/1655788728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf8093420 0x7efdf8093800 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7efdfee60640 1 -- 172.21.15.16:0/1655788728 >> 172.21.15.16:0/1655788728 conn(0x7efdf809a200 msgr2=0x7efdf8097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7efdfee60640 1 -- 172.21.15.16:0/1655788728 shutdown_connections 2024-04-23T19:10:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7efdfee60640 1 -- 172.21.15.16:0/1655788728 wait complete. 2024-04-23T19:10:00.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7f2196752640 1 -- 172.21.15.16:0/1723932747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 msgr2=0x7f219012d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fc7e21c7640 1 -- 172.21.15.16:0/1565386402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12c290 msgr2=0x7fc7dc133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7f2196752640 1 --2- 172.21.15.16:0/1723932747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f219012d180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2180009720 tx=0x7f218002b840 comp rx=0 tx=0).stop 2024-04-23T19:10:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/1565386402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12c290 0x7fc7dc133fc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00097f0 tx=0x7fc7d002b910 comp rx=0 tx=0).stop 2024-04-23T19:10:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7f2196752640 1 -- 172.21.15.16:0/1723932747 shutdown_connections 2024-04-23T19:10:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7f2196752640 1 --2- 172.21.15.16:0/1723932747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f219012d180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7f2196752640 1 -- 172.21.15.16:0/1723932747 >> 172.21.15.16:0/1723932747 conn(0x7f219009a200 msgr2=0x7f2190097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fc7e21c7640 1 -- 172.21.15.16:0/1565386402 shutdown_connections 2024-04-23T19:10:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/1565386402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12c290 0x7fc7dc133fc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fc7e21c7640 1 -- 172.21.15.16:0/1565386402 >> 172.21.15.16:0/1565386402 conn(0x7fc7dc0a15b0 msgr2=0x7fc7dc0a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7f2196752640 1 -- 172.21.15.16:0/1723932747 shutdown_connections 2024-04-23T19:10:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7f2196752640 1 -- 172.21.15.16:0/1723932747 wait complete. 2024-04-23T19:10:00.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fc7e21c7640 1 -- 172.21.15.16:0/1565386402 shutdown_connections 2024-04-23T19:10:00.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fc7e21c7640 1 -- 172.21.15.16:0/1565386402 wait complete. 2024-04-23T19:10:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fb3ab4f2640 1 -- 172.21.15.16:0/104842621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 msgr2=0x7fb3a4128bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fb3ab4f2640 1 --2- 172.21.15.16:0/104842621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a4128bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097f0 tx=0x7fb3a002b910 comp rx=0 tx=0).stop 2024-04-23T19:10:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fb3ab4f2640 1 -- 172.21.15.16:0/104842621 shutdown_connections 2024-04-23T19:10:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fb3ab4f2640 1 --2- 172.21.15.16:0/104842621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a4128bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fb3ab4f2640 1 -- 172.21.15.16:0/104842621 >> 172.21.15.16:0/104842621 conn(0x7fb3a409a200 msgr2=0x7fb3a4097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.560+0000 7fb3ab4f2640 1 -- 172.21.15.16:0/104842621 shutdown_connections 2024-04-23T19:10:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7fb3ab4f2640 1 -- 172.21.15.16:0/104842621 wait complete. 2024-04-23T19:10:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7efdfee60640 1 Processor -- start 2024-04-23T19:10:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7efdfee60640 1 -- start start 2024-04-23T19:10:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7fc7e21c7640 1 Processor -- start 2024-04-23T19:10:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7efdfee60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf812b150 0x7efdf8124ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7efdfee60640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8093d40 con 0x7efdf812b150 2024-04-23T19:10:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7f2196752640 1 Processor -- start 2024-04-23T19:10:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7fc7e21c7640 1 -- start start 2024-04-23T19:10:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7efdfcbfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf812b150 0x7efdf8124ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7efdfcbfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf812b150 0x7efdf8124ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56804/0 (socket says 172.21.15.16:56804) 2024-04-23T19:10:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.561+0000 7efdfcbfc640 1 -- 172.21.15.16:0/3712719464 learned_addr learned my addr 172.21.15.16:0/3712719464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7f2196752640 1 -- start start 2024-04-23T19:10:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fb3ab4f2640 1 Processor -- start 2024-04-23T19:10:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fb3ab4f2640 1 -- start start 2024-04-23T19:10:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fc7e21c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12b7a0 0x7fc7dc1c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7efdfcbfc640 1 -- 172.21.15.16:0/3712719464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf8128c60 con 0x7efdf812b150 2024-04-23T19:10:00.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fc7e21c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc134500 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fc7db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12b7a0 0x7fc7dc1c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7f2196752640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f2190093640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7f2196752640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21900965f0 con 0x7f219012cda0 2024-04-23T19:10:00.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fc7db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12b7a0 0x7fc7dc1c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56810/0 (socket says 172.21.15.16:56810) 2024-04-23T19:10:00.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fc7db7fe640 1 -- 172.21.15.16:0/2334058286 learned_addr learned my addr 172.21.15.16:0/2334058286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fb3ab4f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a41c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fb3ab4f2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40946e0 con 0x7fb3a41287f0 2024-04-23T19:10:00.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7efdfcbfc640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf812b150 0x7efdf8124ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdec02b970 tx=0x7efdec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7f218ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f2190093640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7f218ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f2190093640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56812/0 (socket says 172.21.15.16:56812) 2024-04-23T19:10:00.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7f218ffff640 1 -- 172.21.15.16:0/2229147265 learned_addr learned my addr 172.21.15.16:0/2229147265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fb3a928e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a41c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fb3a928e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a41c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56820/0 (socket says 172.21.15.16:56820) 2024-04-23T19:10:00.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fb3a928e640 1 -- 172.21.15.16:0/3527134831 learned_addr learned my addr 172.21.15.16:0/3527134831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7fc7db7fe640 1 -- 172.21.15.16:0/2334058286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc133310 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.562+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec031aa0 con 0x7efdf812b150 2024-04-23T19:10:00.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec045430 con 0x7efdf812b150 2024-04-23T19:10:00.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec045730 con 0x7efdf812b150 2024-04-23T19:10:00.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7efdfee60640 1 -- 172.21.15.16:0/3712719464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf8127ca0 con 0x7efdf812b150 2024-04-23T19:10:00.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7efdfee60640 1 -- 172.21.15.16:0/3712719464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf8135310 con 0x7efdf812b150 2024-04-23T19:10:00.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7f218ffff640 1 -- 172.21.15.16:0/2229147265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219006eda0 con 0x7f219012cda0 2024-04-23T19:10:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fb3a928e640 1 -- 172.21.15.16:0/3527134831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a40723f0 con 0x7fb3a41287f0 2024-04-23T19:10:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7f218ffff640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f2190093640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c00 tx=0x7f21800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fb3a928e640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a41c1c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097f0 tx=0x7fb3a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fc7db7fe640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12b7a0 0x7fc7dc1c3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c480 tx=0x7fc7cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7cc018070 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7cc0073b0 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7cc0047e0 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc1339f0 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.563+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc1c3e80 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdf8126a70 con 0x7efdf812b150 2024-04-23T19:10:00.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7f218cff9640 1 -- 172.21.15.16:0/2229147265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2180032770 con 0x7f219012cda0 2024-04-23T19:10:00.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3a003ad00 con 0x7fb3a41287f0 2024-04-23T19:10:00.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219006e4a0 con 0x7f219012cda0 2024-04-23T19:10:00.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7fb3ab4f2640 1 -- 172.21.15.16:0/3527134831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a4072cf0 con 0x7fb3a41287f0 2024-04-23T19:10:00.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2190070aa0 con 0x7f219012cda0 2024-04-23T19:10:00.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc7cc01b070 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7fb3ab4f2640 1 -- 172.21.15.16:0/3527134831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a41c2190 con 0x7fb3a41287f0 2024-04-23T19:10:00.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7dc12b2f0 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7efdec038020 con 0x7efdf812b150 2024-04-23T19:10:00.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7efdf57fa640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efddc038610 0x7efddc03aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fc7c3fff640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc7c80385c0 0x7fc7c803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7fc7cc04a950 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fc7c3fff640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fc7c803dd40 0x7fc7c8040180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7cc017ec0 con 0x7fc7c803dd40 2024-04-23T19:10:00.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fc7cc01c8d0 con 0x7fc7dc12b7a0 2024-04-23T19:10:00.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a004d450 con 0x7fb3a41287f0 2024-04-23T19:10:00.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7efdec04e440 con 0x7efdf812b150 2024-04-23T19:10:00.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7efdf57fa640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efddc03e5c0 0x7efddc040a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efddc041130 con 0x7efddc03e5c0 2024-04-23T19:10:00.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fc7daffd640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc7c80385c0 0x7fc7c803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7efdfd3fd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efddc03e5c0 0x7efddc040a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fc7dbfff640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fc7c803dd40 0x7fc7c8040180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3a004d730 con 0x7fb3a41287f0 2024-04-23T19:10:00.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7efdf7fff640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efddc038610 0x7efddc03aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.565+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3a4127b90 con 0x7fb3a41287f0 2024-04-23T19:10:00.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7efdec0303e0 con 0x7efdf812b150 2024-04-23T19:10:00.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7efdfd3fd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efddc03e5c0 0x7efddc040a60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7f218cff9640 1 -- 172.21.15.16:0/2229147265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180005990 con 0x7f219012cda0 2024-04-23T19:10:00.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.564+0000 7f218cff9640 1 -- 172.21.15.16:0/2229147265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2180005c70 con 0x7f219012cda0 2024-04-23T19:10:00.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7f218cff9640 1 -- 172.21.15.16:0/2229147265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f218004c060 con 0x7f219012cda0 2024-04-23T19:10:00.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fc7dbfff640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fc7c803dd40 0x7fc7c8040180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fc7daffd640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc7c80385c0 0x7fc7c803aa80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0036000 tx=0x7fc7d0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb3a004b060 con 0x7fb3a41287f0 2024-04-23T19:10:00.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7f218cff9640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f217c038610 0x7f217c03aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7f218cff9640 1 -- 172.21.15.16:0/2229147265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7f218002f050 con 0x7f219012cda0 2024-04-23T19:10:00.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7f218f7fe640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f217c038610 0x7f217c03aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fb399ffb640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb388038750 0x7fb38803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7fb3a00393b0 con 0x7fb3a41287f0 2024-04-23T19:10:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fb3a8a8d640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb388038750 0x7fb38803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7f2196752640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2170001650 0x7f2170003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fb399ffb640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fb38803e6f0 0x7fb388040b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb388041260 con 0x7fb38803e6f0 2024-04-23T19:10:00.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.566+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2170006b00 con 0x7f2170001650 2024-04-23T19:10:00.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fb3a0044590 con 0x7fb3a41287f0 2024-04-23T19:10:00.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7f218f7fe640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f217c038610 0x7f217c03aad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2184000f60 tx=0x7f2184007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7fb3a9a8f640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fb38803e6f0 0x7fb388040b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7f2194cef640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2170001650 0x7f2170003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efddc041130 con 0x7efddc03e5c0 2024-04-23T19:10:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7cc017ec0 con 0x7fc7c803dd40 2024-04-23T19:10:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7efdf7fff640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efddc038610 0x7efddc03aad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efde4000f60 tx=0x7efde4007a30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7fb3a8a8d640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb388038750 0x7fb38803ac10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb394000f60 tx=0x7fb394007a30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7fb3a9a8f640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fb38803e6f0 0x7fb388040b90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.567+0000 7f2194cef640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2170001650 0x7f2170003b10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.568+0000 7f218cff9640 1 -- 172.21.15.16:0/2229147265 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2170006b00 con 0x7f2170001650 2024-04-23T19:10:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.568+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb388041260 con 0x7fb38803e6f0 2024-04-23T19:10:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb3a408ca80 con 0x7fb38803e6f0 2024-04-23T19:10:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2170005be0 con 0x7f2170001650 2024-04-23T19:10:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7fb399ffb640 1 -- 172.21.15.16:0/3527134831 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb3a408ca80 con 0x7fb38803e6f0 2024-04-23T19:10:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7f218cff9640 1 -- 172.21.15.16:0/2229147265 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2170005be0 con 0x7f2170001650 2024-04-23T19:10:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2170001650 msgr2=0x7f2170003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7f2196752640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2170001650 0x7f2170003b10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fb38803e6f0 msgr2=0x7fb388040b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7fb38f7fe640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fb38803e6f0 0x7fb388040b90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f217c038610 msgr2=0x7f217c03aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.583+0000 7f2196752640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f217c038610 0x7f217c03aad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2184000f60 tx=0x7f2184007a80 comp rx=0 tx=0).stop 2024-04-23T19:10:00.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 msgr2=0x7f2190093640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f2190093640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c00 tx=0x7f21800046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb388038750 msgr2=0x7fb38803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 shutdown_connections 2024-04-23T19:10:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2170001650 0x7f2170003b10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f217c038610 0x7f217c03aad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 --2- 172.21.15.16:0/2229147265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f219012cda0 0x7f2190093640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb388038750 0x7fb38803ac10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb394000f60 tx=0x7fb394007a30 comp rx=0 tx=0).stop 2024-04-23T19:10:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 >> 172.21.15.16:0/2229147265 conn(0x7f219009a200 msgr2=0x7f2190122970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 msgr2=0x7fb3a41c1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a41c1c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097f0 tx=0x7fb3a00046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 shutdown_connections 2024-04-23T19:10:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fb38803e6f0 0x7fb388040b90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb388038750 0x7fb38803ac10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 --2- 172.21.15.16:0/3527134831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a41287f0 0x7fb3a41c1c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 >> 172.21.15.16:0/3527134831 conn(0x7fb3a409a200 msgr2=0x7fb3a4122e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 shutdown_connections 2024-04-23T19:10:00.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 shutdown_connections 2024-04-23T19:10:00.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7f2196752640 1 -- 172.21.15.16:0/2229147265 wait complete. 2024-04-23T19:10:00.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.584+0000 7fb38f7fe640 1 -- 172.21.15.16:0/3527134831 wait complete. 2024-04-23T19:10:00.611 INFO:teuthology.orchestra.run.smithi016.stdout:38654705668 2024-04-23T19:10:00.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T19:10:00.612 INFO:teuthology.orchestra.run.smithi016.stdout:42949672965 2024-04-23T19:10:00.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T19:10:00.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.601+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7dc12aff0 con 0x7fc7c803dd40 2024-04-23T19:10:00.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.601+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdf8127a90 con 0x7efddc03e5c0 2024-04-23T19:10:00.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.601+0000 7efdf57fa640 1 -- 172.21.15.16:0/3712719464 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efdf8127a90 con 0x7efddc03e5c0 2024-04-23T19:10:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.601+0000 7fc7c3fff640 1 -- 172.21.15.16:0/2334058286 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc7dc12aff0 con 0x7fc7c803dd40 2024-04-23T19:10:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efddc03e5c0 msgr2=0x7efddc040a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efddc03e5c0 0x7efddc040a60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fc7c803dd40 msgr2=0x7fc7c8040180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fc7c803dd40 0x7fc7c8040180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efddc038610 msgr2=0x7efddc03aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efddc038610 0x7efddc03aad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efde4000f60 tx=0x7efde4007a30 comp rx=0 tx=0).stop 2024-04-23T19:10:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf812b150 msgr2=0x7efdf8124ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf812b150 0x7efdf8124ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdec02b970 tx=0x7efdec0046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc7c80385c0 msgr2=0x7fc7c803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc7c80385c0 0x7fc7c803aa80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0036000 tx=0x7fc7d0008030 comp rx=0 tx=0).stop 2024-04-23T19:10:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12b7a0 msgr2=0x7fc7dc1c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12b7a0 0x7fc7dc1c3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c480 tx=0x7fc7cc00c940 comp rx=0 tx=0).stop 2024-04-23T19:10:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 shutdown_connections 2024-04-23T19:10:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efddc03e5c0 0x7efddc040a60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efddc038610 0x7efddc03aad0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 --2- 172.21.15.16:0/3712719464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf812b150 0x7efdf8124ee0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 >> 172.21.15.16:0/3712719464 conn(0x7efdf809a200 msgr2=0x7efdf80947d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 shutdown_connections 2024-04-23T19:10:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fc7c803dd40 0x7fc7c8040180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc7c80385c0 0x7fc7c803aa80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 --2- 172.21.15.16:0/2334058286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7dc12b7a0 0x7fc7dc1c3940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 shutdown_connections 2024-04-23T19:10:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 >> 172.21.15.16:0/2334058286 conn(0x7fc7dc0a15b0 msgr2=0x7fc7dc129f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7efde2ffd640 1 -- 172.21.15.16:0/3712719464 wait complete. 2024-04-23T19:10:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 shutdown_connections 2024-04-23T19:10:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.602+0000 7fc7e21c7640 1 -- 172.21.15.16:0/2334058286 wait complete. 2024-04-23T19:10:00.620 INFO:teuthology.orchestra.run.smithi016.stdout:42949672966 2024-04-23T19:10:00.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T19:10:00.621 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2024-04-23T19:10:00.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T19:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f647a6ca640 1 Processor -- start 2024-04-23T19:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f647a6ca640 1 -- start start 2024-04-23T19:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f647a6ca640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f647412b7a0 0x7f647412bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f647a6ca640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f647409aa00 con 0x7f647412c0c0 2024-04-23T19:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f647a6ca640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f64740a20d0 con 0x7f647412b7a0 2024-04-23T19:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f6473fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f647412b7a0 0x7f647412bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f64737fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f647412c0c0 0x7f647412eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60526/0 (socket says 172.21.15.16:60526) 2024-04-23T19:10:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.724+0000 7f64737fe640 1 -- 172.21.15.16:0/565673185 learned_addr learned my addr 172.21.15.16:0/565673185 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.725+0000 7f6473fff640 1 -- 172.21.15.16:0/565673185 >> v1:172.21.15.16:6789/0 conn(0x7f647412c0c0 legacy=0x7f647412eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.725+0000 7f6473fff640 1 -- 172.21.15.16:0/565673185 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647409af50 con 0x7f647412b7a0 2024-04-23T19:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.725+0000 7f6473fff640 1 --2- 172.21.15.16:0/565673185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f647412bb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6464009720 tx=0x7f646402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384061bf34a1fcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f6472ffd640 1 -- 172.21.15.16:0/565673185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64640316d0 con 0x7f647412b7a0 2024-04-23T19:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f6472ffd640 1 -- 172.21.15.16:0/565673185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6464037070 con 0x7f647412b7a0 2024-04-23T19:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f6472ffd640 1 -- 172.21.15.16:0/565673185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64640319c0 con 0x7f647412b7a0 2024-04-23T19:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f647a6ca640 1 -- 172.21.15.16:0/565673185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 msgr2=0x7f647412bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f647a6ca640 1 --2- 172.21.15.16:0/565673185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f647412bb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6464009720 tx=0x7f646402b840 comp rx=0 tx=0).stop 2024-04-23T19:10:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f647a6ca640 1 -- 172.21.15.16:0/565673185 shutdown_connections 2024-04-23T19:10:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f647a6ca640 1 --2- 172.21.15.16:0/565673185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f647412bb80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f647a6ca640 1 -- 172.21.15.16:0/565673185 >> 172.21.15.16:0/565673185 conn(0x7f64740a15b0 msgr2=0x7f64740a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.726+0000 7f647a6ca640 1 -- 172.21.15.16:0/565673185 shutdown_connections 2024-04-23T19:10:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.727+0000 7f647a6ca640 1 -- 172.21.15.16:0/565673185 wait complete. 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.727+0000 7f647a6ca640 1 Processor -- start 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.728+0000 7f647a6ca640 1 -- start start 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.728+0000 7f647a6ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f6474070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.728+0000 7f647a6ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f647412c500 con 0x7f647412b7a0 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.728+0000 7f6473fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f6474070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.728+0000 7f6473fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f6474070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56844/0 (socket says 172.21.15.16:56844) 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.728+0000 7f6473fff640 1 -- 172.21.15.16:0/3597859121 learned_addr learned my addr 172.21.15.16:0/3597859121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.729+0000 7f6473fff640 1 -- 172.21.15.16:0/3597859121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6474071e40 con 0x7f647412b7a0 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.729+0000 7f6473fff640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f6474070410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6464009720 tx=0x7f64640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.729+0000 7f6470ff9640 1 -- 172.21.15.16:0/3597859121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6464032770 con 0x7f647412b7a0 2024-04-23T19:10:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.729+0000 7f6470ff9640 1 -- 172.21.15.16:0/3597859121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6464032910 con 0x7f647412b7a0 2024-04-23T19:10:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.729+0000 7f6470ff9640 1 -- 172.21.15.16:0/3597859121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6464032bf0 con 0x7f647412b7a0 2024-04-23T19:10:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.729+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647408fad0 con 0x7f647412b7a0 2024-04-23T19:10:00.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.729+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6474070950 con 0x7f647412b7a0 2024-04-23T19:10:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.730+0000 7f6470ff9640 1 -- 172.21.15.16:0/3597859121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f646404c060 con 0x7f647412b7a0 2024-04-23T19:10:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.730+0000 7f6470ff9640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f64600385c0 0x7f646003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.730+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6474063660 con 0x7f647412b7a0 2024-04-23T19:10:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.730+0000 7f6470ff9640 1 -- 172.21.15.16:0/3597859121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7f646402f050 con 0x7f647412b7a0 2024-04-23T19:10:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.731+0000 7f64737fe640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f64600385c0 0x7f646003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.731+0000 7f64737fe640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f64600385c0 0x7f646003aa80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f6468007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.740+0000 7f6470ff9640 1 -- 172.21.15.16:0/3597859121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6464040070 con 0x7f647412b7a0 2024-04-23T19:10:00.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.766+0000 7feecbc06640 1 Processor -- start 2024-04-23T19:10:00.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.766+0000 7f8ebf544640 1 Processor -- start 2024-04-23T19:10:00.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7feecbc06640 1 -- start start 2024-04-23T19:10:00.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7f8ebf544640 1 -- start start 2024-04-23T19:10:00.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7feecbc06640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feec412e460 0x7feec4130850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7f8ebf544640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8eb812b7a0 0x7f8eb812bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7feecbc06640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec4096620 con 0x7feec4093420 2024-04-23T19:10:00.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7feecbc06640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7feec406d2b0 con 0x7feec412e460 2024-04-23T19:10:00.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7f8ebf544640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb809aa00 con 0x7f8eb812c0c0 2024-04-23T19:10:00.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7f8ebf544640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb80a20d0 con 0x7f8eb812b7a0 2024-04-23T19:10:00.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7f8ebd2e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8eb812b7a0 0x7f8eb812bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7feec91a1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feec412e460 0x7feec4130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7f8ebd2e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8eb812b7a0 0x7f8eb812bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56868/0 (socket says 172.21.15.16:56868) 2024-04-23T19:10:00.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7f8ebd2e0640 1 -- 172.21.15.16:0/1595112748 learned_addr learned my addr 172.21.15.16:0/1595112748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7feec99a2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feec4093420 0x7feec4093800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60534/0 (socket says 172.21.15.16:60534) 2024-04-23T19:10:00.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.767+0000 7feec99a2640 1 -- 172.21.15.16:0/4023631149 learned_addr learned my addr 172.21.15.16:0/4023631149 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:00.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.768+0000 7f8ebd2e0640 1 -- 172.21.15.16:0/1595112748 >> v1:172.21.15.16:6789/0 conn(0x7f8eb812c0c0 legacy=0x7f8eb812eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.768+0000 7feec91a1640 1 -- 172.21.15.16:0/4023631149 >> v1:172.21.15.16:6789/0 conn(0x7feec4093420 legacy=0x7feec4093800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.768+0000 7f8ebd2e0640 1 -- 172.21.15.16:0/1595112748 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb809af50 con 0x7f8eb812b7a0 2024-04-23T19:10:00.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.768+0000 7feec91a1640 1 -- 172.21.15.16:0/4023631149 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec4072e50 con 0x7feec412e460 2024-04-23T19:10:00.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7feec91a1640 1 --2- 172.21.15.16:0/4023631149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec412e460 0x7feec4130850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009720 tx=0x7feeb402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b14844eb3d4771 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7f8ebd2e0640 1 --2- 172.21.15.16:0/1595112748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb812bb80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40097f0 tx=0x7f8eb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e3fd324c7acabc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7feec89a0640 1 -- 172.21.15.16:0/4023631149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb40316d0 con 0x7feec412e460 2024-04-23T19:10:00.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7feec89a0640 1 -- 172.21.15.16:0/4023631149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb4037070 con 0x7feec412e460 2024-04-23T19:10:00.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7feec89a0640 1 -- 172.21.15.16:0/4023631149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb40319c0 con 0x7feec412e460 2024-04-23T19:10:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7f8eaffff640 1 -- 172.21.15.16:0/1595112748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4035020 con 0x7f8eb812b7a0 2024-04-23T19:10:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7f8eaffff640 1 -- 172.21.15.16:0/1595112748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb40317a0 con 0x7f8eb812b7a0 2024-04-23T19:10:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.769+0000 7fd180a91640 1 Processor -- start 2024-04-23T19:10:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7fd180a91640 1 -- start start 2024-04-23T19:10:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7feecbc06640 1 -- 172.21.15.16:0/4023631149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec412e460 msgr2=0x7feec4130850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7feecbc06640 1 --2- 172.21.15.16:0/4023631149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec412e460 0x7feec4130850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009720 tx=0x7feeb402b840 comp rx=0 tx=0).stop 2024-04-23T19:10:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8eaffff640 1 -- 172.21.15.16:0/1595112748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4035020 con 0x7f8eb812b7a0 2024-04-23T19:10:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7feecbc06640 1 -- 172.21.15.16:0/4023631149 shutdown_connections 2024-04-23T19:10:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7feecbc06640 1 --2- 172.21.15.16:0/4023631149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec412e460 0x7feec4130850 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7feecbc06640 1 -- 172.21.15.16:0/4023631149 >> 172.21.15.16:0/4023631149 conn(0x7feec409a200 msgr2=0x7feec4097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8ebf544640 1 -- 172.21.15.16:0/1595112748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 msgr2=0x7f8eb812bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8ebf544640 1 --2- 172.21.15.16:0/1595112748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb812bb80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40097f0 tx=0x7f8eb402b910 comp rx=0 tx=0).stop 2024-04-23T19:10:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8ebf544640 1 -- 172.21.15.16:0/1595112748 shutdown_connections 2024-04-23T19:10:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8ebf544640 1 --2- 172.21.15.16:0/1595112748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb812bb80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8ebf544640 1 -- 172.21.15.16:0/1595112748 >> 172.21.15.16:0/1595112748 conn(0x7f8eb80a15b0 msgr2=0x7f8eb80a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7feecbc06640 1 -- 172.21.15.16:0/4023631149 shutdown_connections 2024-04-23T19:10:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7feecbc06640 1 -- 172.21.15.16:0/4023631149 wait complete. 2024-04-23T19:10:00.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8ebf544640 1 -- 172.21.15.16:0/1595112748 shutdown_connections 2024-04-23T19:10:00.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7f8ebf544640 1 -- 172.21.15.16:0/1595112748 wait complete. 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7fd180a91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd17c12d440 0x7fd17c131e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7fd180a91640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17c0729e0 con 0x7fd17c12c950 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7fd180a91640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd17c09ed10 con 0x7fd17c12d440 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7fd17a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd17c12c950 0x7fd17c12cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60550/0 (socket says 172.21.15.16:60550) 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.770+0000 7fd17a59c640 1 -- 172.21.15.16:0/1572548299 learned_addr learned my addr 172.21.15.16:0/1572548299 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.771+0000 7fd173fff640 1 --2- 172.21.15.16:0/1572548299 >> v2:172.21.15.16:3300/0 conn(0x7fd17c12d440 0x7fd17c131e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.771+0000 7fd173fff640 1 -- 172.21.15.16:0/1572548299 >> v1:172.21.15.16:6789/0 conn(0x7fd17c12c950 legacy=0x7fd17c12cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.771+0000 7fd179d9b640 1 -- 172.21.15.16:0/1572548299 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920674183 0 0) 0x7fd17c0729e0 con 0x7fd17c12c950 2024-04-23T19:10:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.771+0000 7fd173fff640 1 -- 172.21.15.16:0/1572548299 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c09f270 con 0x7fd17c12d440 2024-04-23T19:10:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.771+0000 7f8ebf544640 1 Processor -- start 2024-04-23T19:10:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.771+0000 7f8ebf544640 1 -- start start 2024-04-23T19:10:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7feecbc06640 1 Processor -- start 2024-04-23T19:10:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7feecbc06640 1 -- start start 2024-04-23T19:10:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7f8ebf544640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb8070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7f8ebf544640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb812c500 con 0x7f8eb812b7a0 2024-04-23T19:10:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7f8ebd2e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb8070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7f8ebd2e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb8070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56882/0 (socket says 172.21.15.16:56882) 2024-04-23T19:10:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7f8ebd2e0640 1 -- 172.21.15.16:0/765176160 learned_addr learned my addr 172.21.15.16:0/765176160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7fd173fff640 1 --2- 172.21.15.16:0/1572548299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12d440 0x7fd17c131e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd164009720 tx=0x7fd16402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14ff025e902442d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7f8ebd2e0640 1 -- 172.21.15.16:0/765176160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb80730f0 con 0x7f8eb812b7a0 2024-04-23T19:10:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7fd179d9b640 1 -- 172.21.15.16:0/1572548299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1640316d0 con 0x7fd17c12d440 2024-04-23T19:10:00.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7feecbc06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4093420 0x7feec41298d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7fd179d9b640 1 -- 172.21.15.16:0/1572548299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164037070 con 0x7fd17c12d440 2024-04-23T19:10:00.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.772+0000 7fd179d9b640 1 -- 172.21.15.16:0/1572548299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1640319c0 con 0x7fd17c12d440 2024-04-23T19:10:00.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7feecbc06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feec4094280 con 0x7feec4093420 2024-04-23T19:10:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7f8ebd2e0640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb8070480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8eb402b950 tx=0x7f8eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7feec99a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4093420 0x7feec41298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7feec99a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4093420 0x7feec41298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56896/0 (socket says 172.21.15.16:56896) 2024-04-23T19:10:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7feec99a2640 1 -- 172.21.15.16:0/3627688664 learned_addr learned my addr 172.21.15.16:0/3627688664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7feec99a2640 1 -- 172.21.15.16:0/3627688664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec41326a0 con 0x7feec4093420 2024-04-23T19:10:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7f8eadffb640 1 -- 172.21.15.16:0/765176160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4003a10 con 0x7f8eb812b7a0 2024-04-23T19:10:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7f8eadffb640 1 -- 172.21.15.16:0/765176160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4003bb0 con 0x7f8eb812b7a0 2024-04-23T19:10:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7f8eadffb640 1 -- 172.21.15.16:0/765176160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4046480 con 0x7f8eb812b7a0 2024-04-23T19:10:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb80728e0 con 0x7f8eb812b7a0 2024-04-23T19:10:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7feec99a2640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4093420 0x7feec41298d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feec000c4a0 tx=0x7feec000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb808f950 con 0x7f8eb812b7a0 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7fd180a91640 1 -- 172.21.15.16:0/1572548299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12d440 msgr2=0x7fd17c131e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7fd180a91640 1 --2- 172.21.15.16:0/1572548299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12d440 0x7fd17c131e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd164009720 tx=0x7fd16402b840 comp rx=0 tx=0).stop 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7fd180a91640 1 -- 172.21.15.16:0/1572548299 shutdown_connections 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.773+0000 7fd180a91640 1 --2- 172.21.15.16:0/1572548299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12d440 0x7fd17c131e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7fd180a91640 1 -- 172.21.15.16:0/1572548299 >> 172.21.15.16:0/1572548299 conn(0x7fd17c071720 msgr2=0x7fd17c071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7fd180a91640 1 -- 172.21.15.16:0/1572548299 shutdown_connections 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7feeba7fc640 1 -- 172.21.15.16:0/3627688664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feec0017070 con 0x7feec4093420 2024-04-23T19:10:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7feeba7fc640 1 -- 172.21.15.16:0/3627688664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0005c10 con 0x7feec4093420 2024-04-23T19:10:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7feeba7fc640 1 -- 172.21.15.16:0/3627688664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feec0007070 con 0x7feec4093420 2024-04-23T19:10:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7fd180a91640 1 -- 172.21.15.16:0/1572548299 wait complete. 2024-04-23T19:10:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7f8eadffb640 1 -- 172.21.15.16:0/765176160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8eb4038020 con 0x7f8eb812b7a0 2024-04-23T19:10:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec412f560 con 0x7feec4093420 2024-04-23T19:10:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec412f120 con 0x7feec4093420 2024-04-23T19:10:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7f8eadffb640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8e9c038610 0x7f8e9c03aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7f8eadffb640 1 -- 172.21.15.16:0/765176160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7f8eb406de90 con 0x7f8eb812b7a0 2024-04-23T19:10:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.774+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb8063660 con 0x7f8eb812b7a0 2024-04-23T19:10:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.775+0000 7f8ebcadf640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8e9c038610 0x7f8e9c03aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.775+0000 7feeba7fc640 1 -- 172.21.15.16:0/3627688664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7feec001a070 con 0x7feec4093420 2024-04-23T19:10:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.775+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec4063660 con 0x7feec4093420 2024-04-23T19:10:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.775+0000 7fd180a91640 1 Processor -- start 2024-04-23T19:10:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.775+0000 7fd180a91640 1 -- start start 2024-04-23T19:10:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7f8ebcadf640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8e9c038610 0x7f8e9c03aad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001560 tx=0x7f8ea8007a50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7feeba7fc640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7feea8038610 0x7feea803aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7feeba7fc640 1 -- 172.21.15.16:0/3627688664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7feec004a8e0 con 0x7feec4093420 2024-04-23T19:10:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7feec91a1640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7feea8038610 0x7feea803aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7fd180a91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12c950 0x7fd17c11d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7fd180a91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c09ef60 con 0x7fd17c12c950 2024-04-23T19:10:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7fd17a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12c950 0x7fd17c11d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7fd17a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12c950 0x7fd17c11d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56910/0 (socket says 172.21.15.16:56910) 2024-04-23T19:10:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7fd17a59c640 1 -- 172.21.15.16:0/2861948340 learned_addr learned my addr 172.21.15.16:0/2861948340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7feec91a1640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7feea8038610 0x7feea803aad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feeb4036000 tx=0x7feeb4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.776+0000 7fd17a59c640 1 -- 172.21.15.16:0/2861948340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c09d380 con 0x7fd17c12c950 2024-04-23T19:10:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.777+0000 7fd17a59c640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12c950 0x7fd17c11d900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00c420 tx=0x7fd16c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.780+0000 7f8eadffb640 1 -- 172.21.15.16:0/765176160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8eb409cb80 con 0x7f8eb812b7a0 2024-04-23T19:10:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.788+0000 7feeba7fc640 1 -- 172.21.15.16:0/3627688664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feec001a760 con 0x7feec4093420 2024-04-23T19:10:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.788+0000 7fd172ffd640 1 -- 172.21.15.16:0/2861948340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c018070 con 0x7fd17c12c950 2024-04-23T19:10:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.788+0000 7fd172ffd640 1 -- 172.21.15.16:0/2861948340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd16c0074d0 con 0x7fd17c12c950 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.788+0000 7fd172ffd640 1 -- 172.21.15.16:0/2861948340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd16c004950 con 0x7fd17c12c950 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.788+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c11f820 con 0x7fd17c12c950 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.788+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c11de40 con 0x7fd17c12c950 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.789+0000 7fd172ffd640 1 -- 172.21.15.16:0/2861948340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd16c01b070 con 0x7fd17c12c950 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.790+0000 7fd172ffd640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd15c038610 0x7fd15c03aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.790+0000 7fd172ffd640 1 -- 172.21.15.16:0/2861948340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7fd16c04aa40 con 0x7fd17c12c950 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.790+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17c063660 con 0x7fd17c12c950 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.790+0000 7fd173fff640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd15c038610 0x7fd15c03aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.790+0000 7fd173fff640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd15c038610 0x7fd15c03aad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd164036000 tx=0x7fd164008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:00.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.799+0000 7fd172ffd640 1 -- 172.21.15.16:0/2861948340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd17c09d380 con 0x7fd17c12c950 2024-04-23T19:10:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.938+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6474067f20 con 0x7f647412b7a0 2024-04-23T19:10:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.939+0000 7f6470ff9640 1 -- 172.21.15.16:0/3597859121 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6464044df0 con 0x7f647412b7a0 2024-04-23T19:10:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:38654705667 2024-04-23T19:10:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.940+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f64600385c0 msgr2=0x7f646003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.940+0000 7f647a6ca640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f64600385c0 0x7f646003aa80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f6468007ac0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.940+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 msgr2=0x7f6474070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.940+0000 7f647a6ca640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f6474070410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6464009720 tx=0x7f64640046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.941+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 shutdown_connections 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.941+0000 7f647a6ca640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f64600385c0 0x7f646003aa80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.941+0000 7f647a6ca640 1 --2- 172.21.15.16:0/3597859121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f647412b7a0 0x7f6474070410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.941+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 >> 172.21.15.16:0/3597859121 conn(0x7f64740a15b0 msgr2=0x7f647409bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.941+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 shutdown_connections 2024-04-23T19:10:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.941+0000 7f647a6ca640 1 -- 172.21.15.16:0/3597859121 wait complete. 2024-04-23T19:10:00.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2024-04-23T19:10:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.976+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7feec4067f20 con 0x7feec4093420 2024-04-23T19:10:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.976+0000 7feeba7fc640 1 -- 172.21.15.16:0/3627688664 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feec001b3b0 con 0x7feec4093420 2024-04-23T19:10:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:42949672964 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7feea8038610 msgr2=0x7feea803aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7feea8038610 0x7feea803aad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feeb4036000 tx=0x7feeb4008030 comp rx=0 tx=0).stop 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4093420 msgr2=0x7feec41298d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4093420 0x7feec41298d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feec000c4a0 tx=0x7feec000c960 comp rx=0 tx=0).stop 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 shutdown_connections 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7feea8038610 0x7feea803aad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 --2- 172.21.15.16:0/3627688664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4093420 0x7feec41298d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 >> 172.21.15.16:0/3627688664 conn(0x7feec409a200 msgr2=0x7feec4127090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 shutdown_connections 2024-04-23T19:10:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.979+0000 7feecbc06640 1 -- 172.21.15.16:0/3627688664 wait complete. 2024-04-23T19:10:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.981+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8eb8071570 con 0x7f8eb812b7a0 2024-04-23T19:10:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.981+0000 7f8eadffb640 1 -- 172.21.15.16:0/765176160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8eb40453e0 con 0x7f8eb812b7a0 2024-04-23T19:10:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:38654705668 2024-04-23T19:10:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8e9c038610 msgr2=0x7f8e9c03aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8e9c038610 0x7f8e9c03aad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001560 tx=0x7f8ea8007a50 comp rx=0 tx=0).stop 2024-04-23T19:10:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 msgr2=0x7f8eb8070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb8070480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8eb402b950 tx=0x7f8eb40046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 shutdown_connections 2024-04-23T19:10:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8e9c038610 0x7f8e9c03aad0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 --2- 172.21.15.16:0/765176160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb812b7a0 0x7f8eb8070480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 >> 172.21.15.16:0/765176160 conn(0x7f8eb80a15b0 msgr2=0x7f8eb809bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 shutdown_connections 2024-04-23T19:10:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:00.983+0000 7f8ebf544640 1 -- 172.21.15.16:0/765176160 wait complete. 2024-04-23T19:10:00.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-04-23T19:10:00.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672964 for osd.1 2024-04-23T19:10:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.012+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd17c067f20 con 0x7fd17c12c950 2024-04-23T19:10:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.012+0000 7fd172ffd640 1 -- 172.21.15.16:0/2861948340 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd16c0119b0 con 0x7fd17c12c950 2024-04-23T19:10:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:42949672965 2024-04-23T19:10:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd15c038610 msgr2=0x7fd15c03aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd15c038610 0x7fd15c03aad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd164036000 tx=0x7fd164008030 comp rx=0 tx=0).stop 2024-04-23T19:10:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12c950 msgr2=0x7fd17c11d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12c950 0x7fd17c11d900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00c420 tx=0x7fd16c00c8e0 comp rx=0 tx=0).stop 2024-04-23T19:10:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 shutdown_connections 2024-04-23T19:10:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd15c038610 0x7fd15c03aad0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 --2- 172.21.15.16:0/2861948340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c12c950 0x7fd17c11d900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 >> 172.21.15.16:0/2861948340 conn(0x7fd17c071720 msgr2=0x7fd17c09bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 shutdown_connections 2024-04-23T19:10:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:01.014+0000 7fd180a91640 1 -- 172.21.15.16:0/2861948340 wait complete. 2024-04-23T19:10:01.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672965 for osd.3 2024-04-23T19:10:01.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T19:10:01.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T19:10:01.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T19:10:02.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T19:10:02.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.073+0000 7fb49e7bd640 1 Processor -- start 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.073+0000 7fb49e7bd640 1 -- start start 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.073+0000 7fb49e7bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb49812b7a0 0x7fb49812bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.074+0000 7fb49e7bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb49809aa00 con 0x7fb49812c0c0 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.074+0000 7fb49e7bd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb4980a20d0 con 0x7fb49812b7a0 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.074+0000 7fb497fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb49812b7a0 0x7fb49812bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.074+0000 7fb4977fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb49812c0c0 0x7fb49812eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60560/0 (socket says 172.21.15.16:60560) 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.074+0000 7fb4977fe640 1 -- 172.21.15.16:0/3911178970 learned_addr learned my addr 172.21.15.16:0/3911178970 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.074+0000 7fb497fff640 1 -- 172.21.15.16:0/3911178970 >> v1:172.21.15.16:6789/0 conn(0x7fb49812c0c0 legacy=0x7fb49812eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.074+0000 7fb497fff640 1 -- 172.21.15.16:0/3911178970 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49809af50 con 0x7fb49812b7a0 2024-04-23T19:10:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.075+0000 7fb497fff640 1 --2- 172.21.15.16:0/3911178970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb49812bb80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb488009720 tx=0x7fb48802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d31c7f849f341b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.075+0000 7fb496ffd640 1 -- 172.21.15.16:0/3911178970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4880316d0 con 0x7fb49812b7a0 2024-04-23T19:10:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.075+0000 7fb496ffd640 1 -- 172.21.15.16:0/3911178970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb488037070 con 0x7fb49812b7a0 2024-04-23T19:10:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.075+0000 7fb496ffd640 1 -- 172.21.15.16:0/3911178970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4880316d0 con 0x7fb49812b7a0 2024-04-23T19:10:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.076+0000 7fb49e7bd640 1 -- 172.21.15.16:0/3911178970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 msgr2=0x7fb49812bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.076+0000 7fb49e7bd640 1 --2- 172.21.15.16:0/3911178970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb49812bb80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb488009720 tx=0x7fb48802b840 comp rx=0 tx=0).stop 2024-04-23T19:10:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.076+0000 7fb49e7bd640 1 -- 172.21.15.16:0/3911178970 shutdown_connections 2024-04-23T19:10:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.076+0000 7fb49e7bd640 1 --2- 172.21.15.16:0/3911178970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb49812bb80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.076+0000 7fb49e7bd640 1 -- 172.21.15.16:0/3911178970 >> 172.21.15.16:0/3911178970 conn(0x7fb4980a15b0 msgr2=0x7fb4980a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.076+0000 7fb49e7bd640 1 -- 172.21.15.16:0/3911178970 shutdown_connections 2024-04-23T19:10:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.076+0000 7fb49e7bd640 1 -- 172.21.15.16:0/3911178970 wait complete. 2024-04-23T19:10:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.078+0000 7fb49e7bd640 1 Processor -- start 2024-04-23T19:10:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.078+0000 7fb49e7bd640 1 -- start start 2024-04-23T19:10:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.079+0000 7fb49e7bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb498070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.079+0000 7fb49e7bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb49812c500 con 0x7fb49812b7a0 2024-04-23T19:10:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.079+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb498070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.079+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb498070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56936/0 (socket says 172.21.15.16:56936) 2024-04-23T19:10:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.079+0000 7fb497fff640 1 -- 172.21.15.16:0/1122000580 learned_addr learned my addr 172.21.15.16:0/1122000580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.079+0000 7fb497fff640 1 -- 172.21.15.16:0/1122000580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4980730f0 con 0x7fb49812b7a0 2024-04-23T19:10:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.080+0000 7fb497fff640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb498070480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb488034040 tx=0x7fb4880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.080+0000 7fb494ff9640 1 -- 172.21.15.16:0/1122000580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488003740 con 0x7fb49812b7a0 2024-04-23T19:10:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.080+0000 7fb494ff9640 1 -- 172.21.15.16:0/1122000580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4880038e0 con 0x7fb49812b7a0 2024-04-23T19:10:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.080+0000 7fb494ff9640 1 -- 172.21.15.16:0/1122000580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488003c00 con 0x7fb49812b7a0 2024-04-23T19:10:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.080+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4980728e0 con 0x7fb49812b7a0 2024-04-23T19:10:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.080+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49808f950 con 0x7fb49812b7a0 2024-04-23T19:10:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.081+0000 7fb494ff9640 1 -- 172.21.15.16:0/1122000580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb488047480 con 0x7fb49812b7a0 2024-04-23T19:10:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.081+0000 7fb494ff9640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4840385c0 0x7fb48403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.081+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb498063660 con 0x7fb49812b7a0 2024-04-23T19:10:02.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.081+0000 7fb494ff9640 1 -- 172.21.15.16:0/1122000580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7fb48802f050 con 0x7fb49812b7a0 2024-04-23T19:10:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.082+0000 7fb4977fe640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4840385c0 0x7fb48403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.082+0000 7fb4977fe640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4840385c0 0x7fb48403aa80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb48c000f60 tx=0x7fb48c007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.086+0000 7fb494ff9640 1 -- 172.21.15.16:0/1122000580 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4880505b0 con 0x7fb49812b7a0 2024-04-23T19:10:02.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.112+0000 7f6b75406640 1 Processor -- start 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.112+0000 7f6b75406640 1 -- start start 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.112+0000 7f6b75406640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b7012b7a0 0x7f6b7012bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.112+0000 7f6b75406640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7009aa00 con 0x7f6b7012c0c0 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.112+0000 7f6b75406640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6b700a20d0 con 0x7f6b7012b7a0 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.112+0000 7f6b6effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b7012b7a0 0x7f6b7012bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.113+0000 7f6b6e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6b7012c0c0 0x7f6b7012eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60568/0 (socket says 172.21.15.16:60568) 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.113+0000 7f6b6effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b7012b7a0 0x7f6b7012bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56948/0 (socket says 172.21.15.16:56948) 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.113+0000 7f6b6e7fc640 1 -- 172.21.15.16:0/3195515312 learned_addr learned my addr 172.21.15.16:0/3195515312 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.113+0000 7f6b6effd640 1 -- 172.21.15.16:0/3195515312 >> v1:172.21.15.16:6789/0 conn(0x7f6b7012c0c0 legacy=0x7f6b7012eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.113+0000 7f6b6effd640 1 -- 172.21.15.16:0/3195515312 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7009af50 con 0x7f6b7012b7a0 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.114+0000 7f6b6effd640 1 --2- 172.21.15.16:0/3195515312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b7012bb80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009720 tx=0x7f6b6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e6179e1a9c0d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.114+0000 7f6b6dffb640 1 -- 172.21.15.16:0/3195515312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b600316d0 con 0x7f6b7012b7a0 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.114+0000 7f6b6dffb640 1 -- 172.21.15.16:0/3195515312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b60037070 con 0x7f6b7012b7a0 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.114+0000 7f6b6dffb640 1 -- 172.21.15.16:0/3195515312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b600319c0 con 0x7f6b7012b7a0 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.114+0000 7f6b75406640 1 -- 172.21.15.16:0/3195515312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 msgr2=0x7f6b7012bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.114+0000 7f6b75406640 1 --2- 172.21.15.16:0/3195515312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b7012bb80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009720 tx=0x7f6b6002b840 comp rx=0 tx=0).stop 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.115+0000 7f6b75406640 1 -- 172.21.15.16:0/3195515312 shutdown_connections 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.115+0000 7f6b75406640 1 --2- 172.21.15.16:0/3195515312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b7012bb80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.115+0000 7f6b75406640 1 -- 172.21.15.16:0/3195515312 >> 172.21.15.16:0/3195515312 conn(0x7f6b700a15b0 msgr2=0x7f6b700a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.115+0000 7f6b75406640 1 -- 172.21.15.16:0/3195515312 shutdown_connections 2024-04-23T19:10:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.115+0000 7f6b75406640 1 -- 172.21.15.16:0/3195515312 wait complete. 2024-04-23T19:10:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.116+0000 7f6b75406640 1 Processor -- start 2024-04-23T19:10:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.116+0000 7f6b75406640 1 -- start start 2024-04-23T19:10:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.116+0000 7f6b75406640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b70070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.116+0000 7f6b75406640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7012c500 con 0x7f6b7012b7a0 2024-04-23T19:10:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.116+0000 7f6b6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b70070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.116+0000 7f6b6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b70070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56952/0 (socket says 172.21.15.16:56952) 2024-04-23T19:10:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.116+0000 7f6b6effd640 1 -- 172.21.15.16:0/744895314 learned_addr learned my addr 172.21.15.16:0/744895314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.117+0000 7f6b6effd640 1 -- 172.21.15.16:0/744895314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b70071e40 con 0x7f6b7012b7a0 2024-04-23T19:10:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.117+0000 7f6b6effd640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b70070410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009720 tx=0x7f6b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.117+0000 7f6b577fe640 1 -- 172.21.15.16:0/744895314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b60032770 con 0x7f6b7012b7a0 2024-04-23T19:10:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.117+0000 7f6b577fe640 1 -- 172.21.15.16:0/744895314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b60032910 con 0x7f6b7012b7a0 2024-04-23T19:10:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.117+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7008fad0 con 0x7f6b7012b7a0 2024-04-23T19:10:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.117+0000 7f6b577fe640 1 -- 172.21.15.16:0/744895314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b60032bf0 con 0x7f6b7012b7a0 2024-04-23T19:10:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.117+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b70070950 con 0x7f6b7012b7a0 2024-04-23T19:10:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.119+0000 7f6b577fe640 1 -- 172.21.15.16:0/744895314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6b6004c060 con 0x7f6b7012b7a0 2024-04-23T19:10:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.119+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b70063660 con 0x7f6b7012b7a0 2024-04-23T19:10:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.119+0000 7f6b577fe640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b5c038570 0x7f6b5c03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.119+0000 7f6b577fe640 1 -- 172.21.15.16:0/744895314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7f6b6002f050 con 0x7f6b7012b7a0 2024-04-23T19:10:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.119+0000 7f6b6e7fc640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b5c038570 0x7f6b5c03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.120+0000 7f6b6e7fc640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b5c038570 0x7f6b5c03aa30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007fd0 tx=0x7f6b64007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4ad5e26640 1 Processor -- start 2024-04-23T19:10:02.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4ad5e26640 1 -- start start 2024-04-23T19:10:02.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4ad5e26640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ad012c290 0x7f4ad0133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4ad5e26640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad009aa00 con 0x7f4ad012b7a0 2024-04-23T19:10:02.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4ad5e26640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad00a20d0 con 0x7f4ad012c290 2024-04-23T19:10:02.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4acf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4ad012b7a0 0x7f4ad012bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60570/0 (socket says 172.21.15.16:60570) 2024-04-23T19:10:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4acf7fe640 1 -- 172.21.15.16:0/3398707903 learned_addr learned my addr 172.21.15.16:0/3398707903 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.125+0000 7f4aceffd640 1 --2- 172.21.15.16:0/3398707903 >> v2:172.21.15.16:3300/0 conn(0x7f4ad012c290 0x7f4ad0133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.126+0000 7f4aceffd640 1 -- 172.21.15.16:0/3398707903 >> v1:172.21.15.16:6789/0 conn(0x7f4ad012b7a0 legacy=0x7f4ad012bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.126+0000 7f4aceffd640 1 -- 172.21.15.16:0/3398707903 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad009af50 con 0x7f4ad012c290 2024-04-23T19:10:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.127+0000 7f4aceffd640 1 --2- 172.21.15.16:0/3398707903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012c290 0x7f4ad0133fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009720 tx=0x7f4ac402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d325caa15fc2384 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.128+0000 7f4ace7fc640 1 -- 172.21.15.16:0/3398707903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac40316d0 con 0x7f4ad012c290 2024-04-23T19:10:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.128+0000 7f4ace7fc640 1 -- 172.21.15.16:0/3398707903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac4037070 con 0x7f4ad012c290 2024-04-23T19:10:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.128+0000 7f4ace7fc640 1 -- 172.21.15.16:0/3398707903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac40319c0 con 0x7f4ad012c290 2024-04-23T19:10:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.128+0000 7f6b577fe640 1 -- 172.21.15.16:0/744895314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b60040070 con 0x7f6b7012b7a0 2024-04-23T19:10:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.129+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3398707903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012c290 msgr2=0x7f4ad0133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.129+0000 7f4ad5e26640 1 --2- 172.21.15.16:0/3398707903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012c290 0x7f4ad0133fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009720 tx=0x7f4ac402b840 comp rx=0 tx=0).stop 2024-04-23T19:10:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.129+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3398707903 shutdown_connections 2024-04-23T19:10:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.129+0000 7f4ad5e26640 1 --2- 172.21.15.16:0/3398707903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012c290 0x7f4ad0133fc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.129+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3398707903 >> 172.21.15.16:0/3398707903 conn(0x7f4ad00a15b0 msgr2=0x7f4ad00a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.129+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3398707903 shutdown_connections 2024-04-23T19:10:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.129+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3398707903 wait complete. 2024-04-23T19:10:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.130+0000 7f4ad5e26640 1 Processor -- start 2024-04-23T19:10:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.130+0000 7f4ad5e26640 1 -- start start 2024-04-23T19:10:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.131+0000 7f4ad5e26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012b7a0 0x7f4ad0131d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.131+0000 7f4ad5e26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0134500 con 0x7f4ad012b7a0 2024-04-23T19:10:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.131+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012b7a0 0x7f4ad0131d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.131+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012b7a0 0x7f4ad0131d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56972/0 (socket says 172.21.15.16:56972) 2024-04-23T19:10:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.131+0000 7f4acf7fe640 1 -- 172.21.15.16:0/3777511583 learned_addr learned my addr 172.21.15.16:0/3777511583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.131+0000 7f4acf7fe640 1 -- 172.21.15.16:0/3777511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01337e0 con 0x7f4ad012b7a0 2024-04-23T19:10:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.132+0000 7f4acf7fe640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012b7a0 0x7f4ad0131d70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c380 tx=0x7f4ac000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.132+0000 7f4ab7fff640 1 -- 172.21.15.16:0/3777511583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac0005c10 con 0x7f4ad012b7a0 2024-04-23T19:10:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.132+0000 7f4ab7fff640 1 -- 172.21.15.16:0/3777511583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac0005db0 con 0x7f4ad012b7a0 2024-04-23T19:10:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.132+0000 7f4ab7fff640 1 -- 172.21.15.16:0/3777511583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac0004930 con 0x7f4ad012b7a0 2024-04-23T19:10:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.132+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad006d650 con 0x7f4ad012b7a0 2024-04-23T19:10:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.132+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad006ea50 con 0x7f4ad012b7a0 2024-04-23T19:10:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.133+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad0063660 con 0x7f4ad012b7a0 2024-04-23T19:10:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.134+0000 7f4ab7fff640 1 -- 172.21.15.16:0/3777511583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4ac001a020 con 0x7f4ad012b7a0 2024-04-23T19:10:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.134+0000 7f4ab7fff640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4abc038570 0x7f4abc03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.134+0000 7f4ab7fff640 1 -- 172.21.15.16:0/3777511583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7f4ac004ade0 con 0x7f4ad012b7a0 2024-04-23T19:10:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.134+0000 7f4aceffd640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4abc038570 0x7f4abc03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.135+0000 7f4aceffd640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4abc038570 0x7f4abc03aa30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4036000 tx=0x7f4ac4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.143+0000 7f4ab7fff640 1 -- 172.21.15.16:0/3777511583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ac00127f0 con 0x7f4ad012b7a0 2024-04-23T19:10:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.154+0000 7ff851957640 1 Processor -- start 2024-04-23T19:10:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.154+0000 7ff851957640 1 -- start start 2024-04-23T19:10:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.155+0000 7ff851957640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff84c12b7a0 0x7ff84c12bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.155+0000 7ff851957640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84c09aa00 con 0x7ff84c12c0c0 2024-04-23T19:10:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.155+0000 7ff851957640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff84c0a20d0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.155+0000 7ff84affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff84c12b7a0 0x7ff84c12bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.155+0000 7ff84affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff84c12b7a0 0x7ff84c12bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56988/0 (socket says 172.21.15.16:56988) 2024-04-23T19:10:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.155+0000 7ff84affd640 1 -- 172.21.15.16:0/4040848928 learned_addr learned my addr 172.21.15.16:0/4040848928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.155+0000 7ff84affd640 1 -- 172.21.15.16:0/4040848928 >> v1:172.21.15.16:6789/0 conn(0x7ff84c12c0c0 legacy=0x7ff84c12eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.156+0000 7ff84affd640 1 -- 172.21.15.16:0/4040848928 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c09af50 con 0x7ff84c12b7a0 2024-04-23T19:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.156+0000 7ff84affd640 1 --2- 172.21.15.16:0/4040848928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c12bb80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0097f0 tx=0x7ff83c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a7ce5f571de38a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.156+0000 7ff849ffb640 1 -- 172.21.15.16:0/4040848928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c035020 con 0x7ff84c12b7a0 2024-04-23T19:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.156+0000 7ff849ffb640 1 -- 172.21.15.16:0/4040848928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c0317a0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.156+0000 7ff849ffb640 1 -- 172.21.15.16:0/4040848928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c031aa0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.157+0000 7ff851957640 1 -- 172.21.15.16:0/4040848928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 msgr2=0x7ff84c12bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.157+0000 7ff851957640 1 --2- 172.21.15.16:0/4040848928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c12bb80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0097f0 tx=0x7ff83c02b910 comp rx=0 tx=0).stop 2024-04-23T19:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.157+0000 7ff851957640 1 -- 172.21.15.16:0/4040848928 shutdown_connections 2024-04-23T19:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.157+0000 7ff851957640 1 --2- 172.21.15.16:0/4040848928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c12bb80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.157+0000 7ff851957640 1 -- 172.21.15.16:0/4040848928 >> 172.21.15.16:0/4040848928 conn(0x7ff84c0a15b0 msgr2=0x7ff84c0a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.157+0000 7ff851957640 1 -- 172.21.15.16:0/4040848928 shutdown_connections 2024-04-23T19:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.157+0000 7ff851957640 1 -- 172.21.15.16:0/4040848928 wait complete. 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.158+0000 7ff851957640 1 Processor -- start 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.158+0000 7ff851957640 1 -- start start 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.159+0000 7ff851957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.159+0000 7ff851957640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c12c500 con 0x7ff84c12b7a0 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.159+0000 7ff84affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.159+0000 7ff84affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57000/0 (socket says 172.21.15.16:57000) 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.159+0000 7ff84affd640 1 -- 172.21.15.16:0/4114747584 learned_addr learned my addr 172.21.15.16:0/4114747584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.159+0000 7ff84affd640 1 -- 172.21.15.16:0/4114747584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c071e40 con 0x7ff84c12b7a0 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.159+0000 7ff84affd640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c070410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0097f0 tx=0x7ff83c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.160+0000 7ff833fff640 1 -- 172.21.15.16:0/4114747584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c03ad00 con 0x7ff84c12b7a0 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.160+0000 7ff833fff640 1 -- 172.21.15.16:0/4114747584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c0443f0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.160+0000 7ff833fff640 1 -- 172.21.15.16:0/4114747584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c0446d0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.160+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c08fad0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.160+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c070950 con 0x7ff84c12b7a0 2024-04-23T19:10:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.161+0000 7ff833fff640 1 -- 172.21.15.16:0/4114747584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff83c04b060 con 0x7ff84c12b7a0 2024-04-23T19:10:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.161+0000 7ff833fff640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff8380385c0 0x7ff83803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.161+0000 7ff833fff640 1 -- 172.21.15.16:0/4114747584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7ff83c06d7f0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.161+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c063660 con 0x7ff84c12b7a0 2024-04-23T19:10:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.161+0000 7ff84a7fc640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff8380385c0 0x7ff83803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.166+0000 7ff84a7fc640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff8380385c0 0x7ff83803aa80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff840000f60 tx=0x7ff840007a30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.170+0000 7ff833fff640 1 -- 172.21.15.16:0/4114747584 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff83c03f070 con 0x7ff84c12b7a0 2024-04-23T19:10:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.315+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb498067f20 con 0x7fb49812b7a0 2024-04-23T19:10:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.315+0000 7fb494ff9640 1 -- 172.21.15.16:0/1122000580 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb488055030 con 0x7fb49812b7a0 2024-04-23T19:10:02.317 INFO:teuthology.orchestra.run.smithi016.stdout:38654705668 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4840385c0 msgr2=0x7fb48403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4840385c0 0x7fb48403aa80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb48c000f60 tx=0x7fb48c007a80 comp rx=0 tx=0).stop 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 msgr2=0x7fb498070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb498070480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb488034040 tx=0x7fb4880046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 shutdown_connections 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4840385c0 0x7fb48403aa80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 --2- 172.21.15.16:0/1122000580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb49812b7a0 0x7fb498070480 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 >> 172.21.15.16:0/1122000580 conn(0x7fb4980a15b0 msgr2=0x7fb49809bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 shutdown_connections 2024-04-23T19:10:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.319+0000 7fb49e7bd640 1 -- 172.21.15.16:0/1122000580 wait complete. 2024-04-23T19:10:02.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2024-04-23T19:10:02.335 DEBUG:teuthology.parallel:result is None 2024-04-23T19:10:02.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.341+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4ad009b4d0 con 0x7f4ad012b7a0 2024-04-23T19:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.341+0000 7f4ab7fff640 1 -- 172.21.15.16:0/3777511583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ac0012a50 con 0x7f4ad012b7a0 2024-04-23T19:10:02.342 INFO:teuthology.orchestra.run.smithi016.stdout:42949672965 2024-04-23T19:10:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4abc038570 msgr2=0x7f4abc03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4abc038570 0x7f4abc03aa30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4036000 tx=0x7f4ac4008030 comp rx=0 tx=0).stop 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012b7a0 msgr2=0x7f4ad0131d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012b7a0 0x7f4ad0131d70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c380 tx=0x7f4ac000c840 comp rx=0 tx=0).stop 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 shutdown_connections 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4abc038570 0x7f4abc03aa30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 --2- 172.21.15.16:0/3777511583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad012b7a0 0x7f4ad0131d70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 >> 172.21.15.16:0/3777511583 conn(0x7f4ad00a15b0 msgr2=0x7f4ad012e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 shutdown_connections 2024-04-23T19:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.343+0000 7f4ad5e26640 1 -- 172.21.15.16:0/3777511583 wait complete. 2024-04-23T19:10:02.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.1 2024-04-23T19:10:02.356 DEBUG:teuthology.parallel:result is None 2024-04-23T19:10:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.365+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6b70067f20 con 0x7f6b7012b7a0 2024-04-23T19:10:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.365+0000 7f6b577fe640 1 -- 172.21.15.16:0/744895314 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b60044df0 con 0x7f6b7012b7a0 2024-04-23T19:10:02.366 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2024-04-23T19:10:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b5c038570 msgr2=0x7f6b5c03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b5c038570 0x7f6b5c03aa30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007fd0 tx=0x7f6b64007ac0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 msgr2=0x7f6b70070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b70070410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009720 tx=0x7f6b600046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 shutdown_connections 2024-04-23T19:10:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b5c038570 0x7f6b5c03aa30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 --2- 172.21.15.16:0/744895314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7012b7a0 0x7f6b70070410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 >> 172.21.15.16:0/744895314 conn(0x7f6b700a15b0 msgr2=0x7f6b7009bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 shutdown_connections 2024-04-23T19:10:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.367+0000 7f6b75406640 1 -- 172.21.15.16:0/744895314 wait complete. 2024-04-23T19:10:02.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-04-23T19:10:02.378 DEBUG:teuthology.parallel:result is None 2024-04-23T19:10:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.414+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff84c067f20 con 0x7ff84c12b7a0 2024-04-23T19:10:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.415+0000 7ff833fff640 1 -- 172.21.15.16:0/4114747584 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff83c0433a0 con 0x7ff84c12b7a0 2024-04-23T19:10:02.416 INFO:teuthology.orchestra.run.smithi016.stdout:42949672966 2024-04-23T19:10:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff8380385c0 msgr2=0x7ff83803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff8380385c0 0x7ff83803aa80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff840000f60 tx=0x7ff840007a30 comp rx=0 tx=0).stop 2024-04-23T19:10:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 msgr2=0x7ff84c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c070410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0097f0 tx=0x7ff83c0046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 shutdown_connections 2024-04-23T19:10:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff8380385c0 0x7ff83803aa80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 --2- 172.21.15.16:0/4114747584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84c12b7a0 0x7ff84c070410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 >> 172.21.15.16:0/4114747584 conn(0x7ff84c0a15b0 msgr2=0x7ff84c09bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 shutdown_connections 2024-04-23T19:10:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.417+0000 7ff851957640 1 -- 172.21.15.16:0/4114747584 wait complete. 2024-04-23T19:10:02.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.3 2024-04-23T19:10:02.429 DEBUG:teuthology.parallel:result is None 2024-04-23T19:10:02.429 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T19:10:02.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:10:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.568+0000 7f9ff6e4e640 1 Processor -- start 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.568+0000 7f9ff6e4e640 1 -- start start 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9ff6e4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ff012ee30 0x7f9ff012f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9ff6e4e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff012ec50 con 0x7f9ff012f750 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9ff6e4e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff012fba0 con 0x7f9ff012ee30 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9ff4bea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ff012ee30 0x7f9ff012f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9feffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9ff012f750 0x7f9ff006cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60586/0 (socket says 172.21.15.16:60586) 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9feffff640 1 -- 172.21.15.16:0/3672012939 learned_addr learned my addr 172.21.15.16:0/3672012939 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9ff4bea640 1 -- 172.21.15.16:0/3672012939 >> v1:172.21.15.16:6789/0 conn(0x7f9ff012f750 legacy=0x7f9ff006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9ff4bea640 1 -- 172.21.15.16:0/3672012939 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff0095f40 con 0x7f9ff012ee30 2024-04-23T19:10:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.569+0000 7f9fef7fe640 1 -- 172.21.15.16:0/3672012939 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 825484326 0 0) 0x7f9fe4001110 con 0x7f9ff012f750 2024-04-23T19:10:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.570+0000 7f9ff4bea640 1 --2- 172.21.15.16:0/3672012939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff012f210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009720 tx=0x7f9fe002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a52ec828aff81e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.570+0000 7f9fef7fe640 1 -- 172.21.15.16:0/3672012939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe00316d0 con 0x7f9ff012ee30 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.570+0000 7f9fef7fe640 1 -- 172.21.15.16:0/3672012939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe0037070 con 0x7f9ff012ee30 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.570+0000 7f9fef7fe640 1 -- 172.21.15.16:0/3672012939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe00319c0 con 0x7f9ff012ee30 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3672012939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 msgr2=0x7f9ff012f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 --2- 172.21.15.16:0/3672012939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff012f210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009720 tx=0x7f9fe002b840 comp rx=0 tx=0).stop 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3672012939 shutdown_connections 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 --2- 172.21.15.16:0/3672012939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff012f210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3672012939 >> 172.21.15.16:0/3672012939 conn(0x7f9ff009e2b0 msgr2=0x7f9ff009e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3672012939 shutdown_connections 2024-04-23T19:10:02.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3672012939 wait complete. 2024-04-23T19:10:02.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.571+0000 7f9ff6e4e640 1 Processor -- start 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.572+0000 7f9ff6e4e640 1 -- start start 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.572+0000 7f9ff6e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff0129bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.572+0000 7f9ff6e4e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00734d0 con 0x7f9ff012ee30 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.572+0000 7f9ff4bea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff0129bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.572+0000 7f9ff4bea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff0129bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57020/0 (socket says 172.21.15.16:57020) 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.572+0000 7f9ff4bea640 1 -- 172.21.15.16:0/3314310425 learned_addr learned my addr 172.21.15.16:0/3314310425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.572+0000 7f9ff4bea640 1 -- 172.21.15.16:0/3314310425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff012c010 con 0x7f9ff012ee30 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.573+0000 7f9ff4bea640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff0129bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009720 tx=0x7f9fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.573+0000 7f9fed7fa640 1 -- 172.21.15.16:0/3314310425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe0032770 con 0x7f9ff012ee30 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.573+0000 7f9fed7fa640 1 -- 172.21.15.16:0/3314310425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe0032910 con 0x7f9ff012ee30 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.573+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff0126de0 con 0x7f9ff012ee30 2024-04-23T19:10:02.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.573+0000 7f9fed7fa640 1 -- 172.21.15.16:0/3314310425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe0032bf0 con 0x7f9ff012ee30 2024-04-23T19:10:02.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.573+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff012a310 con 0x7f9ff012ee30 2024-04-23T19:10:02.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.574+0000 7f9fed7fa640 1 -- 172.21.15.16:0/3314310425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9fe004c060 con 0x7f9ff012ee30 2024-04-23T19:10:02.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.574+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff012b730 con 0x7f9ff012ee30 2024-04-23T19:10:02.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.574+0000 7f9fed7fa640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9fdc0385c0 0x7f9fdc03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.574+0000 7f9fed7fa640 1 -- 172.21.15.16:0/3314310425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7f9fe002f050 con 0x7f9ff012ee30 2024-04-23T19:10:02.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.574+0000 7f9feffff640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9fdc0385c0 0x7f9fdc03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.575+0000 7f9feffff640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9fdc0385c0 0x7f9fdc03aa80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4000c00 tx=0x7f9fe4001310 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.579+0000 7f9fed7fa640 1 -- 172.21.15.16:0/3314310425 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fe0040070 con 0x7f9ff012ee30 2024-04-23T19:10:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.759+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ff0063660 con 0x7f9fdc0385c0 2024-04-23T19:10:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.761+0000 7f9fed7fa640 1 -- 172.21.15.16:0/3314310425 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31844 (secure 0 0 0) 0x7f9ff0063660 con 0x7f9fdc0385c0 2024-04-23T19:10:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:10:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9fdc0385c0 msgr2=0x7f9fdc03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9fdc0385c0 0x7f9fdc03aa80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4000c00 tx=0x7f9fe4001310 comp rx=0 tx=0).stop 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 msgr2=0x7f9ff0129bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff0129bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009720 tx=0x7f9fe00046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 shutdown_connections 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9fdc0385c0 0x7f9fdc03aa80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 --2- 172.21.15.16:0/3314310425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff012ee30 0x7f9ff0129bf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 >> 172.21.15.16:0/3314310425 conn(0x7f9ff009e2b0 msgr2=0x7f9ff0098890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 shutdown_connections 2024-04-23T19:10:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.763+0000 7f9ff6e4e640 1 -- 172.21.15.16:0/3314310425 wait complete. 2024-04-23T19:10:02.774 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-23T19:10:02.639688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":118064,"kb_used_data":1656,"kb_used_omap":6,"kb_used_meta":116345,"kb_avail":377369296,"statfs":{"total":386547056640,"available":386426159104,"internally_reserved":0,"allocated":1695744,"data_stored":1334608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":119138071},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.202503"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576977+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:10:00.576977+0000","last_peered":"2024-04-23T19:10:00.576977+0000","last_clean":"2024-04-23T19:10:00.576977+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:10:00.576977+0000","last_undegraded":"2024-04-23T19:10:00.576977+0000","last_fullsized":"2024-04-23T19:10:00.576977+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.025384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T19:09:59.451676+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:09:59.451676+0000","last_peered":"2024-04-23T19:09:59.451676+0000","last_clean":"2024-04-23T19:09:59.451676+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:09:59.451676+0000","last_undegraded":"2024-04-23T19:09:59.451676+0000","last_fullsized":"2024-04-23T19:09:59.451676+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:31:58.176909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576956+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:10:00.576956+0000","last_peered":"2024-04-23T19:10:00.576956+0000","last_clean":"2024-04-23T19:10:00.576956+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:10:00.576956+0000","last_undegraded":"2024-04-23T19:10:00.576956+0000","last_fullsized":"2024-04-23T19:10:00.576956+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:36.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T19:09:59.451504+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:09:59.451504+0000","last_peered":"2024-04-23T19:09:59.451504+0000","last_clean":"2024-04-23T19:09:59.451504+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:09:59.451504+0000","last_undegraded":"2024-04-23T19:09:59.451504+0000","last_fullsized":"2024-04-23T19:09:59.451504+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:42.755016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576956+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:10:00.576956+0000","last_peered":"2024-04-23T19:10:00.576956+0000","last_clean":"2024-04-23T19:10:00.576956+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:10:00.576956+0000","last_undegraded":"2024-04-23T19:10:00.576956+0000","last_fullsized":"2024-04-23T19:10:00.576956+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:51.612339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.444409+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:10:00.444409+0000","last_peered":"2024-04-23T19:10:00.444409+0000","last_clean":"2024-04-23T19:10:00.444409+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:10:00.444409+0000","last_undegraded":"2024-04-23T19:10:00.444409+0000","last_fullsized":"2024-04-23T19:10:00.444409+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:45.326443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576982+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:10:00.576982+0000","last_peered":"2024-04-23T19:10:00.576982+0000","last_clean":"2024-04-23T19:10:00.576982+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:10:00.576982+0000","last_undegraded":"2024-04-23T19:10:00.576982+0000","last_fullsized":"2024-04-23T19:10:00.576982+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:17.746806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576960+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:10:00.576960+0000","last_peered":"2024-04-23T19:10:00.576960+0000","last_clean":"2024-04-23T19:10:00.576960+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:10:00.576960+0000","last_undegraded":"2024-04-23T19:10:00.576960+0000","last_fullsized":"2024-04-23T19:10:00.576960+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:09.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":167,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:01.389405+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:10:01.389405+0000","last_peered":"2024-04-23T19:10:01.389405+0000","last_clean":"2024-04-23T19:10:01.389405+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:10:01.389405+0000","last_undegraded":"2024-04-23T19:10:01.389405+0000","last_fullsized":"2024-04-23T19:10:01.389405+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:35.747545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":40478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":626826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":630846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35464,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336376,"statfs":{"total":96636764160,"available":96600449024,"internally_reserved":0,"allocated":122880,"data_stored":36458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:10:02.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:10:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.874+0000 7fa4be679640 1 Processor -- start 2024-04-23T19:10:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.874+0000 7fa4be679640 1 -- start start 2024-04-23T19:10:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4be679640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4b812c290 0x7fa4b8133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4be679640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b809aa00 con 0x7fa4b812b7a0 2024-04-23T19:10:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4be679640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b80a20d0 con 0x7fa4b812c290 2024-04-23T19:10:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4b77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4b812c290 0x7fa4b8133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4b7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa4b812b7a0 0x7fa4b812bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60600/0 (socket says 172.21.15.16:60600) 2024-04-23T19:10:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4b7fff640 1 -- 172.21.15.16:0/1907056685 learned_addr learned my addr 172.21.15.16:0/1907056685 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4b77fe640 1 -- 172.21.15.16:0/1907056685 >> v1:172.21.15.16:6789/0 conn(0x7fa4b812b7a0 legacy=0x7fa4b812bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.875+0000 7fa4b77fe640 1 -- 172.21.15.16:0/1907056685 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b809af50 con 0x7fa4b812c290 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.876+0000 7fa4b77fe640 1 --2- 172.21.15.16:0/1907056685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812c290 0x7fa4b8133fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009720 tx=0x7fa4ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3247528487d85eac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.876+0000 7fa4b6ffd640 1 -- 172.21.15.16:0/1907056685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac0316d0 con 0x7fa4b812c290 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.876+0000 7fa4b6ffd640 1 -- 172.21.15.16:0/1907056685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ac037070 con 0x7fa4b812c290 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.876+0000 7fa4b6ffd640 1 -- 172.21.15.16:0/1907056685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac0319c0 con 0x7fa4b812c290 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.876+0000 7fa4be679640 1 -- 172.21.15.16:0/1907056685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812c290 msgr2=0x7fa4b8133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.876+0000 7fa4be679640 1 --2- 172.21.15.16:0/1907056685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812c290 0x7fa4b8133fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009720 tx=0x7fa4ac02b840 comp rx=0 tx=0).stop 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.877+0000 7fa4be679640 1 -- 172.21.15.16:0/1907056685 shutdown_connections 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.877+0000 7fa4be679640 1 --2- 172.21.15.16:0/1907056685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812c290 0x7fa4b8133fc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.877+0000 7fa4be679640 1 -- 172.21.15.16:0/1907056685 >> 172.21.15.16:0/1907056685 conn(0x7fa4b80a15b0 msgr2=0x7fa4b80a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.877+0000 7fa4be679640 1 -- 172.21.15.16:0/1907056685 shutdown_connections 2024-04-23T19:10:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.877+0000 7fa4be679640 1 -- 172.21.15.16:0/1907056685 wait complete. 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.877+0000 7fa4be679640 1 Processor -- start 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4be679640 1 -- start start 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4be679640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812b7a0 0x7fa4b81c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4be679640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8134500 con 0x7fa4b812b7a0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812b7a0 0x7fa4b81c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812b7a0 0x7fa4b81c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57026/0 (socket says 172.21.15.16:57026) 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4b7fff640 1 -- 172.21.15.16:0/3392030222 learned_addr learned my addr 172.21.15.16:0/3392030222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4b7fff640 1 -- 172.21.15.16:0/3392030222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b806da40 con 0x7fa4b812b7a0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.878+0000 7fa4b7fff640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812b7a0 0x7fa4b81c3950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800c4a0 tx=0x7fa4a800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.879+0000 7fa4b4ff9640 1 -- 172.21.15.16:0/3392030222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4a8017070 con 0x7fa4b812b7a0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.879+0000 7fa4b4ff9640 1 -- 172.21.15.16:0/3392030222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a8005c10 con 0x7fa4b812b7a0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.879+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b8133690 con 0x7fa4b812b7a0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.879+0000 7fa4b4ff9640 1 -- 172.21.15.16:0/3392030222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4a8007070 con 0x7fa4b812b7a0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.879+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b806e8b0 con 0x7fa4b812b7a0 2024-04-23T19:10:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.880+0000 7fa4b4ff9640 1 -- 172.21.15.16:0/3392030222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fa4a801a070 con 0x7fa4b812b7a0 2024-04-23T19:10:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.880+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b806e6c0 con 0x7fa4b812b7a0 2024-04-23T19:10:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.880+0000 7fa4b4ff9640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fa4a40385c0 0x7fa4a403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.880+0000 7fa4b4ff9640 1 -- 172.21.15.16:0/3392030222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7fa4a804a850 con 0x7fa4b812b7a0 2024-04-23T19:10:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.880+0000 7fa4b77fe640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fa4a40385c0 0x7fa4a403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.881+0000 7fa4b77fe640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fa4a40385c0 0x7fa4a403aa80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac036000 tx=0x7fa4ac008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:02.885+0000 7fa4b4ff9640 1 -- 172.21.15.16:0/3392030222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4a80124c0 con 0x7fa4b812b7a0 2024-04-23T19:10:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.065+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4b8063660 con 0x7fa4a40385c0 2024-04-23T19:10:03.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.067+0000 7fa4b4ff9640 1 -- 172.21.15.16:0/3392030222 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31844 (secure 0 0 0) 0x7fa4b8063660 con 0x7fa4a40385c0 2024-04-23T19:10:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:10:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:10:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.068+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fa4a40385c0 msgr2=0x7fa4a403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.068+0000 7fa4be679640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fa4a40385c0 0x7fa4a403aa80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac036000 tx=0x7fa4ac008030 comp rx=0 tx=0).stop 2024-04-23T19:10:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812b7a0 msgr2=0x7fa4b81c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812b7a0 0x7fa4b81c3950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800c4a0 tx=0x7fa4a800c960 comp rx=0 tx=0).stop 2024-04-23T19:10:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 shutdown_connections 2024-04-23T19:10:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fa4a40385c0 0x7fa4a403aa80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 --2- 172.21.15.16:0/3392030222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b812b7a0 0x7fa4b81c3950 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 >> 172.21.15.16:0/3392030222 conn(0x7fa4b80a15b0 msgr2=0x7fa4b812a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 shutdown_connections 2024-04-23T19:10:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.069+0000 7fa4be679640 1 -- 172.21.15.16:0/3392030222 wait complete. 2024-04-23T19:10:03.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-23T19:10:02.639688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":118064,"kb_used_data":1656,"kb_used_omap":6,"kb_used_meta":116345,"kb_avail":377369296,"statfs":{"total":386547056640,"available":386426159104,"internally_reserved":0,"allocated":1695744,"data_stored":1334608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":119138071},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.202503"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576977+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:10:00.576977+0000","last_peered":"2024-04-23T19:10:00.576977+0000","last_clean":"2024-04-23T19:10:00.576977+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:10:00.576977+0000","last_undegraded":"2024-04-23T19:10:00.576977+0000","last_fullsized":"2024-04-23T19:10:00.576977+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.025384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T19:09:59.451676+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:09:59.451676+0000","last_peered":"2024-04-23T19:09:59.451676+0000","last_clean":"2024-04-23T19:09:59.451676+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:09:59.451676+0000","last_undegraded":"2024-04-23T19:09:59.451676+0000","last_fullsized":"2024-04-23T19:09:59.451676+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:31:58.176909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576956+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:10:00.576956+0000","last_peered":"2024-04-23T19:10:00.576956+0000","last_clean":"2024-04-23T19:10:00.576956+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:10:00.576956+0000","last_undegraded":"2024-04-23T19:10:00.576956+0000","last_fullsized":"2024-04-23T19:10:00.576956+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:36.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T19:09:59.451504+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:09:59.451504+0000","last_peered":"2024-04-23T19:09:59.451504+0000","last_clean":"2024-04-23T19:09:59.451504+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:09:59.451504+0000","last_undegraded":"2024-04-23T19:09:59.451504+0000","last_fullsized":"2024-04-23T19:09:59.451504+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:42.755016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576956+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:10:00.576956+0000","last_peered":"2024-04-23T19:10:00.576956+0000","last_clean":"2024-04-23T19:10:00.576956+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:10:00.576956+0000","last_undegraded":"2024-04-23T19:10:00.576956+0000","last_fullsized":"2024-04-23T19:10:00.576956+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:51.612339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.444409+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:10:00.444409+0000","last_peered":"2024-04-23T19:10:00.444409+0000","last_clean":"2024-04-23T19:10:00.444409+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:10:00.444409+0000","last_undegraded":"2024-04-23T19:10:00.444409+0000","last_fullsized":"2024-04-23T19:10:00.444409+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:45.326443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576982+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:10:00.576982+0000","last_peered":"2024-04-23T19:10:00.576982+0000","last_clean":"2024-04-23T19:10:00.576982+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:10:00.576982+0000","last_undegraded":"2024-04-23T19:10:00.576982+0000","last_fullsized":"2024-04-23T19:10:00.576982+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:17.746806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:00.576960+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:10:00.576960+0000","last_peered":"2024-04-23T19:10:00.576960+0000","last_clean":"2024-04-23T19:10:00.576960+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:10:00.576960+0000","last_undegraded":"2024-04-23T19:10:00.576960+0000","last_fullsized":"2024-04-23T19:10:00.576960+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:09.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":167,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T19:10:01.389405+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:10:01.389405+0000","last_peered":"2024-04-23T19:10:01.389405+0000","last_clean":"2024-04-23T19:10:01.389405+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:10:01.389405+0000","last_undegraded":"2024-04-23T19:10:01.389405+0000","last_fullsized":"2024-04-23T19:10:01.389405+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:35.747545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":40478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":626826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":630846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35464,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336376,"statfs":{"total":96636764160,"available":96600449024,"internally_reserved":0,"allocated":122880,"data_stored":36458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:10:03.083 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T19:10:03.083 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T19:10:03.083 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-23T19:10:03.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T19:10:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.181+0000 7f0c115e6640 1 Processor -- start 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.181+0000 7f0c115e6640 1 -- start start 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.181+0000 7f0c115e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c0c096400 0x7f0c0c131d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.181+0000 7f0c115e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c0c095d90 con 0x7f0c0c12cda0 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.181+0000 7f0c115e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0c12c9e0 con 0x7f0c0c096400 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.182+0000 7f0c0affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0c0c12cda0 0x7f0c0c12d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60614/0 (socket says 172.21.15.16:60614) 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.182+0000 7f0c0a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c0c096400 0x7f0c0c131d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.182+0000 7f0c0affd640 1 -- 172.21.15.16:0/552575878 learned_addr learned my addr 172.21.15.16:0/552575878 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.182+0000 7f0c09ffb640 1 -- 172.21.15.16:0/552575878 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242107489 0 0) 0x7f0c0c095d90 con 0x7f0c0c12cda0 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.182+0000 7f0c09ffb640 1 -- 172.21.15.16:0/552575878 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bf4003520 con 0x7f0c0c12cda0 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.182+0000 7f0c0a7fc640 1 -- 172.21.15.16:0/552575878 >> v1:172.21.15.16:6789/0 conn(0x7f0c0c12cda0 legacy=0x7f0c0c12d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.182+0000 7f0c0a7fc640 1 -- 172.21.15.16:0/552575878 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c12d890 con 0x7f0c0c096400 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c0a7fc640 1 --2- 172.21.15.16:0/552575878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c131d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009720 tx=0x7f0c0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f676b873feb87c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c09ffb640 1 -- 172.21.15.16:0/552575878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c000316d0 con 0x7f0c0c096400 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c09ffb640 1 -- 172.21.15.16:0/552575878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c00037070 con 0x7f0c0c096400 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c09ffb640 1 -- 172.21.15.16:0/552575878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c000319a0 con 0x7f0c0c096400 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c115e6640 1 -- 172.21.15.16:0/552575878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 msgr2=0x7f0c0c131d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c115e6640 1 --2- 172.21.15.16:0/552575878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c131d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009720 tx=0x7f0c0002b840 comp rx=0 tx=0).stop 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c115e6640 1 -- 172.21.15.16:0/552575878 shutdown_connections 2024-04-23T19:10:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c115e6640 1 --2- 172.21.15.16:0/552575878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c131d60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c115e6640 1 -- 172.21.15.16:0/552575878 >> 172.21.15.16:0/552575878 conn(0x7f0c0c09a200 msgr2=0x7f0c0c097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c115e6640 1 -- 172.21.15.16:0/552575878 shutdown_connections 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.183+0000 7f0c115e6640 1 -- 172.21.15.16:0/552575878 wait complete. 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.184+0000 7f0c115e6640 1 Processor -- start 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.184+0000 7f0c115e6640 1 -- start start 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c115e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c12dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c115e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c096cf0 con 0x7f0c0c096400 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c12dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c12dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57042/0 (socket says 172.21.15.16:57042) 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c0affd640 1 -- 172.21.15.16:0/1669325272 learned_addr learned my addr 172.21.15.16:0/1669325272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c0affd640 1 -- 172.21.15.16:0/1669325272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c12fce0 con 0x7f0c0c096400 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c0affd640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c12dd60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400cb10 tx=0x7f0c0400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:03.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0bf37fe640 1 -- 172.21.15.16:0/1669325272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c0400abc0 con 0x7f0c0c096400 2024-04-23T19:10:03.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0bf37fe640 1 -- 172.21.15.16:0/1669325272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0400ad60 con 0x7f0c0c096400 2024-04-23T19:10:03.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0bf37fe640 1 -- 172.21.15.16:0/1669325272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c04012550 con 0x7f0c0c096400 2024-04-23T19:10:03.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c131ae0 con 0x7f0c0c096400 2024-04-23T19:10:03.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.185+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c12e2a0 con 0x7f0c0c096400 2024-04-23T19:10:03.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.186+0000 7f0bf37fe640 1 -- 172.21.15.16:0/1669325272 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0c040126f0 con 0x7f0c0c096400 2024-04-23T19:10:03.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.186+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c063660 con 0x7f0c0c096400 2024-04-23T19:10:03.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.186+0000 7f0bf37fe640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0bf80385c0 0x7f0bf803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:03.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.186+0000 7f0bf37fe640 1 -- 172.21.15.16:0/1669325272 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7f0c0404bc50 con 0x7f0c0c096400 2024-04-23T19:10:03.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.187+0000 7f0c0a7fc640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0bf80385c0 0x7f0bf803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:03.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.187+0000 7f0c0a7fc640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0bf80385c0 0x7f0bf803aa80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c00008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:03.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.191+0000 7f0bf37fe640 1 -- 172.21.15.16:0/1669325272 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c0401b920 con 0x7f0c0c096400 2024-04-23T19:10:03.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.415+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0c0c067f20 con 0x7f0c0c096400 2024-04-23T19:10:03.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.415+0000 7f0bf37fe640 1 -- 172.21.15.16:0/1669325272 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0c0401b400 con 0x7f0c0c096400 2024-04-23T19:10:03.416 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:10:03.416 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T19:10:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0bf80385c0 msgr2=0x7f0bf803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0bf80385c0 0x7f0bf803aa80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c00008030 comp rx=0 tx=0).stop 2024-04-23T19:10:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 msgr2=0x7f0c0c12dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c12dd60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400cb10 tx=0x7f0c0400a060 comp rx=0 tx=0).stop 2024-04-23T19:10:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 shutdown_connections 2024-04-23T19:10:03.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0bf80385c0 0x7f0bf803aa80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 --2- 172.21.15.16:0/1669325272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c0c096400 0x7f0c0c12dd60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 >> 172.21.15.16:0/1669325272 conn(0x7f0c0c09a200 msgr2=0x7f0c0c121e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 shutdown_connections 2024-04-23T19:10:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:10:03.417+0000 7f0c115e6640 1 -- 172.21.15.16:0/1669325272 wait complete. 2024-04-23T19:10:03.429 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-23T19:10:03.429 INFO:teuthology.run_tasks:Running task rgw... 2024-04-23T19:10:03.445 DEBUG:tasks.rgw:config is {'client.0': None} 2024-04-23T19:10:03.445 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-04-23T19:10:03.445 INFO:tasks.rgw:Creating data pools 2024-04-23T19:10:03.445 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-04-23T19:10:03.445 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd erasure-code-profile set client.0 k=2 m=1 crush-failure-domain=osd --cluster ceph 2024-04-23T19:10:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.560+0000 7fcd99ae0640 1 Processor -- start 2024-04-23T19:10:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.560+0000 7fcd99ae0640 1 -- start start 2024-04-23T19:10:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.560+0000 7fcd99ae0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd94117f20 0x7fcd9411a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.560+0000 7fcd99ae0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9411a850 con 0x7fcd94110120 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.560+0000 7fcd99ae0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9411ba30 con 0x7fcd94117f20 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.560+0000 7fcd92ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd94117f20 0x7fcd9411a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.561+0000 7fcd937fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd94110120 0x7fcd94110500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49680/0 (socket says 172.21.15.63:49680) 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.561+0000 7fcd937fe640 1 -- 172.21.15.63:0/3441980016 learned_addr learned my addr 172.21.15.63:0/3441980016 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.561+0000 7fcd92ffd640 1 -- 172.21.15.63:0/3441980016 >> v1:172.21.15.16:6789/0 conn(0x7fcd94110120 legacy=0x7fcd94110500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.561+0000 7fcd92ffd640 1 -- 172.21.15.63:0/3441980016 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9411bbb0 con 0x7fcd94117f20 2024-04-23T19:10:03.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.561+0000 7fcd927fc640 1 -- 172.21.15.63:0/3441980016 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262281588 0 0) 0x7fcd9411a850 con 0x7fcd94110120 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd92ffd640 1 --2- 172.21.15.63:0/3441980016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94117f20 0x7fcd9411a310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009720 tx=0x7fcd8402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b16361c322e8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd927fc640 1 -- 172.21.15.63:0/3441980016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd840316d0 con 0x7fcd94117f20 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd927fc640 1 -- 172.21.15.63:0/3441980016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd84037070 con 0x7fcd94117f20 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd927fc640 1 -- 172.21.15.63:0/3441980016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd840319c0 con 0x7fcd94117f20 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd99ae0640 1 -- 172.21.15.63:0/3441980016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94117f20 msgr2=0x7fcd9411a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd99ae0640 1 --2- 172.21.15.63:0/3441980016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94117f20 0x7fcd9411a310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009720 tx=0x7fcd8402b840 comp rx=0 tx=0).stop 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd99ae0640 1 -- 172.21.15.63:0/3441980016 shutdown_connections 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd99ae0640 1 --2- 172.21.15.63:0/3441980016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94117f20 0x7fcd9411a310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd99ae0640 1 -- 172.21.15.63:0/3441980016 >> 172.21.15.63:0/3441980016 conn(0x7fcd9406f2a0 msgr2=0x7fcd940716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.562+0000 7fcd99ae0640 1 -- 172.21.15.63:0/3441980016 shutdown_connections 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.563+0000 7fcd99ae0640 1 -- 172.21.15.63:0/3441980016 wait complete. 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.563+0000 7fcd99ae0640 1 Processor -- start 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.563+0000 7fcd99ae0640 1 -- start start 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd99ae0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94110120 0x7fcd941a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd99ae0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9411af40 con 0x7fcd94110120 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94110120 0x7fcd941a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94110120 0x7fcd941a6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37438/0 (socket says 172.21.15.63:37438) 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd937fe640 1 -- 172.21.15.63:0/1480278448 learned_addr learned my addr 172.21.15.63:0/1480278448 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd937fe640 1 -- 172.21.15.63:0/1480278448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd941a7410 con 0x7fcd94110120 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd937fe640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94110120 0x7fcd941a6ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800c3e0 tx=0x7fcd8800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.564+0000 7fcd7bfff640 1 -- 172.21.15.63:0/1480278448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd88007410 con 0x7fcd94110120 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.565+0000 7fcd7bfff640 1 -- 172.21.15.63:0/1480278448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880075d0 con 0x7fcd94110120 2024-04-23T19:10:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.565+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd941a7700 con 0x7fcd94110120 2024-04-23T19:10:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.565+0000 7fcd7bfff640 1 -- 172.21.15.63:0/1480278448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd88004950 con 0x7fcd94110120 2024-04-23T19:10:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.565+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd941ab760 con 0x7fcd94110120 2024-04-23T19:10:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.565+0000 7fcd7bfff640 1 -- 172.21.15.63:0/1480278448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcd88011070 con 0x7fcd94110120 2024-04-23T19:10:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.566+0000 7fcd7bfff640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd800385c0 0x7fcd8003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.566+0000 7fcd7bfff640 1 -- 172.21.15.63:0/1480278448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3482+0+0 (secure 0 0 0) 0x7fcd8804b270 con 0x7fcd94110120 2024-04-23T19:10:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.566+0000 7fcd92ffd640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd800385c0 0x7fcd8003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.566+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd94067960 con 0x7fcd94110120 2024-04-23T19:10:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.566+0000 7fcd92ffd640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd800385c0 0x7fcd8003aa80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd84036000 tx=0x7fcd84008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.571+0000 7fcd7bfff640 1 -- 172.21.15.63:0/1480278448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd88011c60 con 0x7fcd94110120 2024-04-23T19:10:03.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:03.763+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]} v 0) -- 0x7fcd941a7930 con 0x7fcd94110120 2024-04-23T19:10:04.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.451+0000 7fcd7bfff640 1 -- 172.21.15.63:0/1480278448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]=0 v16) ==== 151+0+0 (secure 0 0 0) 0x7fcd8801c990 con 0x7fcd94110120 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd800385c0 msgr2=0x7fcd8003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd800385c0 0x7fcd8003aa80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd84036000 tx=0x7fcd84008030 comp rx=0 tx=0).stop 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94110120 msgr2=0x7fcd941a6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94110120 0x7fcd941a6ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800c3e0 tx=0x7fcd8800c8a0 comp rx=0 tx=0).stop 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 shutdown_connections 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd800385c0 0x7fcd8003aa80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 --2- 172.21.15.63:0/1480278448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd94110120 0x7fcd941a6ed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.453+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 >> 172.21.15.63:0/1480278448 conn(0x7fcd9406f2a0 msgr2=0x7fcd9406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:04.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.454+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 shutdown_connections 2024-04-23T19:10:04.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.454+0000 7fcd99ae0640 1 -- 172.21.15.63:0/1480278448 wait complete. 2024-04-23T19:10:04.468 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.576+0000 7f6ff477e640 1 Processor -- start 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.576+0000 7f6ff477e640 1 -- start start 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.576+0000 7f6ff477e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fec10f010 0x7f6fec10f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.576+0000 7f6ff477e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fec10fab0 con 0x7f6fec118110 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.576+0000 7f6ff477e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6fec11cc30 con 0x7f6fec10f010 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.576+0000 7f6ff251a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fec10f010 0x7f6fec10f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.577+0000 7f6ff251a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fec10f010 0x7f6fec10f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37442/0 (socket says 172.21.15.63:37442) 2024-04-23T19:10:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.577+0000 7f6ff251a640 1 -- 172.21.15.63:0/2151146945 learned_addr learned my addr 172.21.15.63:0/2151146945 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.577+0000 7f6ff251a640 1 -- 172.21.15.63:0/2151146945 >> v1:172.21.15.16:6789/0 conn(0x7f6fec118110 legacy=0x7f6fec11a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.577+0000 7f6ff251a640 1 -- 172.21.15.63:0/2151146945 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec11cdb0 con 0x7f6fec10f010 2024-04-23T19:10:04.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff251a640 1 --2- 172.21.15.63:0/2151146945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec10f3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80097f0 tx=0x7f6fe802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48550178688875d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff1518640 1 -- 172.21.15.63:0/2151146945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe8035020 con 0x7f6fec10f010 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff1518640 1 -- 172.21.15.63:0/2151146945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe80317a0 con 0x7f6fec10f010 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff1518640 1 -- 172.21.15.63:0/2151146945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe8031aa0 con 0x7f6fec10f010 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff477e640 1 -- 172.21.15.63:0/2151146945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 msgr2=0x7f6fec10f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff477e640 1 --2- 172.21.15.63:0/2151146945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec10f3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80097f0 tx=0x7f6fe802b910 comp rx=0 tx=0).stop 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff477e640 1 -- 172.21.15.63:0/2151146945 shutdown_connections 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff477e640 1 --2- 172.21.15.63:0/2151146945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec10f3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff477e640 1 -- 172.21.15.63:0/2151146945 >> 172.21.15.63:0/2151146945 conn(0x7f6fec06f460 msgr2=0x7f6fec071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.578+0000 7f6ff477e640 1 -- 172.21.15.63:0/2151146945 shutdown_connections 2024-04-23T19:10:04.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.579+0000 7f6ff477e640 1 -- 172.21.15.63:0/2151146945 wait complete. 2024-04-23T19:10:04.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.579+0000 7f6ff477e640 1 Processor -- start 2024-04-23T19:10:04.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.579+0000 7f6ff477e640 1 -- start start 2024-04-23T19:10:04.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.579+0000 7f6ff477e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:04.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.579+0000 7f6ff477e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec11c140 con 0x7f6fec10f010 2024-04-23T19:10:04.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6ff251a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:04.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6ff251a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec1a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37458/0 (socket says 172.21.15.63:37458) 2024-04-23T19:10:04.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6ff251a640 1 -- 172.21.15.63:0/4052541978 learned_addr learned my addr 172.21.15.63:0/4052541978 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6ff251a640 1 -- 172.21.15.63:0/4052541978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec1a5310 con 0x7f6fec10f010 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6ff251a640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec1a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80090b0 tx=0x7f6fe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6fe2ffd640 1 -- 172.21.15.63:0/4052541978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe8049070 con 0x7f6fec10f010 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6fe2ffd640 1 -- 172.21.15.63:0/4052541978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe8031a80 con 0x7f6fec10f010 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec1a55a0 con 0x7f6fec10f010 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6fe2ffd640 1 -- 172.21.15.63:0/4052541978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe8044550 con 0x7f6fec10f010 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.580+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec1a94c0 con 0x7f6fec10f010 2024-04-23T19:10:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.581+0000 7f6fe2ffd640 1 -- 172.21.15.63:0/4052541978 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6fe804c070 con 0x7f6fec10f010 2024-04-23T19:10:04.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.581+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec0505e0 con 0x7f6fec10f010 2024-04-23T19:10:04.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.581+0000 7f6fe2ffd640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6fd0038570 0x7f6fd003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:04.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.581+0000 7f6fe2ffd640 1 -- 172.21.15.63:0/4052541978 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3774+0+0 (secure 0 0 0) 0x7f6fe806db60 con 0x7f6fec10f010 2024-04-23T19:10:04.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.582+0000 7f6ff1d19640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6fd0038570 0x7f6fd003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:04.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.582+0000 7f6ff1d19640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6fd0038570 0x7f6fd003aa30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000f60 tx=0x7f6fdc007a50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:04.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.586+0000 7f6fe2ffd640 1 -- 172.21.15.63:0/4052541978 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fe803f070 con 0x7f6fec10f010 2024-04-23T19:10:04.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:04.778+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"} v 0) -- 0x7f6fec118110 con 0x7f6fec10f010 2024-04-23T19:10:06.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.456+0000 7f6fe2ffd640 1 -- 172.21.15.63:0/4052541978 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"}]=0 pool 'default.rgw.buckets.data' created v18) ==== 227+0+0 (secure 0 0 0) 0x7f6fe80393b0 con 0x7f6fec10f010 2024-04-23T19:10:06.456 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'default.rgw.buckets.data' created 2024-04-23T19:10:06.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6fd0038570 msgr2=0x7f6fd003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:06.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6fd0038570 0x7f6fd003aa30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000f60 tx=0x7f6fdc007a50 comp rx=0 tx=0).stop 2024-04-23T19:10:06.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 msgr2=0x7f6fec1a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:06.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec1a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80090b0 tx=0x7f6fe80046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:06.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 shutdown_connections 2024-04-23T19:10:06.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6fd0038570 0x7f6fd003aa30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:06.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 --2- 172.21.15.63:0/4052541978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec10f010 0x7f6fec1a4dd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:06.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.458+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 >> 172.21.15.63:0/4052541978 conn(0x7f6fec06f460 msgr2=0x7f6fec1184f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:06.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.459+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 shutdown_connections 2024-04-23T19:10:06.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.459+0000 7f6ff477e640 1 -- 172.21.15.63:0/4052541978 wait complete. 2024-04-23T19:10:06.481 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-04-23T19:10:06.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.586+0000 7fd34e366640 1 Processor -- start 2024-04-23T19:10:06.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.586+0000 7fd34e366640 1 -- start start 2024-04-23T19:10:06.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.587+0000 7fd34e366640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3481125f0 0x7fd3481129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:06.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.587+0000 7fd34e366640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd348117ba0 con 0x7fd348116fe0 2024-04-23T19:10:06.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.587+0000 7fd34e366640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd348113090 con 0x7fd3481125f0 2024-04-23T19:10:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.587+0000 7fd3477fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd348116fe0 0x7fd348117490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49698/0 (socket says 172.21.15.63:49698) 2024-04-23T19:10:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.587+0000 7fd347fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3481125f0 0x7fd3481129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.587+0000 7fd3477fe640 1 -- 172.21.15.63:0/2033770582 learned_addr learned my addr 172.21.15.63:0/2033770582 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.587+0000 7fd347fff640 1 -- 172.21.15.63:0/2033770582 >> v1:172.21.15.16:6789/0 conn(0x7fd348116fe0 legacy=0x7fd348117490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.588+0000 7fd347fff640 1 -- 172.21.15.63:0/2033770582 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd348117d70 con 0x7fd3481125f0 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.588+0000 7fd347fff640 1 --2- 172.21.15.63:0/2033770582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481129d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd330009720 tx=0x7fd33002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4830f9d9ee3d88f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd346ffd640 1 -- 172.21.15.63:0/2033770582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3300316d0 con 0x7fd3481125f0 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd346ffd640 1 -- 172.21.15.63:0/2033770582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd330037070 con 0x7fd3481125f0 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd346ffd640 1 -- 172.21.15.63:0/2033770582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3300319c0 con 0x7fd3481125f0 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd34e366640 1 -- 172.21.15.63:0/2033770582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 msgr2=0x7fd3481129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd34e366640 1 --2- 172.21.15.63:0/2033770582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481129d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd330009720 tx=0x7fd33002b840 comp rx=0 tx=0).stop 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd34e366640 1 -- 172.21.15.63:0/2033770582 shutdown_connections 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd34e366640 1 --2- 172.21.15.63:0/2033770582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481129d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd34e366640 1 -- 172.21.15.63:0/2033770582 >> 172.21.15.63:0/2033770582 conn(0x7fd34806d8c0 msgr2=0x7fd34806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd34e366640 1 -- 172.21.15.63:0/2033770582 shutdown_connections 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.589+0000 7fd34e366640 1 -- 172.21.15.63:0/2033770582 wait complete. 2024-04-23T19:10:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.590+0000 7fd34e366640 1 Processor -- start 2024-04-23T19:10:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.590+0000 7fd34e366640 1 -- start start 2024-04-23T19:10:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.590+0000 7fd34e366640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.590+0000 7fd34e366640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd34811e650 con 0x7fd3481125f0 2024-04-23T19:10:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.590+0000 7fd347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37486/0 (socket says 172.21.15.63:37486) 2024-04-23T19:10:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd347fff640 1 -- 172.21.15.63:0/93497672 learned_addr learned my addr 172.21.15.63:0/93497672 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:06.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd347fff640 1 -- 172.21.15.63:0/93497672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3481ad2e0 con 0x7fd3481125f0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd347fff640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481acd10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd330034040 tx=0x7fd3300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd344ff9640 1 -- 172.21.15.63:0/93497672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd330032770 con 0x7fd3481125f0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd344ff9640 1 -- 172.21.15.63:0/93497672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd330032910 con 0x7fd3481125f0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd344ff9640 1 -- 172.21.15.63:0/93497672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd330032c30 con 0x7fd3481125f0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3481ad510 con 0x7fd3481125f0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.591+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3481a6da0 con 0x7fd3481125f0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.592+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3480505e0 con 0x7fd3481125f0 2024-04-23T19:10:06.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.592+0000 7fd344ff9640 1 -- 172.21.15.63:0/93497672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd33004c060 con 0x7fd3481125f0 2024-04-23T19:10:06.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.593+0000 7fd344ff9640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd328038570 0x7fd32803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:06.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.593+0000 7fd344ff9640 1 -- 172.21.15.63:0/93497672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4249+0+0 (secure 0 0 0) 0x7fd33002f050 con 0x7fd3481125f0 2024-04-23T19:10:06.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.593+0000 7fd3477fe640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd328038570 0x7fd32803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:06.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.593+0000 7fd3477fe640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd328038570 0x7fd32803aa30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd338007fd0 tx=0x7fd338007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:06.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.599+0000 7fd344ff9640 1 -- 172.21.15.63:0/93497672 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd330040070 con 0x7fd3481125f0 2024-04-23T19:10:06.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:06.786+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) -- 0x7fd348116fe0 con 0x7fd3481125f0 2024-04-23T19:10:07.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.462+0000 7fd344ff9640 1 -- 172.21.15.63:0/93497672 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v19) ==== 185+0+0 (secure 0 0 0) 0x7fd330032dd0 con 0x7fd3481125f0 2024-04-23T19:10:07.463 INFO:teuthology.orchestra.run.smithi063.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-23T19:10:07.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.464+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd328038570 msgr2=0x7fd32803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:07.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.464+0000 7fd34e366640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd328038570 0x7fd32803aa30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd338007fd0 tx=0x7fd338007ac0 comp rx=0 tx=0).stop 2024-04-23T19:10:07.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.464+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 msgr2=0x7fd3481acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:07.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.464+0000 7fd34e366640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481acd10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd330034040 tx=0x7fd3300046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.465+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 shutdown_connections 2024-04-23T19:10:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.466+0000 7fd34e366640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd328038570 0x7fd32803aa30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.466+0000 7fd34e366640 1 --2- 172.21.15.63:0/93497672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3481125f0 0x7fd3481acd10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:07.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.466+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 >> 172.21.15.63:0/93497672 conn(0x7fd34806d8c0 msgr2=0x7fd348114130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.466+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 shutdown_connections 2024-04-23T19:10:07.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.467+0000 7fd34e366640 1 -- 172.21.15.63:0/93497672 wait complete. 2024-04-23T19:10:07.493 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-04-23T19:10:07.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc668cb1640 1 Processor -- start 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc668cb1640 1 -- start start 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc668cb1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc664117540 0x7fc664119930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc668cb1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc664119f30 con 0x7fc6641148e0 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc668cb1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc66411b110 con 0x7fc664117540 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc65bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc664117540 0x7fc664119930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc66259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc6641148e0 0x7fc664114cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49700/0 (socket says 172.21.15.63:49700) 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc66259c640 1 -- 172.21.15.63:0/904818012 learned_addr learned my addr 172.21.15.63:0/904818012 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.603+0000 7fc65bfff640 1 -- 172.21.15.63:0/904818012 >> v1:172.21.15.16:6789/0 conn(0x7fc6641148e0 legacy=0x7fc664114cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.604+0000 7fc65bfff640 1 -- 172.21.15.63:0/904818012 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66411b2c0 con 0x7fc664117540 2024-04-23T19:10:07.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.604+0000 7fc65bfff640 1 --2- 172.21.15.63:0/904818012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc664117540 0x7fc664119930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00a8c0 tx=0x7fc64c02cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31287f34eecd9a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.604+0000 7fc661d9b640 1 -- 172.21.15.63:0/904818012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc64c03a070 con 0x7fc664117540 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc661d9b640 1 -- 172.21.15.63:0/904818012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c02da00 con 0x7fc664117540 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc661d9b640 1 -- 172.21.15.63:0/904818012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc64c00b040 con 0x7fc664117540 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc668cb1640 1 -- 172.21.15.63:0/904818012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc664117540 msgr2=0x7fc664119930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc668cb1640 1 --2- 172.21.15.63:0/904818012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc664117540 0x7fc664119930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00a8c0 tx=0x7fc64c02cd20 comp rx=0 tx=0).stop 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc668cb1640 1 -- 172.21.15.63:0/904818012 shutdown_connections 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc668cb1640 1 --2- 172.21.15.63:0/904818012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc664117540 0x7fc664119930 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc668cb1640 1 -- 172.21.15.63:0/904818012 >> 172.21.15.63:0/904818012 conn(0x7fc66406f460 msgr2=0x7fc664071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc668cb1640 1 -- 172.21.15.63:0/904818012 shutdown_connections 2024-04-23T19:10:07.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.605+0000 7fc668cb1640 1 -- 172.21.15.63:0/904818012 wait complete. 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.606+0000 7fc668cb1640 1 Processor -- start 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.606+0000 7fc668cb1640 1 -- start start 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.606+0000 7fc668cb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6641148e0 0x7fc6641a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.606+0000 7fc668cb1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc664073d70 con 0x7fc6641148e0 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.606+0000 7fc66259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6641148e0 0x7fc6641a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.606+0000 7fc66259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6641148e0 0x7fc6641a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37504/0 (socket says 172.21.15.63:37504) 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.606+0000 7fc66259c640 1 -- 172.21.15.63:0/4094510298 learned_addr learned my addr 172.21.15.63:0/4094510298 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.607+0000 7fc66259c640 1 -- 172.21.15.63:0/4094510298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6641a3240 con 0x7fc6641148e0 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.607+0000 7fc66259c640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6641148e0 0x7fc6641a2d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc65400c480 tx=0x7fc65400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.607+0000 7fc65affd640 1 -- 172.21.15.63:0/4094510298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc654017070 con 0x7fc6641148e0 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.607+0000 7fc65affd640 1 -- 172.21.15.63:0/4094510298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc654005c10 con 0x7fc6641148e0 2024-04-23T19:10:07.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.607+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6641a3530 con 0x7fc6641148e0 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.607+0000 7fc65affd640 1 -- 172.21.15.63:0/4094510298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6540070d0 con 0x7fc6641148e0 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.607+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641a75a0 con 0x7fc6641148e0 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.608+0000 7fc65affd640 1 -- 172.21.15.63:0/4094510298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc65401a070 con 0x7fc6641148e0 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.608+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6641a3760 con 0x7fc6641148e0 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.608+0000 7fc65affd640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc644038570 0x7fc64403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.608+0000 7fc65affd640 1 -- 172.21.15.63:0/4094510298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4260+0+0 (secure 0 0 0) 0x7fc65404acc0 con 0x7fc6641148e0 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.608+0000 7fc65bfff640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc644038570 0x7fc64403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:07.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.609+0000 7fc65bfff640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc644038570 0x7fc64403aa30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc64c037000 tx=0x7fc64c031000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:07.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.614+0000 7fc65affd640 1 -- 172.21.15.63:0/4094510298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc654050020 con 0x7fc6641148e0 2024-04-23T19:10:07.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:07.798+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) -- 0x7fc664117540 con 0x7fc6641148e0 2024-04-23T19:10:08.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.463+0000 7fc65affd640 1 -- 172.21.15.63:0/4094510298 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v20) ==== 169+0+0 (secure 0 0 0) 0x7fc654010940 con 0x7fc6641148e0 2024-04-23T19:10:08.464 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'default.rgw.buckets.index' created 2024-04-23T19:10:08.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc644038570 msgr2=0x7fc64403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:08.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc644038570 0x7fc64403aa30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc64c037000 tx=0x7fc64c031000 comp rx=0 tx=0).stop 2024-04-23T19:10:08.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6641148e0 msgr2=0x7fc6641a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:08.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6641148e0 0x7fc6641a2d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc65400c480 tx=0x7fc65400c940 comp rx=0 tx=0).stop 2024-04-23T19:10:08.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 shutdown_connections 2024-04-23T19:10:08.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc644038570 0x7fc64403aa30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:08.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 --2- 172.21.15.63:0/4094510298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6641148e0 0x7fc6641a2d00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:08.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 >> 172.21.15.63:0/4094510298 conn(0x7fc66406f460 msgr2=0x7fc664119310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:08.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 shutdown_connections 2024-04-23T19:10:08.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.466+0000 7fc668cb1640 1 -- 172.21.15.63:0/4094510298 wait complete. 2024-04-23T19:10:08.486 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f28198b2640 1 Processor -- start 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f28198b2640 1 -- start start 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f28198b2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2814110120 0x7f2814110500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f28198b2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2814110b90 con 0x7f2814114fe0 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f28198b2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2814115ba0 con 0x7f2814110120 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f2812ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2814110120 0x7f2814110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f28127fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2814114fe0 0x7f2814115490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:47712/0 (socket says 172.21.15.63:47712) 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f28127fc640 1 -- 172.21.15.63:0/3389041117 learned_addr learned my addr 172.21.15.63:0/3389041117 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f2811ffb640 1 -- 172.21.15.63:0/3389041117 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300079954 0 0) 0x7f2814110b90 con 0x7f2814114fe0 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f2811ffb640 1 -- 172.21.15.63:0/3389041117 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27fc003520 con 0x7f2814114fe0 2024-04-23T19:10:08.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f2812ffd640 1 -- 172.21.15.63:0/3389041117 >> v1:172.21.15.16:6789/0 conn(0x7f2814114fe0 legacy=0x7f2814115490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:08.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.589+0000 7f2812ffd640 1 -- 172.21.15.63:0/3389041117 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2814115d50 con 0x7f2814110120 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.590+0000 7f2812ffd640 1 --2- 172.21.15.63:0/3389041117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f2814110500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2804009720 tx=0x7f280402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7012acccf3d49576 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.590+0000 7f2811ffb640 1 -- 172.21.15.63:0/3389041117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28040316d0 con 0x7f2814110120 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.590+0000 7f2811ffb640 1 -- 172.21.15.63:0/3389041117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2804037070 con 0x7f2814110120 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f2811ffb640 1 -- 172.21.15.63:0/3389041117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28040319c0 con 0x7f2814110120 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f28198b2640 1 -- 172.21.15.63:0/3389041117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 msgr2=0x7f2814110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f28198b2640 1 --2- 172.21.15.63:0/3389041117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f2814110500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2804009720 tx=0x7f280402b840 comp rx=0 tx=0).stop 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f28198b2640 1 -- 172.21.15.63:0/3389041117 shutdown_connections 2024-04-23T19:10:08.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f28198b2640 1 --2- 172.21.15.63:0/3389041117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f2814110500 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:08.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f28198b2640 1 -- 172.21.15.63:0/3389041117 >> 172.21.15.63:0/3389041117 conn(0x7f281406f460 msgr2=0x7f2814071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:08.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f28198b2640 1 -- 172.21.15.63:0/3389041117 shutdown_connections 2024-04-23T19:10:08.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.591+0000 7f28198b2640 1 -- 172.21.15.63:0/3389041117 wait complete. 2024-04-23T19:10:08.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.592+0000 7f28198b2640 1 Processor -- start 2024-04-23T19:10:08.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.592+0000 7f28198b2640 1 -- start start 2024-04-23T19:10:08.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.592+0000 7f28198b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f28141a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:08.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.592+0000 7f28198b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f281411b420 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.592+0000 7f2812ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f28141a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.592+0000 7f2812ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f28141a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55500/0 (socket says 172.21.15.63:55500) 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.592+0000 7f2812ffd640 1 -- 172.21.15.63:0/3504241604 learned_addr learned my addr 172.21.15.63:0/3504241604 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.593+0000 7f2812ffd640 1 -- 172.21.15.63:0/3504241604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28141a5230 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.593+0000 7f2812ffd640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f28141a4cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2804034040 tx=0x7f28040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.593+0000 7f27fbfff640 1 -- 172.21.15.63:0/3504241604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2804032560 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.593+0000 7f27fbfff640 1 -- 172.21.15.63:0/3504241604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2804032700 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.593+0000 7f27fbfff640 1 -- 172.21.15.63:0/3504241604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2804032a20 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.593+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28141a54c0 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.593+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141a9510 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.594+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28140505e0 con 0x7f2814110120 2024-04-23T19:10:08.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.594+0000 7f27fbfff640 1 -- 172.21.15.63:0/3504241604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f280404c060 con 0x7f2814110120 2024-04-23T19:10:08.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.595+0000 7f27fbfff640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f28000385f0 0x7f280003aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:08.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.595+0000 7f27fbfff640 1 -- 172.21.15.63:0/3504241604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4624+0+0 (secure 0 0 0) 0x7f280402f050 con 0x7f2814110120 2024-04-23T19:10:08.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.595+0000 7f28127fc640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f28000385f0 0x7f280003aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:08.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.595+0000 7f28127fc640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f28000385f0 0x7f280003aab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28080011e0 tx=0x7f2808007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:08.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.601+0000 7f27fbfff640 1 -- 172.21.15.63:0/3504241604 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2804040070 con 0x7f2814110120 2024-04-23T19:10:08.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:08.788+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) -- 0x7f28141a5690 con 0x7f2814110120 2024-04-23T19:10:09.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.466+0000 7f27fbfff640 1 -- 172.21.15.63:0/3504241604 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v21) ==== 187+0+0 (secure 0 0 0) 0x7f280404d910 con 0x7f2814110120 2024-04-23T19:10:09.467 INFO:teuthology.orchestra.run.smithi063.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-23T19:10:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.470+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f28000385f0 msgr2=0x7f280003aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.470+0000 7f28198b2640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f28000385f0 0x7f280003aab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28080011e0 tx=0x7f2808007970 comp rx=0 tx=0).stop 2024-04-23T19:10:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.470+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 msgr2=0x7f28141a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.470+0000 7f28198b2640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f28141a4cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2804034040 tx=0x7f28040046c0 comp rx=0 tx=0).stop 2024-04-23T19:10:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.471+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 shutdown_connections 2024-04-23T19:10:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.471+0000 7f28198b2640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f28000385f0 0x7f280003aab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.471+0000 7f28198b2640 1 --2- 172.21.15.63:0/3504241604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2814110120 0x7f28141a4cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.471+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 >> 172.21.15.63:0/3504241604 conn(0x7f281406f460 msgr2=0x7f281410ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.471+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 shutdown_connections 2024-04-23T19:10:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:09.472+0000 7f28198b2640 1 -- 172.21.15.63:0/3504241604 wait complete. 2024-04-23T19:10:09.498 DEBUG:tasks.rgw:Pools created 2024-04-23T19:10:09.498 INFO:tasks.rgw:Starting rgw... 2024-04-23T19:10:09.498 INFO:tasks.rgw:rgw client.0 config is {} 2024-04-23T19:10:09.498 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-23T19:10:09.498 DEBUG:teuthology.orchestra.run.smithi063:> sudo echo -n http://smithi063.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-23T19:10:09.526 INFO:teuthology.orchestra.run.smithi063.stdout:http://smithi063.front.sepia.ceph.com:80 2024-04-23T19:10:09.526 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-23T19:10:09.592 INFO:tasks.rgw.client.0:Restarting daemon 2024-04-23T19:10:09.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-04-23T19:10:09.635 INFO:tasks.rgw.client.0:Started 2024-04-23T19:10:09.635 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi063.front.sepia.ceph.com:80/ 2024-04-23T19:10:09.635 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-04-23T19:10:09.669 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-23T19:10:09.669 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-04-23T19:10:09.671 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-23T19:10:09.671 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-23T19:10:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80: Connection refused 2024-04-23T19:10:10.672 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-04-23T19:10:10.698 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-23T19:10:10.699 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-23T19:10:10.699 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-04-23T19:10:10.699 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-23T19:10:10.699 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80: Connection refused 2024-04-23T19:10:12.701 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-04-23T19:10:12.726 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-23T19:10:12.726 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-23T19:10:12.726 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-04-23T19:10:12.726 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-23T19:10:12.726 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80: Connection refused 2024-04-23T19:10:16.727 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-04-23T19:10:16.753 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-23T19:10:16.753 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-23T19:10:16.753 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-04-23T19:10:16.753 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-23T19:10:16.753 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80: Connection refused 2024-04-23T19:10:18.574 INFO:tasks.rgw.client.0.smithi063.stdout:2024-04-23T19:10:18.573+0000 7eff5b3dd880 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-23T19:10:24.754 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-04-23T19:10:24.778 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-23T19:10:24.778 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-04-23T19:10:24.780 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 62333 0 --:--:-- --:--:-- --:--:-- 93500 2024-04-23T19:10:24.781 INFO:teuthology.orchestra.run.smithi063.stdout:anonymous 2024-04-23T19:10:24.781 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-04-23T19:10:24.876 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-04-23T19:10:24.876 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-23T19:10:24.876 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-23T19:10:24.876 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-23T19:10:24.926 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:24.926 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31b071f800 1 Processor -- start 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31b071f800 1 -- start start 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31b071f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5634acc4f560 0x5634acc51950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31b071f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5634acafd7f0 con 0x5634acc4b170 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31b071f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5634acaece20 con 0x5634acc4f560 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31abbc2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5634acc4f560 0x5634acc51950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31ac3c3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5634acc4b170 0x5634acc4b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52598/0 (socket says 172.21.15.63:52598) 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.927+0000 7f31ac3c3640 1 -- 172.21.15.63:0/527170672 learned_addr learned my addr 172.21.15.63:0/527170672 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.928+0000 7f31abbc2640 1 -- 172.21.15.63:0/527170672 >> v1:172.21.15.16:6789/0 conn(0x5634acc4b170 legacy=0x5634acc4b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.928+0000 7f31abbc2640 1 -- 172.21.15.63:0/527170672 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634acb0bb50 con 0x5634acc4f560 2024-04-23T19:10:24.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.930+0000 7f31abbc2640 1 --2- 172.21.15.63:0/527170672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4f560 0x5634acc51950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31a000a470 tx=0x7f31a001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0276b2d176b5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.930+0000 7f31ab3c1640 1 -- 172.21.15.63:0/527170672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a0057070 con 0x5634acc4f560 2024-04-23T19:10:24.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.930+0000 7f31ab3c1640 1 -- 172.21.15.63:0/527170672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a0052570 con 0x5634acc4f560 2024-04-23T19:10:24.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.930+0000 7f31ab3c1640 1 -- 172.21.15.63:0/527170672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a0052870 con 0x5634acc4f560 2024-04-23T19:10:24.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.931+0000 7f31b071f800 1 -- 172.21.15.63:0/527170672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4f560 msgr2=0x5634acc51950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:24.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.931+0000 7f31b071f800 1 --2- 172.21.15.63:0/527170672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4f560 0x5634acc51950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31a000a470 tx=0x7f31a001ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:24.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.931+0000 7f31b071f800 1 -- 172.21.15.63:0/527170672 shutdown_connections 2024-04-23T19:10:24.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.931+0000 7f31b071f800 1 --2- 172.21.15.63:0/527170672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4f560 0x5634acc51950 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:24.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.931+0000 7f31b071f800 1 -- 172.21.15.63:0/527170672 >> 172.21.15.63:0/527170672 conn(0x5634acc467d0 msgr2=0x5634acc48bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:24.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.931+0000 7f31b071f800 1 -- 172.21.15.63:0/527170672 shutdown_connections 2024-04-23T19:10:24.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.931+0000 7f31b071f800 1 -- 172.21.15.63:0/527170672 wait complete. 2024-04-23T19:10:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.946+0000 7f31b071f800 1 Processor -- start 2024-04-23T19:10:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.946+0000 7f31b071f800 1 -- start start 2024-04-23T19:10:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.946+0000 7f31b071f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.946+0000 7f31b071f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634acb966e0 con 0x5634acc4b170 2024-04-23T19:10:24.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.946+0000 7f31ac3c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.946+0000 7f31ac3c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd3c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47790/0 (socket says 172.21.15.63:47790) 2024-04-23T19:10:24.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.946+0000 7f31ac3c3640 1 -- 172.21.15.63:0/3581754222 learned_addr learned my addr 172.21.15.63:0/3581754222 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:24.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.947+0000 7f31ac3c3640 1 -- 172.21.15.63:0/3581754222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634acd3c920 con 0x5634acc4b170 2024-04-23T19:10:24.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.947+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/3581754222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd3c350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f319c001e10 tx=0x7f319c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1d578778476d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.947+0000 7f309a60c640 1 -- 172.21.15.63:0/3581754222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c012020 con 0x5634acc4b170 2024-04-23T19:10:24.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.947+0000 7f309a60c640 1 -- 172.21.15.63:0/3581754222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319c00e5a0 con 0x5634acc4b170 2024-04-23T19:10:24.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.947+0000 7f309a60c640 1 -- 172.21.15.63:0/3581754222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c00e8e0 con 0x5634acc4b170 2024-04-23T19:10:24.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 -- 172.21.15.63:0/3581754222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 msgr2=0x5634acd3c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:24.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 --2- 172.21.15.63:0/3581754222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd3c350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f319c001e10 tx=0x7f319c00b780 comp rx=0 tx=0).stop 2024-04-23T19:10:24.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 -- 172.21.15.63:0/3581754222 shutdown_connections 2024-04-23T19:10:24.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 --2- 172.21.15.63:0/3581754222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd3c350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:24.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 -- 172.21.15.63:0/3581754222 >> 172.21.15.63:0/3581754222 conn(0x5634acc467d0 msgr2=0x5634acd37c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 -- 172.21.15.63:0/3581754222 shutdown_connections 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 -- 172.21.15.63:0/3581754222 wait complete. 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 Processor -- start 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.948+0000 7f31b071f800 1 -- start start 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f31b071f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd43f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f31b071f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634acc52220 con 0x5634acc4b170 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f31ac3c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd43f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f31ac3c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd43f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47796/0 (socket says 172.21.15.63:47796) 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f31ac3c3640 1 -- 172.21.15.63:0/915944573 learned_addr learned my addr 172.21.15.63:0/915944573 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f31ac3c3640 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634acd444a0 con 0x5634acc4b170 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd43f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f319c00b7c0 tx=0x7f319c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.949+0000 7f3098e09640 1 -- 172.21.15.63:0/915944573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c012030 con 0x5634acc4b170 2024-04-23T19:10:24.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.950+0000 7f3098e09640 1 -- 172.21.15.63:0/915944573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319c001470 con 0x5634acc4b170 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.950+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5634acd44730 con 0x5634acc4b170 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.950+0000 7f3098e09640 1 -- 172.21.15.63:0/915944573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c01a8d0 con 0x5634acc4b170 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.950+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634acd47390 con 0x5634acc4b170 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f3098e09640 1 -- 172.21.15.63:0/915944573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f319c015020 con 0x5634acc4b170 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f3098e09640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3094036eb0 0x7f3094039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f3098e09640 1 -- 172.21.15.63:0/915944573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f319c020030 con 0x5634acc4b170 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f31abbc2640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3094036eb0 0x7f3094039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acc4f560 0x5634acd4b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd4b6d0 con 0x5634acc4f560 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f31acbc4640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acc4f560 0x5634acd4b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.951+0000 7f31abbc2640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3094036eb0 0x7f3094039370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31a0054000 tx=0x7f31a004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.952+0000 7f31acbc4640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acc4f560 0x5634acd4b150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.952+0000 7f31acbc4640 1 -- 172.21.15.63:0/915944573 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f31a400a4a0 con 0x5634acc4f560 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.952+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd48720 con 0x5634acc4f560 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.952+0000 7f31acbc4640 1 -- 172.21.15.63:0/915944573 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f31a400a4a0 con 0x5634acc4f560 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.952+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd4d4d0 con 0x5634acc4f560 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.953+0000 7f31acbc4640 1 -- 172.21.15.63:0/915944573 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f31a400a4a0 con 0x5634acc4f560 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.953+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd4c810 con 0x5634acc4f560 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.953+0000 7f31acbc4640 1 -- 172.21.15.63:0/915944573 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f31a400a4a0 con 0x5634acc4f560 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.953+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5634acd4e880 0x5634acd50c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.953+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd511f0 con 0x5634acd4e880 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.953+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5634acd4e880 0x5634acd50c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.954+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5634acd4e880 0x5634acd50c70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.954+0000 7f31ac3c3640 1 -- 172.21.15.63:0/915944573 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f319c018950 con 0x5634acd4e880 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.954+0000 7f31b071f800 1 Processor -- start 2024-04-23T19:10:24.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.954+0000 7f31b071f800 1 -- start start 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.955+0000 7f31b071f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acd5f5a0 0x5634acd5f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.955+0000 7f31b071f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634acb3ce30 con 0x5634acd5f5a0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.955+0000 7f31acbc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acd5f5a0 0x5634acd5f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.955+0000 7f31acbc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acd5f5a0 0x5634acd5f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47804/0 (socket says 172.21.15.63:47804) 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.955+0000 7f31acbc4640 1 -- 172.21.15.63:0/3518445611 learned_addr learned my addr 172.21.15.63:0/3518445611 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.955+0000 7f31acbc4640 1 -- 172.21.15.63:0/3518445611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634acd5ff00 con 0x5634acd5f5a0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.955+0000 7f31acbc4640 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acd5f5a0 0x5634acd5f9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5634acd45310 tx=0x7f31a40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.956+0000 7f3091ffb640 1 -- 172.21.15.63:0/3518445611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a4013440 con 0x5634acd5f5a0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.956+0000 7f3091ffb640 1 -- 172.21.15.63:0/3518445611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a40135e0 con 0x5634acd5f5a0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.956+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5634acd60970 con 0x5634acd5f5a0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.956+0000 7f3091ffb640 1 -- 172.21.15.63:0/3518445611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a40138c0 con 0x5634acd5f5a0 2024-04-23T19:10:24.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.956+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634acd60e70 con 0x5634acd5f5a0 2024-04-23T19:10:24.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f3091ffb640 1 -- 172.21.15.63:0/3518445611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f31a4011040 con 0x5634acd5f5a0 2024-04-23T19:10:24.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f3091ffb640 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3084036e60 0x7f3084039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f3091ffb640 1 -- 172.21.15.63:0/3518445611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f31a401d070 con 0x5634acd5f5a0 2024-04-23T19:10:24.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3084036e60 0x7f3084039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3084036e60 0x7f3084039320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f319c018c60 tx=0x7f319c02df20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f31b071f800 1 Processor -- start 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f31b071f800 1 -- start start 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.957+0000 7f31b071f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acbc0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f31b071f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634acd3cca0 con 0x5634acbc0730 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f31acbc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acbc0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f31acbc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acbc0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47810/0 (socket says 172.21.15.63:47810) 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f31acbc4640 1 -- 172.21.15.63:0/507060169 learned_addr learned my addr 172.21.15.63:0/507060169 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f31acbc4640 1 -- 172.21.15.63:0/507060169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634acbbef30 con 0x5634acbc0730 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f31acbc4640 1 --2- 172.21.15.63:0/507060169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acbc0b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31a404ccb0 tx=0x7f31a4012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6d4ac330dd27f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f308b7fe640 1 -- 172.21.15.63:0/507060169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a4012df0 con 0x5634acbc0730 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f308b7fe640 1 -- 172.21.15.63:0/507060169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a4001310 con 0x5634acbc0730 2024-04-23T19:10:24.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.958+0000 7f308b7fe640 1 -- 172.21.15.63:0/507060169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a4001630 con 0x5634acbc0730 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 -- 172.21.15.63:0/507060169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 msgr2=0x5634acbc0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 --2- 172.21.15.63:0/507060169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acbc0b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31a404ccb0 tx=0x7f31a4012910 comp rx=0 tx=0).stop 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 -- 172.21.15.63:0/507060169 shutdown_connections 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 --2- 172.21.15.63:0/507060169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acbc0b50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 -- 172.21.15.63:0/507060169 >> 172.21.15.63:0/507060169 conn(0x5634acd68bd0 msgr2=0x5634acd66c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 -- 172.21.15.63:0/507060169 shutdown_connections 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 -- 172.21.15.63:0/507060169 wait complete. 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 Processor -- start 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.959+0000 7f31b071f800 1 -- start start 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.960+0000 7f31b071f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acd76040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.960+0000 7f31b071f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634acbbf400 con 0x5634acbc0730 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.960+0000 7f31acbc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acd76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.960+0000 7f31acbc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acd76040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47820/0 (socket says 172.21.15.63:47820) 2024-04-23T19:10:24.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.960+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 learned_addr learned my addr 172.21.15.63:0/1745978077 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.960+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634acd76580 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.960+0000 7f31acbc4640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acd76040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31a4015ac0 tx=0x7f31a404cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.961+0000 7f3089ffb640 1 -- 172.21.15.63:0/1745978077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a402ae30 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.961+0000 7f3089ffb640 1 -- 172.21.15.63:0/1745978077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a4021c60 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.961+0000 7f3089ffb640 1 -- 172.21.15.63:0/1745978077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a402a350 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.961+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5634acd76810 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.961+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634acd794c0 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.961+0000 7f3089ffb640 1 -- 172.21.15.63:0/1745978077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f31a4040040 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f3089ffb640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f307c036e60 0x7f307c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f3089ffb640 1 -- 172.21.15.63:0/1745978077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f31a4032070 con 0x5634acbc0730 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f307c036e60 0x7f307c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acd81cb0 0x5634acd84170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd848c0 con 0x5634acd81cb0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f31abbc2640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acd81cb0 0x5634acd84170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.962+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f307c036e60 0x7f307c039320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f319c044800 tx=0x7f319c034000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31abbc2640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acd81cb0 0x5634acd84170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 20 realm 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd84c90 con 0x5634acd81cb0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd85530 con 0x5634acd81cb0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.963+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd858d0 con 0x5634acd81cb0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31b071f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd85180 con 0x5634acd81cb0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31b071f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31b071f800 20 searching for the correct realm 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5634acd87d80 0x5634acd8a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31acbc4640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5634acd87d80 0x5634acd8a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.964+0000 7f31acbc4640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5634acd87d80 0x5634acd8a170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.965+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.965+0000 7f31acbc4640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f31a40123a0 0x7f31a40468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.965+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f31a40123a0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.965+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f31a40123a0 0x7f31a40468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.965+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f31a40123a0 0x7f31a40468c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.966+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.966+0000 7f31ac3c3640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f319c037070 0x7f319c04c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.966+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.966+0000 7f31abbc2640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f319c037070 0x7f319c04c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.967+0000 7f31abbc2640 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f319c037070 0x7f319c04c3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.967+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.967+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.967+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.967+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.967+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.967+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.968+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.968+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.968+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.968+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f31a40123a0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.969+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.970+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.970+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f31a40123a0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.970+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.970+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.970+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.970+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.971+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.971+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.971+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.971+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.971+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.971+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.972+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.973+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.973+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f31a40123a0 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.973+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.973+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.973+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v23'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.973+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.974+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.974+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.974+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.974+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.974+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.974+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f31a40123a0 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.974+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.975+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd87d80 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.975+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.975+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f319c037070 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.975+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.975+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x5634acd81cb0 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.975+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.975+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8a840 con 0x7f31a40123a0 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:24.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd86820 con 0x5634acd81cb0 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd870c0 con 0x5634acd81cb0 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:24.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.976+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd86460 con 0x7f319c037070 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 20 zone default found 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 4 Realm: () 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 20 zonegroup default 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8ad10 con 0x7f319c037070 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.977+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cba0 con 0x5634acd87d80 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31b071f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31b071f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cf40 con 0x5634acd87d80 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31b071f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31b071f800 20 started sync module instance, tier type = 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31b071f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.978+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8bfe0 con 0x7f31a40123a0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.979+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.979+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94784238175184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8c7f0 con 0x7f31a40123a0 2024-04-23T19:10:24.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.979+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8d900 con 0x5634acd81cb0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.980+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94784238175184] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.981+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.981+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94784238178112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8dd20 con 0x5634acd81cb0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.981+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8e740 con 0x5634acd87d80 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.983+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.983+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94784238182960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8e740 con 0x5634acd87d80 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.983+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8f740 con 0x7f31a40123a0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.983+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94784238178112] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.984+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94784238182960] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.986+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.986+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94784238187056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8eb30 con 0x7f31a40123a0 2024-04-23T19:10:24.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.986+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd90150 con 0x5634acd81cb0 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.989+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.989+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94784238190208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd90870 con 0x5634acd81cb0 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.989+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd91390 con 0x7f319c037070 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.992+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94784238187056] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.992+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.992+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94784238194272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd91350 con 0x7f319c037070 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.992+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd91da0 con 0x7f319c037070 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.994+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94784238190208] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:24.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.995+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94784238194272] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.998+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.998+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94784238179168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd918f0 con 0x7f319c037070 2024-04-23T19:10:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:24.998+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd92640 con 0x7f319c037070 2024-04-23T19:10:25.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94784238179168] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94784238199184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd918f0 con 0x7f319c037070 2024-04-23T19:10:25.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 20 add_watcher() i=0 2024-04-23T19:10:25.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 20 add_watcher() i=1 2024-04-23T19:10:25.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 20 add_watcher() i=2 2024-04-23T19:10:25.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 20 add_watcher() i=3 2024-04-23T19:10:25.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 20 add_watcher() i=4 2024-04-23T19:10:25.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 20 add_watcher() i=5 2024-04-23T19:10:25.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.000+0000 7f31b071f800 20 add_watcher() i=6 2024-04-23T19:10:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.001+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94784238199184] v30'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.001+0000 7f31b071f800 20 add_watcher() i=7 2024-04-23T19:10:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.001+0000 7f31b071f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.001+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5634acd90370 con 0x5634acd81cb0 2024-04-23T19:10:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.002+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.002+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94784238200240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd90370 con 0x5634acd81cb0 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94784238200240] v30'24 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f30627e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f31b071f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f31b071f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f31b071f800 5 note: GC not initialized 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f30617e2640 20 reqs_thread_entry: start 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f31b071f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.003+0000 7f31b071f800 20 Filter name: none 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.004+0000 7f2fd2ffd640 20 reqs_thread_entry: start 2024-04-23T19:10:25.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.004+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acdc8220 con 0x5634acd87d80 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.004+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.004+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cba0 con 0x5634acd87d80 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.004+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.004+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cba0 con 0x5634acd81cb0 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd870c0 con 0x5634acd81cb0 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd870c0 con 0x7f31a40123a0 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5634acd870c0 con 0x5634acd87d80 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.005+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd81cb0 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.006+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.006+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd87d80 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.006+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.006+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f31a40123a0 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.006+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.006+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd87d80 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.007+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.007+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd81cb0 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.007+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.007+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.007+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.007+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.008+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(74 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.008+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd87d80 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.008+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(75 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.008+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.008+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(76 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.008+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(77 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(78 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd87d80 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(79 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f31a40123a0 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(80 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.009+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd81cb0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(81 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd81cb0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(82 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd87d80 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(83 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f31a40123a0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(84 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.010+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.011+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(85 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.011+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd81cb0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.011+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(86 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.011+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.012+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(87 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.012+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd87d80 2024-04-23T19:10:25.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.012+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(88 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.012+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f31a40123a0 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.012+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(89 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.012+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f319c037070 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.013+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(90 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.013+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd81cb0 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.013+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(91 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.013+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x5634acd87d80 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.013+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(92 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.013+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f31a406b040 con 0x7f31a40123a0 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.013+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(93 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.014+0000 7f30627e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.014+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94784238200240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cba0 con 0x5634acd81cb0 2024-04-23T19:10:25.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.015+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94784238200240] v30'25 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.015+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94784238175184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd870c0 con 0x7f31a40123a0 2024-04-23T19:10:25.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.016+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94784238175184] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.016+0000 7f31b071f800 20 remove_watcher() i=0 2024-04-23T19:10:25.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.016+0000 7f31b071f800 2 removed watcher, disabling cache 2024-04-23T19:10:25.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.016+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94784238178112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8c3d0 con 0x5634acd81cb0 2024-04-23T19:10:25.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.018+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94784238178112] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.018+0000 7f31b071f800 20 remove_watcher() i=1 2024-04-23T19:10:25.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.018+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94784238182960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8c3d0 con 0x5634acd87d80 2024-04-23T19:10:25.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.019+0000 7f31acbc4640 1 -- 172.21.15.63:0/1745978077 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94784238182960] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a4029070 con 0x5634acd87d80 2024-04-23T19:10:25.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.019+0000 7f31b071f800 20 remove_watcher() i=2 2024-04-23T19:10:25.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.019+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94784238187056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cf40 con 0x7f31a40123a0 2024-04-23T19:10:25.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.020+0000 7f31ac3c3640 1 -- 172.21.15.63:0/1745978077 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94784238187056] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f319c018950 con 0x7f31a40123a0 2024-04-23T19:10:25.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.020+0000 7f31b071f800 20 remove_watcher() i=3 2024-04-23T19:10:25.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.020+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94784238190208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cf40 con 0x5634acd81cb0 2024-04-23T19:10:25.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.021+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94784238190208] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x5634acd81cb0 2024-04-23T19:10:25.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.021+0000 7f31b071f800 20 remove_watcher() i=4 2024-04-23T19:10:25.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.021+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94784238194272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8fe80 con 0x7f319c037070 2024-04-23T19:10:25.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.023+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94784238194272] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.023+0000 7f31b071f800 20 remove_watcher() i=5 2024-04-23T19:10:25.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.023+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94784238179168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acd8cf40 con 0x7f319c037070 2024-04-23T19:10:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.025+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94784238179168] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.025+0000 7f31b071f800 20 remove_watcher() i=6 2024-04-23T19:10:25.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.025+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94784238199184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5634acdc81e0 con 0x7f319c037070 2024-04-23T19:10:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94784238199184] v30'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f31a005a040 con 0x7f319c037070 2024-04-23T19:10:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 20 remove_watcher() i=7 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f319c037070 msgr2=0x7f319c04c3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f319c037070 0x7f319c04c3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acd81cb0 msgr2=0x5634acd84170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acd81cb0 0x5634acd84170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f31a40123a0 msgr2=0x7f31a40468c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f31a40123a0 0x7f31a40468c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5634acd87d80 msgr2=0x5634acd8a170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5634acd87d80 0x5634acd8a170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f307c036e60 msgr2=0x7f307c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f307c036e60 0x7f307c039320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f319c044800 tx=0x7f319c034000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 msgr2=0x5634acd76040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 --2- 172.21.15.63:0/1745978077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acbc0730 0x5634acd76040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31a4015ac0 tx=0x7f31a404cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 reap_dead start 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31abbc2640 1 -- 172.21.15.63:0/1745978077 reap_dead start 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 shutdown_connections 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 >> 172.21.15.63:0/1745978077 conn(0x5634acd68bd0 msgr2=0x5634acd66280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 shutdown_connections 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.027+0000 7f31b071f800 1 -- 172.21.15.63:0/1745978077 wait complete. 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3084036e60 msgr2=0x7f3084039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3084036e60 0x7f3084039320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f319c018c60 tx=0x7f319c02df20 comp rx=0 tx=0).stop 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acd5f5a0 msgr2=0x5634acd5f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acd5f5a0 0x5634acd5f9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5634acd45310 tx=0x7f31a40178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 shutdown_connections 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3084036e60 0x7f3084039320 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/3518445611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acd5f5a0 0x5634acd5f9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 >> 172.21.15.63:0/3518445611 conn(0x5634acd53640 msgr2=0x5634acd53a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 shutdown_connections 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/3518445611 wait complete. 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5634acd4e880 msgr2=0x5634acd50c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5634acd4e880 0x5634acd50c70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acc4f560 msgr2=0x5634acd4b150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acc4f560 0x5634acd4b150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3094036eb0 msgr2=0x7f3094039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3094036eb0 0x7f3094039370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31a0054000 tx=0x7f31a004e000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 msgr2=0x5634acd43f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd43f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f319c00b7c0 tx=0x7f319c000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 shutdown_connections 2024-04-23T19:10:25.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5634acd4e880 0x5634acd50c70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5634acc4f560 0x5634acd4b150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3094036eb0 0x7f3094039370 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 --2- 172.21.15.63:0/915944573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634acc4b170 0x5634acd43f60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 >> 172.21.15.63:0/915944573 conn(0x5634acc467d0 msgr2=0x5634acd36490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 shutdown_connections 2024-04-23T19:10:25.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.028+0000 7f31b071f800 1 -- 172.21.15.63:0/915944573 wait complete. 2024-04-23T19:10:25.052 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-04-23T19:10:25.052 DEBUG:tasks.util.rgw: json result: [] 2024-04-23T19:10:25.052 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-04-23T19:10:25.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-04-23T19:10:25.052 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-04-23T19:10:25.097 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:25.098 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.098+0000 7f047e813800 1 Processor -- start 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.098+0000 7f047e813800 1 -- start start 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.098+0000 7f047e813800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5612b9941c30 0x5612b9944020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.098+0000 7f047e813800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5612b97f37f0 con 0x5612b9944560 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.098+0000 7f047e813800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5612b97e2e20 con 0x5612b9941c30 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.099+0000 7f0479e21640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5612b9941c30 0x5612b9944020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.099+0000 7f0479620640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5612b9944560 0x5612b9946950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52602/0 (socket says 172.21.15.63:52602) 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.099+0000 7f0479620640 1 -- 172.21.15.63:0/1560881550 learned_addr learned my addr 172.21.15.63:0/1560881550 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.099+0000 7f0479e21640 1 -- 172.21.15.63:0/1560881550 >> v1:172.21.15.16:6789/0 conn(0x5612b9944560 legacy=0x5612b9946950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.099+0000 7f0479e21640 1 -- 172.21.15.63:0/1560881550 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9801b50 con 0x5612b9941c30 2024-04-23T19:10:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.101+0000 7f0479e21640 1 --2- 172.21.15.63:0/1560881550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9944020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f046c00a470 tx=0x7f046c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581da7382525a0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.101+0000 7f0478e1f640 1 -- 172.21.15.63:0/1560881550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c057070 con 0x5612b9941c30 2024-04-23T19:10:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.101+0000 7f0478e1f640 1 -- 172.21.15.63:0/1560881550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c052570 con 0x5612b9941c30 2024-04-23T19:10:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.101+0000 7f0478e1f640 1 -- 172.21.15.63:0/1560881550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c052870 con 0x5612b9941c30 2024-04-23T19:10:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.101+0000 7f047e813800 1 -- 172.21.15.63:0/1560881550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 msgr2=0x5612b9944020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.101+0000 7f047e813800 1 --2- 172.21.15.63:0/1560881550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9944020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f046c00a470 tx=0x7f046c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.102+0000 7f047e813800 1 -- 172.21.15.63:0/1560881550 shutdown_connections 2024-04-23T19:10:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.102+0000 7f047e813800 1 --2- 172.21.15.63:0/1560881550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9944020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.102+0000 7f047e813800 1 -- 172.21.15.63:0/1560881550 >> 172.21.15.63:0/1560881550 conn(0x5612b993b600 msgr2=0x5612b993da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.102+0000 7f047e813800 1 -- 172.21.15.63:0/1560881550 shutdown_connections 2024-04-23T19:10:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.102+0000 7f047e813800 1 -- 172.21.15.63:0/1560881550 wait complete. 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.114+0000 7f047e813800 1 Processor -- start 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.114+0000 7f047e813800 1 -- start start 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.114+0000 7f047e813800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a32300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.114+0000 7f047e813800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5612b988c6f0 con 0x5612b9941c30 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0479e21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a32300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0479e21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a32300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47840/0 (socket says 172.21.15.63:47840) 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0479e21640 1 -- 172.21.15.63:0/2543424503 learned_addr learned my addr 172.21.15.63:0/2543424503 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0479e21640 1 -- 172.21.15.63:0/2543424503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9a328d0 con 0x5612b9941c30 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0479e21640 1 --2- 172.21.15.63:0/2543424503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a32300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f046c01ced0 tx=0x7f046c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3cab41752fda4d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0367e07640 1 -- 172.21.15.63:0/2543424503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c057040 con 0x5612b9941c30 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0367e07640 1 -- 172.21.15.63:0/2543424503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c0529a0 con 0x5612b9941c30 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.115+0000 7f0367e07640 1 -- 172.21.15.63:0/2543424503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c052cc0 con 0x5612b9941c30 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 -- 172.21.15.63:0/2543424503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 msgr2=0x5612b9a32300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 --2- 172.21.15.63:0/2543424503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a32300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f046c01ced0 tx=0x7f046c04c000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 -- 172.21.15.63:0/2543424503 shutdown_connections 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 --2- 172.21.15.63:0/2543424503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a32300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 -- 172.21.15.63:0/2543424503 >> 172.21.15.63:0/2543424503 conn(0x5612b993b600 msgr2=0x5612b9a2dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 -- 172.21.15.63:0/2543424503 shutdown_connections 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 -- 172.21.15.63:0/2543424503 wait complete. 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.116+0000 7f047e813800 1 Processor -- start 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f047e813800 1 -- start start 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f047e813800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a39d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f047e813800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5612b994a380 con 0x5612b9941c30 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f0479e21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a39d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f0479e21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a39d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47844/0 (socket says 172.21.15.63:47844) 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f0479e21640 1 -- 172.21.15.63:0/2229562511 learned_addr learned my addr 172.21.15.63:0/2229562511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f0479e21640 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9a3a2c0 con 0x5612b9941c30 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f0479e21640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a39d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f046c000c00 tx=0x7f046c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.117+0000 7f0366604640 1 -- 172.21.15.63:0/2229562511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c073070 con 0x5612b9941c30 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.118+0000 7f0366604640 1 -- 172.21.15.63:0/2229562511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c05d780 con 0x5612b9941c30 2024-04-23T19:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.118+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5612b9a3a550 con 0x5612b9941c30 2024-04-23T19:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.118+0000 7f0366604640 1 -- 172.21.15.63:0/2229562511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c057070 con 0x5612b9941c30 2024-04-23T19:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.118+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b9a3d200 con 0x5612b9941c30 2024-04-23T19:10:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.118+0000 7f0366604640 1 -- 172.21.15.63:0/2229562511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f046c076070 con 0x5612b9941c30 2024-04-23T19:10:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f0366604640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036e60 0x7f0360039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f0366604640 1 -- 172.21.15.63:0/2229562511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f046c08e5e0 con 0x5612b9941c30 2024-04-23T19:10:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f0479620640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036e60 0x7f0360039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9944560 0x5612b9a40fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a415e0 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f047a622640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9944560 0x5612b9a40fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f0479620640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036e60 0x7f0360039320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0470000f60 tx=0x7f04700012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f047a622640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9944560 0x5612b9a40fc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.119+0000 7f047a622640 1 -- 172.21.15.63:0/2229562511 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f047400a4a0 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a3e570 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047a622640 1 -- 172.21.15.63:0/2229562511 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f047400a4a0 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a433e0 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047a622640 1 -- 172.21.15.63:0/2229562511 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f047400a4a0 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a42720 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047a622640 1 -- 172.21.15.63:0/2229562511 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f047400a4a0 con 0x5612b9944560 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5612b9a44790 0x5612b9a46b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.120+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a470c0 con 0x5612b9a44790 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.121+0000 7f0479e21640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5612b9a44790 0x5612b9a46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.121+0000 7f0479e21640 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5612b9a44790 0x5612b9a46b80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.121+0000 7f0479e21640 1 -- 172.21.15.63:0/2229562511 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f046c075030 con 0x5612b9a44790 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047e813800 1 Processor -- start 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047e813800 1 -- start start 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047e813800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9a55530 0x5612b9a55950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047e813800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5612b9832e30 con 0x5612b9a55530 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047a622640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9a55530 0x5612b9a55950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047a622640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9a55530 0x5612b9a55950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47860/0 (socket says 172.21.15.63:47860) 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047a622640 1 -- 172.21.15.63:0/4127611946 learned_addr learned my addr 172.21.15.63:0/4127611946 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.122+0000 7f047a622640 1 -- 172.21.15.63:0/4127611946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9a55e90 con 0x5612b9a55530 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.123+0000 7f047a622640 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9a55530 0x5612b9a55950 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5612b9a419c0 tx=0x7f04740178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.123+0000 7f035f7fe640 1 -- 172.21.15.63:0/4127611946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474013440 con 0x5612b9a55530 2024-04-23T19:10:25.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.123+0000 7f035f7fe640 1 -- 172.21.15.63:0/4127611946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04740135e0 con 0x5612b9a55530 2024-04-23T19:10:25.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.123+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5612b9a56900 con 0x5612b9a55530 2024-04-23T19:10:25.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.123+0000 7f035f7fe640 1 -- 172.21.15.63:0/4127611946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04740138c0 con 0x5612b9a55530 2024-04-23T19:10:25.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.123+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b9a56e00 con 0x5612b9a55530 2024-04-23T19:10:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f035f7fe640 1 -- 172.21.15.63:0/4127611946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0474011040 con 0x5612b9a55530 2024-04-23T19:10:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f035f7fe640 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f035f7fe640 1 -- 172.21.15.63:0/4127611946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f047401d070 con 0x5612b9a55530 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f0479e21640 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f047e813800 1 Processor -- start 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f047e813800 1 -- start start 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f0479e21640 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x5612b9a42ac0 tx=0x7f046c077eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f047e813800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b99416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.124+0000 7f047e813800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5612b9a32b90 con 0x5612b99412a0 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f047a622640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b99416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f047a622640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b99416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47870/0 (socket says 172.21.15.63:47870) 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f047a622640 1 -- 172.21.15.63:0/185933700 learned_addr learned my addr 172.21.15.63:0/185933700 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f047a622640 1 -- 172.21.15.63:0/185933700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b993fab0 con 0x5612b99412a0 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f047a622640 1 --2- 172.21.15.63:0/185933700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b99416c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04740013c0 tx=0x7f047404cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8f4c700b8c35f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f035d7fa640 1 -- 172.21.15.63:0/185933700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474042030 con 0x5612b99412a0 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f035d7fa640 1 -- 172.21.15.63:0/185933700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0474015560 con 0x5612b99412a0 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.125+0000 7f035d7fa640 1 -- 172.21.15.63:0/185933700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f047402ad80 con 0x5612b99412a0 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 -- 172.21.15.63:0/185933700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 msgr2=0x5612b99416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 --2- 172.21.15.63:0/185933700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b99416c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04740013c0 tx=0x7f047404cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 -- 172.21.15.63:0/185933700 shutdown_connections 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 --2- 172.21.15.63:0/185933700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b99416c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 -- 172.21.15.63:0/185933700 >> 172.21.15.63:0/185933700 conn(0x5612b9a5ebb0 msgr2=0x5612b9a5cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 -- 172.21.15.63:0/185933700 shutdown_connections 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 -- 172.21.15.63:0/185933700 wait complete. 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 Processor -- start 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 -- start start 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b9a6c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.126+0000 7f047e813800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5612b993ffb0 con 0x5612b99412a0 2024-04-23T19:10:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f047a622640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b9a6c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f047a622640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b9a6c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47876/0 (socket says 172.21.15.63:47876) 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 learned_addr learned my addr 172.21.15.63:0/1111082365 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9a6c5a0 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f047a622640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b9a6c060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f047404cbd0 tx=0x7f047404cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f034b7fe640 1 -- 172.21.15.63:0/1111082365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474039070 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f034b7fe640 1 -- 172.21.15.63:0/1111082365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0474013e20 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f034b7fe640 1 -- 172.21.15.63:0/1111082365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474012720 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5612b9a6c830 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.127+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b9a6f4e0 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.128+0000 7f034b7fe640 1 -- 172.21.15.63:0/1111082365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0474045070 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.128+0000 7f034b7fe640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.128+0000 7f034b7fe640 1 -- 172.21.15.63:0/1111082365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f0474043070 con 0x5612b99412a0 2024-04-23T19:10:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.128+0000 7f0479e21640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.129+0000 7f0479e21640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5612b9a474c0 tx=0x7f046c083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.129+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.129+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9a77ca0 0x5612b9a7a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.129+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a7a8b0 con 0x5612b9a77ca0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.129+0000 7f0479620640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9a77ca0 0x5612b9a7a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.129+0000 7f0479620640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9a77ca0 0x5612b9a7a160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0470009050 con 0x5612b9a77ca0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 realm 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a770f0 con 0x5612b9a77ca0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0470009050 con 0x5612b9a77ca0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a7b3a0 con 0x5612b9a77ca0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0470009050 con 0x5612b9a77ca0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a7bc70 con 0x5612b9a77ca0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0470009050 con 0x5612b9a77ca0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.130+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a7c010 con 0x5612b9a77ca0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0470009050 con 0x5612b9a77ca0 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047e813800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047e813800 20 searching for the correct realm 2024-04-23T19:10:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5612b9a7def0 0x5612b9a802e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x5612b9a7def0 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047a622640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5612b9a7def0 0x5612b9a802e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047a622640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5612b9a7def0 0x5612b9a802e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047a622640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f047402a830 0x7f047404ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.131+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x7f047402a830 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.132+0000 7f0479e21640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f047402a830 0x7f047404ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.132+0000 7f0479e21640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f047402a830 0x7f047404ea40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.132+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.132+0000 7f0479e21640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c078060 0x7f046c08e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.132+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x7f046c078060 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.132+0000 7f0479620640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c078060 0x7f046c08e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.133+0000 7f0479620640 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c078060 0x7f046c08e9c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.133+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0470009050 con 0x7f046c078060 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.133+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x5612b9a77ca0 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.133+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0470009050 con 0x5612b9a77ca0 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.133+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x5612b9a77ca0 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.133+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.133+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x7f046c078060 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x7f046c078060 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x7f047402a830 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x5612b9a7def0 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.134+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x5612b9a77ca0 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.135+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.135+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x7f046c078060 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.135+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.135+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x5612b9a77ca0 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.135+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.135+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80990 con 0x7f047402a830 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a7def0 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a7def0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f046c078060 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.136+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a7def0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a7def0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a77ca0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a77ca0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.137+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f046c078060 2024-04-23T19:10:25.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a7def0 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f047402a830 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a77ca0 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v23'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.138+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f046c078060 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.139+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.139+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a7def0 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.139+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.139+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f046c078060 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.140+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.140+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f047402a830 2024-04-23T19:10:25.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.140+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.140+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a7def0 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.140+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.140+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f046c078060 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x5612b9a77ca0 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f046c0a9060 con 0x7f047402a830 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a7c990 con 0x5612b9a77ca0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.141+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a7d230 con 0x5612b9a77ca0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a80e10 con 0x7f046c078060 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 20 zone default found 2024-04-23T19:10:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 4 Realm: () 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 20 zonegroup default 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.142+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a812c0 con 0x7f046c078060 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82ca0 con 0x5612b9a7def0 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a830e0 con 0x5612b9a7def0 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 20 started sync module instance, tier type = 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.143+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82130 con 0x7f047402a830 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.147+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.147+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94638424204576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a81d90 con 0x7f047402a830 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.147+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a84010 con 0x5612b9a77ca0 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.148+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94638424204576] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.148+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.148+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94638424210272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a84430 con 0x5612b9a77ca0 2024-04-23T19:10:25.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.148+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a84df0 con 0x5612b9a7def0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.149+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94638424210272] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.149+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.149+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94638424211328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a843a0 con 0x5612b9a7def0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.150+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a85cb0 con 0x7f047402a830 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.151+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.151+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94638424217600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a86500 con 0x7f047402a830 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.151+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a86fb0 con 0x5612b9a77ca0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.151+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94638424211328] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.152+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.152+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94638424216032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a873d0 con 0x5612b9a77ca0 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.152+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a87df0 con 0x7f046c078060 2024-04-23T19:10:25.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.152+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94638424217600] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.153+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94638424216032] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.154+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.154+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94638424222400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a87db0 con 0x7f046c078060 2024-04-23T19:10:25.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.154+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a88690 con 0x7f046c078060 2024-04-23T19:10:25.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.157+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(56 notify.6 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.157+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94638424228320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a87db0 con 0x7f046c078060 2024-04-23T19:10:25.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.157+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a891c0 con 0x7f046c078060 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.158+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v30'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.158+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94638424231120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a89670 con 0x7f046c078060 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.158+0000 7f047e813800 20 add_watcher() i=0 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.158+0000 7f047e813800 20 add_watcher() i=1 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.158+0000 7f047e813800 20 add_watcher() i=2 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.158+0000 7f047e813800 20 add_watcher() i=3 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.158+0000 7f047e813800 20 add_watcher() i=4 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.159+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(55 notify.5 [watch watch cookie 94638424222400] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.159+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94638424231120] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.159+0000 7f047e813800 20 add_watcher() i=5 2024-04-23T19:10:25.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.162+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(57 notify.6 [watch watch cookie 94638424228320] v30'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.162+0000 7f047e813800 20 add_watcher() i=6 2024-04-23T19:10:25.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.162+0000 7f047e813800 20 add_watcher() i=7 2024-04-23T19:10:25.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.162+0000 7f047e813800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:25.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.162+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a891c0 con 0x5612b9a77ca0 2024-04-23T19:10:25.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.162+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.163+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94638424226016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a891c0 con 0x5612b9a77ca0 2024-04-23T19:10:25.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.164+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94638424226016] v30'26 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.164+0000 7f032b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.164+0000 7f047e813800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:25.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.164+0000 7f047e813800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:25.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.164+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a949b0 con 0x5612b9a77ca0 2024-04-23T19:10:25.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.165+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.165+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95000 con 0x7f046c078060 2024-04-23T19:10:25.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.166+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'10 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.166+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95440 con 0x5612b9a7def0 2024-04-23T19:10:25.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.167+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.167+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95360 con 0x5612b9a77ca0 2024-04-23T19:10:25.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.168+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'27 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.168+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95a30 con 0x7f047402a830 2024-04-23T19:10:25.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.169+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'20 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.169+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95a30 con 0x5612b9a7def0 2024-04-23T19:10:25.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.170+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'13 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.170+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95ee0 con 0x5612b9a77ca0 2024-04-23T19:10:25.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.172+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.172+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95ee0 con 0x7f046c078060 2024-04-23T19:10:25.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.173+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'13 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.173+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95e60 con 0x7f047402a830 2024-04-23T19:10:25.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.175+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.175+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95e60 con 0x5612b9a7def0 2024-04-23T19:10:25.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.178+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'16 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.178+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a95e60 con 0x7f047402a830 2024-04-23T19:10:25.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.181+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'21 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.181+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92840 con 0x7f046c078060 2024-04-23T19:10:25.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.182+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'12 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.182+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92840 con 0x7f047402a830 2024-04-23T19:10:25.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.183+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'10 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.183+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92840 con 0x7f046c078060 2024-04-23T19:10:25.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.186+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'19 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.186+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a77ca0 2024-04-23T19:10:25.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.187+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'28 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.187+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.188+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'14 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.188+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x7f047402a830 2024-04-23T19:10:25.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.189+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'22 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.189+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.191+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.191+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a77ca0 2024-04-23T19:10:25.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.192+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.192+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a77ca0 2024-04-23T19:10:25.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.193+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'29 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.193+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.194+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.194+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a926a0 con 0x7f047402a830 2024-04-23T19:10:25.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.195+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.195+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a926a0 con 0x5612b9a7def0 2024-04-23T19:10:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.196+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.196+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.197+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.197+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.198+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'6 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.198+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.199+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'14 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.199+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x7f046c078060 2024-04-23T19:10:25.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.201+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.201+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.202+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.202+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x7f046c078060 2024-04-23T19:10:25.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.205+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'14 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.205+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x5612b9a7def0 2024-04-23T19:10:25.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.206+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'16 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.206+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x7f046c078060 2024-04-23T19:10:25.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.207+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'20 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.207+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a92670 con 0x7f047402a830 2024-04-23T19:10:25.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.208+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.208+0000 7f032a7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:25.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.208+0000 7f047e813800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:25.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f047e813800 20 Filter name: none 2024-04-23T19:10:25.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f029bfff640 20 reqs_thread_entry: start 2024-04-23T19:10:25.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f047e813800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f047e813800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5612b9ac16b0 con 0x7f047402a830 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(94 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f047e813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f047e813800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.209+0000 7f047e813800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:could not remove user: unable to remove user, user does not exist 2024-04-23T19:10:25.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.210+0000 7f032b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.210+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94638424226016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a91c30 con 0x5612b9a77ca0 2024-04-23T19:10:25.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.211+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94638424226016] v30'30 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.211+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94638424204576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a878e0 con 0x7f047402a830 2024-04-23T19:10:25.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.212+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94638424204576] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.212+0000 7f047e813800 20 remove_watcher() i=0 2024-04-23T19:10:25.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.212+0000 7f047e813800 2 removed watcher, disabling cache 2024-04-23T19:10:25.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.212+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94638424210272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82520 con 0x5612b9a77ca0 2024-04-23T19:10:25.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.213+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94638424210272] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.213+0000 7f047e813800 20 remove_watcher() i=1 2024-04-23T19:10:25.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.213+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94638424211328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82520 con 0x5612b9a7def0 2024-04-23T19:10:25.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.214+0000 7f047a622640 1 -- 172.21.15.63:0/1111082365 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94638424211328] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047403b030 con 0x5612b9a7def0 2024-04-23T19:10:25.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.214+0000 7f047e813800 20 remove_watcher() i=2 2024-04-23T19:10:25.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.214+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94638424217600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82520 con 0x7f047402a830 2024-04-23T19:10:25.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.215+0000 7f0479e21640 1 -- 172.21.15.63:0/1111082365 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94638424217600] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c089030 con 0x7f047402a830 2024-04-23T19:10:25.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.215+0000 7f047e813800 20 remove_watcher() i=3 2024-04-23T19:10:25.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.215+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94638424216032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82520 con 0x5612b9a77ca0 2024-04-23T19:10:25.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.218+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94638424216032] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x5612b9a77ca0 2024-04-23T19:10:25.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.218+0000 7f047e813800 20 remove_watcher() i=4 2024-04-23T19:10:25.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.218+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94638424222400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82520 con 0x7f046c078060 2024-04-23T19:10:25.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.220+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94638424222400] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.220+0000 7f047e813800 20 remove_watcher() i=5 2024-04-23T19:10:25.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.220+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94638424228320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a86ac0 con 0x7f046c078060 2024-04-23T19:10:25.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.221+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94638424228320] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.221+0000 7f047e813800 20 remove_watcher() i=6 2024-04-23T19:10:25.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.221+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94638424231120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5612b9a82520 con 0x7f046c078060 2024-04-23T19:10:25.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94638424231120] v30'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c078060 2024-04-23T19:10:25.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 20 remove_watcher() i=7 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c078060 msgr2=0x7f046c08e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c078060 0x7f046c08e9c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9a77ca0 msgr2=0x5612b9a7a160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9a77ca0 0x5612b9a7a160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f047402a830 msgr2=0x7f047404ea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f047402a830 0x7f047404ea40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5612b9a7def0 msgr2=0x5612b9a802e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5612b9a7def0 0x5612b9a802e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 msgr2=0x7f0344039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5612b9a474c0 tx=0x7f046c083000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 msgr2=0x5612b9a6c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f0479620640 1 -- 172.21.15.63:0/1111082365 reap_dead start 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b9a6c060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f047404cbd0 tx=0x7f047404cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 shutdown_connections 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/1111082365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b99412a0 0x5612b9a6c060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 >> 172.21.15.63:0/1111082365 conn(0x5612b9a5ebb0 msgr2=0x5612b9a5c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 shutdown_connections 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/1111082365 wait complete. 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 msgr2=0x7f0354039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x5612b9a42ac0 tx=0x7f046c077eb0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9a55530 msgr2=0x5612b9a55950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.222+0000 7f047e813800 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9a55530 0x5612b9a55950 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5612b9a419c0 tx=0x7f04740178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 shutdown_connections 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/4127611946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9a55530 0x5612b9a55950 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 >> 172.21.15.63:0/4127611946 conn(0x5612b9a495d0 msgr2=0x5612b9a499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 shutdown_connections 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/4127611946 wait complete. 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5612b9a44790 msgr2=0x5612b9a46b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5612b9a44790 0x5612b9a46b80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9944560 msgr2=0x5612b9a40fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9944560 0x5612b9a40fc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036e60 msgr2=0x7f0360039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036e60 0x7f0360039320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0470000f60 tx=0x7f04700012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 msgr2=0x5612b9a39d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a39d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f046c000c00 tx=0x7f046c003020 comp rx=0 tx=0).stop 2024-04-23T19:10:25.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 shutdown_connections 2024-04-23T19:10:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5612b9a44790 0x5612b9a46b80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5612b9944560 0x5612b9a40fc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036e60 0x7f0360039320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 --2- 172.21.15.63:0/2229562511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9941c30 0x5612b9a39d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 >> 172.21.15.63:0/2229562511 conn(0x5612b993b600 msgr2=0x5612b9a2c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 shutdown_connections 2024-04-23T19:10:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.223+0000 7f047e813800 1 -- 172.21.15.63:0/2229562511 wait complete. 2024-04-23T19:10:25.231 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-23T19:10:25.232 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-04-23T19:10:25.232 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-04-23T19:10:25.232 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-04-23T19:10:25.276 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:25.276 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.276+0000 7fbfbdc3f800 1 Processor -- start 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.276+0000 7fbfbdc3f800 1 -- start start 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.276+0000 7fbfbdc3f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557ec0a54620 0x557ec0a56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.276+0000 7fbfbdc3f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557ec09027f0 con 0x557ec0a4e400 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.276+0000 7fbfbdc3f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x557ec08f1e20 con 0x557ec0a54620 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.277+0000 7fbfbaf3d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557ec0a54620 0x557ec0a56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.277+0000 7fbfbb73e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x557ec0a4e400 0x557ec0a4e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52606/0 (socket says 172.21.15.63:52606) 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.277+0000 7fbfbb73e640 1 -- 172.21.15.63:0/3467584657 learned_addr learned my addr 172.21.15.63:0/3467584657 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:25.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.277+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/3467584657 >> v1:172.21.15.16:6789/0 conn(0x557ec0a4e400 legacy=0x557ec0a4e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.277+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/3467584657 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec0910b50 con 0x557ec0a54620 2024-04-23T19:10:25.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.279+0000 7fbfbaf3d640 1 --2- 172.21.15.63:0/3467584657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a54620 0x557ec0a56a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000a470 tx=0x7fbfb001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e614418ee11347 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.279+0000 7fbfba73c640 1 -- 172.21.15.63:0/3467584657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb0057070 con 0x557ec0a54620 2024-04-23T19:10:25.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.279+0000 7fbfba73c640 1 -- 172.21.15.63:0/3467584657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb0052570 con 0x557ec0a54620 2024-04-23T19:10:25.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.279+0000 7fbfba73c640 1 -- 172.21.15.63:0/3467584657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb0052870 con 0x557ec0a54620 2024-04-23T19:10:25.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.279+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3467584657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a54620 msgr2=0x557ec0a56a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.279+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/3467584657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a54620 0x557ec0a56a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000a470 tx=0x7fbfb001ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:25.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.280+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3467584657 shutdown_connections 2024-04-23T19:10:25.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.280+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/3467584657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a54620 0x557ec0a56a10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.280+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3467584657 >> 172.21.15.63:0/3467584657 conn(0x557ec0a49db0 msgr2=0x557ec0a4c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.280+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3467584657 shutdown_connections 2024-04-23T19:10:25.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.280+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3467584657 wait complete. 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.292+0000 7fbfbdc3f800 1 Processor -- start 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.292+0000 7fbfbdc3f800 1 -- start start 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.293+0000 7fbfbdc3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0b41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.293+0000 7fbfbdc3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ec099b6f0 con 0x557ec0a4e400 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.293+0000 7fbfbb73e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0b41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.293+0000 7fbfbb73e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0b41390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47902/0 (socket says 172.21.15.63:47902) 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.293+0000 7fbfbb73e640 1 -- 172.21.15.63:0/1507107312 learned_addr learned my addr 172.21.15.63:0/1507107312 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.293+0000 7fbfbb73e640 1 -- 172.21.15.63:0/1507107312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec0b41960 con 0x557ec0a4e400 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.293+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/1507107312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0b41390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbfac001de0 tx=0x7fbfac00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d5973b9421cfb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbea960a640 1 -- 172.21.15.63:0/1507107312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac012020 con 0x557ec0a4e400 2024-04-23T19:10:25.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbea960a640 1 -- 172.21.15.63:0/1507107312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfac00e5a0 con 0x557ec0a4e400 2024-04-23T19:10:25.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbea960a640 1 -- 172.21.15.63:0/1507107312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac00e8c0 con 0x557ec0a4e400 2024-04-23T19:10:25.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1507107312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 msgr2=0x557ec0b41390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1507107312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0b41390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbfac001de0 tx=0x7fbfac00b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1507107312 shutdown_connections 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1507107312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0b41390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1507107312 >> 172.21.15.63:0/1507107312 conn(0x557ec0a49db0 msgr2=0x557ec0b3ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1507107312 shutdown_connections 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.294+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1507107312 wait complete. 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbdc3f800 1 Processor -- start 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbdc3f800 1 -- start start 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbdc3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0a50d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbdc3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ec0a572b0 con 0x557ec0a4e400 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbb73e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0a50d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbb73e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0a50d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47914/0 (socket says 172.21.15.63:47914) 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbb73e640 1 -- 172.21.15.63:0/1047490904 learned_addr learned my addr 172.21.15.63:0/1047490904 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.295+0000 7fbfbb73e640 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec0a4f420 con 0x557ec0a4e400 2024-04-23T19:10:25.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.296+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0a50d30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c010 tx=0x7fbfac000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.296+0000 7fbea7e07640 1 -- 172.21.15.63:0/1047490904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac012070 con 0x557ec0a4e400 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.296+0000 7fbea7e07640 1 -- 172.21.15.63:0/1047490904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfac01a7a0 con 0x557ec0a4e400 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.296+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ec0a4f6b0 con 0x557ec0a4e400 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.296+0000 7fbea7e07640 1 -- 172.21.15.63:0/1047490904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac01aa80 con 0x557ec0a4e400 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.296+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec0a4fbd0 con 0x557ec0a4e400 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbea7e07640 1 -- 172.21.15.63:0/1047490904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbfac015020 con 0x557ec0a4e400 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbea7e07640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbea0036eb0 0x7fbea0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbea7e07640 1 -- 172.21.15.63:0/1047490904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fbfac020070 con 0x557ec0a4e400 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbfbaf3d640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbea0036eb0 0x7fbea0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0a54620 0x557ec0b52350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b52920 con 0x557ec0a54620 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0a54620 0x557ec0b52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.297+0000 7fbfbaf3d640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbea0036eb0 0x7fbea0039370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0054000 tx=0x7fbfb004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.298+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0a54620 0x557ec0b52350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.298+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/1047490904 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbfb400a4a0 con 0x557ec0a54620 2024-04-23T19:10:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.298+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b53230 con 0x557ec0a54620 2024-04-23T19:10:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.298+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/1047490904 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbfb400a4a0 con 0x557ec0a54620 2024-04-23T19:10:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.298+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b545e0 con 0x557ec0a54620 2024-04-23T19:10:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.298+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/1047490904 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbfb400a4a0 con 0x557ec0a54620 2024-04-23T19:10:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.298+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b53a80 con 0x557ec0a54620 2024-04-23T19:10:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.299+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/1047490904 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbfb400a4a0 con 0x557ec0a54620 2024-04-23T19:10:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.299+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557ec0b55a50 0x557ec0b57e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.299+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b585b0 con 0x557ec0b55a50 2024-04-23T19:10:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.299+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557ec0b55a50 0x557ec0b57e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.299+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557ec0b55a50 0x557ec0b57e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbb73e640 1 -- 172.21.15.63:0/1047490904 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbfac02b070 con 0x557ec0b55a50 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbdc3f800 1 Processor -- start 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbdc3f800 1 -- start start 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbdc3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b669a0 0x557ec0b66dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbdc3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ec0a51270 con 0x557ec0b669a0 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbbf3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b669a0 0x557ec0b66dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbbf3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b669a0 0x557ec0b66dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47918/0 (socket says 172.21.15.63:47918) 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.300+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/1458585473 learned_addr learned my addr 172.21.15.63:0/1458585473 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.301+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/1458585473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec0b67300 con 0x557ec0b669a0 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.301+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b669a0 0x557ec0b66dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x557ec0b52d00 tx=0x7fbfb40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.301+0000 7fbea5602640 1 -- 172.21.15.63:0/1458585473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb4013440 con 0x557ec0b669a0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.301+0000 7fbea5602640 1 -- 172.21.15.63:0/1458585473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb40135e0 con 0x557ec0b669a0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.301+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ec0b67d70 con 0x557ec0b669a0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.301+0000 7fbea5602640 1 -- 172.21.15.63:0/1458585473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb40138c0 con 0x557ec0b669a0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.301+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec0b68270 con 0x557ec0b669a0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.302+0000 7fbea5602640 1 -- 172.21.15.63:0/1458585473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbfb4011040 con 0x557ec0b669a0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.302+0000 7fbea5602640 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe94036e60 0x7fbe94039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.302+0000 7fbea5602640 1 -- 172.21.15.63:0/1458585473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fbfb401d070 con 0x557ec0b669a0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.302+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe94036e60 0x7fbe94039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbdc3f800 1 Processor -- start 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe94036e60 0x7fbe94039320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x557ec0b53e20 tx=0x7fbfac04af80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbdc3f800 1 -- start start 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbdc3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbdc3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ec0941e30 con 0x557ec0b73d30 2024-04-23T19:10:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbbf3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbbf3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b74150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47922/0 (socket says 172.21.15.63:47922) 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/3739902921 learned_addr learned my addr 172.21.15.63:0/3739902921 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/3739902921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec0b74720 con 0x557ec0b73d30 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.303+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/3739902921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b74150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4001470 tx=0x7fbfb4012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c918ba91c8f07c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbe9effd640 1 -- 172.21.15.63:0/3739902921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb4015430 con 0x557ec0b73d30 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbe9effd640 1 -- 172.21.15.63:0/3739902921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb402ac50 con 0x557ec0b73d30 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbe9effd640 1 -- 172.21.15.63:0/3739902921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb40155d0 con 0x557ec0b73d30 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3739902921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 msgr2=0x557ec0b74150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/3739902921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b74150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4001470 tx=0x7fbfb4012a30 comp rx=0 tx=0).stop 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3739902921 shutdown_connections 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/3739902921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b74150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3739902921 >> 172.21.15.63:0/3739902921 conn(0x557ec0b70000 msgr2=0x557ec0b6e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3739902921 shutdown_connections 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/3739902921 wait complete. 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.304+0000 7fbfbdc3f800 1 Processor -- start 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbdc3f800 1 -- start start 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbdc3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b7b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbdc3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ec0b74d30 con 0x557ec0b73d30 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbbf3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b7b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbbf3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b7b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47924/0 (socket says 172.21.15.63:47924) 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 learned_addr learned my addr 172.21.15.63:0/273663434 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec0b7b680 con 0x557ec0b73d30 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b7b140 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfb404cb80 tx=0x7fbfb404cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.305+0000 7fbe9d7fa640 1 -- 172.21.15.63:0/273663434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb4030070 con 0x557ec0b73d30 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.306+0000 7fbe9d7fa640 1 -- 172.21.15.63:0/273663434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4013e20 con 0x557ec0b73d30 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.306+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ec0b7b970 con 0x557ec0b73d30 2024-04-23T19:10:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.306+0000 7fbe9d7fa640 1 -- 172.21.15.63:0/273663434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb40125b0 con 0x557ec0b73d30 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.306+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec0b7e5c0 con 0x557ec0b73d30 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.306+0000 7fbe9d7fa640 1 -- 172.21.15.63:0/273663434 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbfb404e050 con 0x557ec0b73d30 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.306+0000 7fbe9d7fa640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe84036eb0 0x7fbe84039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.307+0000 7fbe9d7fa640 1 -- 172.21.15.63:0/273663434 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fbfb4044040 con 0x557ec0b73d30 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.307+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe84036eb0 0x7fbe84039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.307+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe84036eb0 0x7fbe84039370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfac04b0b0 tx=0x7fbfac030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.307+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.307+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0b86e30 0x557ec0b892f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.307+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b89a40 con 0x557ec0b86e30 2024-04-23T19:10:25.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.307+0000 7fbfbaf3d640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0b86e30 0x557ec0b892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbaf3d640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0b86e30 0x557ec0b892f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 20 realm 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b89e10 con 0x557ec0b86e30 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8a6e0 con 0x557ec0b86e30 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.308+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8aa80 con 0x557ec0b86e30 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbdc3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8b2d0 con 0x557ec0b86e30 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbdc3f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbdc3f800 20 searching for the correct realm 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557ec0b8d040 0x557ec0b8f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557ec0b8d040 0x557ec0b8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.309+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557ec0b8d040 0x557ec0b8f430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.310+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.310+0000 7fbfbbf3f640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbfb403e110 0x7fbfb404e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.310+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfb403e110 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.310+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbfb403e110 0x7fbfb404e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.310+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbfb403e110 0x7fbfb404e9b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.310+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.311+0000 7fbfbb73e640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbfac017050 0x7fbfac04c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.311+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.311+0000 7fbfbaf3d640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbfac017050 0x7fbfac04c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.311+0000 7fbfbaf3d640 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbfac017050 0x7fbfac04c430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.312+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfb403e110 2024-04-23T19:10:25.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.313+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfb403e110 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.314+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.315+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.316+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfb403e110 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v23'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.317+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.318+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.318+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfb403e110 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.318+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.318+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b8d040 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.318+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.318+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfac017050 2024-04-23T19:10:25.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x557ec0b86e30 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8f9c0 con 0x7fbfb403e110 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8bc50 con 0x557ec0b86e30 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.319+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8c4f0 con 0x557ec0b86e30 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8c4f0 con 0x7fbfac017050 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:25.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 20 zone default found 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 4 Realm: () 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 20 zonegroup default 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.320+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8fe40 con 0x7fbfac017050 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b8c210 con 0x557ec0b8d040 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b91dc0 con 0x557ec0b8d040 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 20 started sync module instance, tier type = 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.321+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b91160 con 0x7fbfb403e110 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.322+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.322+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94002887595344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b90dc0 con 0x7fbfb403e110 2024-04-23T19:10:25.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.322+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b92cf0 con 0x557ec0b86e30 2024-04-23T19:10:25.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.323+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94002887595344] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.324+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.324+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94002887600192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b93110 con 0x557ec0b86e30 2024-04-23T19:10:25.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.324+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b93ad0 con 0x557ec0b8d040 2024-04-23T19:10:25.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.326+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.326+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94002887600192] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.326+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94002887603744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b94320 con 0x557ec0b8d040 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.326+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b92c80 con 0x7fbfb403e110 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.327+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94002887603744] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.327+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.328+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94002887606352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b94dc0 con 0x7fbfb403e110 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.328+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b95780 con 0x557ec0b86e30 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.329+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.329+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94002887611088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b95ba0 con 0x557ec0b86e30 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.329+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b96680 con 0x7fbfac017050 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.329+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94002887606352] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.330+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.330+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94002887611088] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.330+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94002887608592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b96640 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.330+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b956f0 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.331+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94002887608592] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.331+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.331+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94002887614864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b965b0 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.331+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b97570 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.332+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94002887614864] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94002887617552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b974e0 con 0x7fbfac017050 2024-04-23T19:10:25.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=0 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=1 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=2 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=3 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=4 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=5 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=6 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94002887617552] v30'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.333+0000 7fbfbdc3f800 20 add_watcher() i=7 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.334+0000 7fbfbdc3f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.334+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b9a940 con 0x557ec0b86e30 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.334+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.334+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94002887619072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b97e00 con 0x557ec0b86e30 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.335+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94002887619072] v30'31 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.335+0000 7fbe71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.335+0000 7fbfbdc3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.335+0000 7fbfbdc3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.335+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b9a5a0 con 0x557ec0b86e30 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.337+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.337+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba39b0 con 0x7fbfac017050 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.338+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'11 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.338+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba3e60 con 0x557ec0b8d040 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.341+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.341+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba3e60 con 0x557ec0b86e30 2024-04-23T19:10:25.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.344+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'32 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.344+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba42a0 con 0x7fbfb403e110 2024-04-23T19:10:25.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.346+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'23 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.346+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba4810 con 0x557ec0b8d040 2024-04-23T19:10:25.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.349+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'15 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.349+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba4cc0 con 0x557ec0b86e30 2024-04-23T19:10:25.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.351+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.351+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba4cc0 con 0x7fbfac017050 2024-04-23T19:10:25.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.352+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'15 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.352+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba4cc0 con 0x7fbfb403e110 2024-04-23T19:10:25.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.353+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.353+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba4af0 con 0x557ec0b8d040 2024-04-23T19:10:25.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.354+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'17 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.354+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba4b80 con 0x7fbfb403e110 2024-04-23T19:10:25.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.357+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'24 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.357+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x7fbfac017050 2024-04-23T19:10:25.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.358+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'13 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.358+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x7fbfb403e110 2024-04-23T19:10:25.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.360+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'11 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.360+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x7fbfac017050 2024-04-23T19:10:25.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.362+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'21 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.363+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b86e30 2024-04-23T19:10:25.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.366+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'33 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.366+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b8d040 2024-04-23T19:10:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.368+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'17 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.368+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x7fbfb403e110 2024-04-23T19:10:25.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.370+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'25 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.370+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b8d040 2024-04-23T19:10:25.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.371+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.371+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b86e30 2024-04-23T19:10:25.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.373+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.373+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b86e30 2024-04-23T19:10:25.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.374+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'34 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.374+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b8d040 2024-04-23T19:10:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.377+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.377+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x7fbfb403e110 2024-04-23T19:10:25.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.378+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.378+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b8d040 2024-04-23T19:10:25.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.379+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.379+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b8d040 2024-04-23T19:10:25.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.382+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.382+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b8d040 2024-04-23T19:10:25.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.383+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'7 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.383+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x557ec0b8d040 2024-04-23T19:10:25.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.385+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'16 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.385+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba5120 con 0x7fbfac017050 2024-04-23T19:10:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.386+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.386+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba50d0 con 0x557ec0b8d040 2024-04-23T19:10:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.387+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.387+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba50d0 con 0x7fbfac017050 2024-04-23T19:10:25.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.389+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'16 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.389+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba50d0 con 0x557ec0b8d040 2024-04-23T19:10:25.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.390+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'19 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.390+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba50d0 con 0x7fbfac017050 2024-04-23T19:10:25.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.391+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'22 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.391+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba50d0 con 0x7fbfb403e110 2024-04-23T19:10:25.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbe70fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:25.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbfbdc3f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:25.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbfbdc3f800 20 Filter name: none 2024-04-23T19:10:25.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbdea7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:25.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbfbdc3f800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-23T19:10:25.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbfbdc3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557ec0bcff90 con 0x7fbfac017050 2024-04-23T19:10:25.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.393+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(94 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.394+0000 7fbfbdc3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.394+0000 7fbfbdc3f800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-23T19:10:25.395 INFO:teuthology.orchestra.run.smithi063.stderr:could not remove user: unable to remove user, user does not exist 2024-04-23T19:10:25.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.394+0000 7fbfbdc3f800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-23T19:10:25.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.394+0000 7fbe71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.394+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94002887619072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0ba2b70 con 0x557ec0b86e30 2024-04-23T19:10:25.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.395+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94002887619072] v30'35 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.395+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94002887595344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b971d0 con 0x7fbfb403e110 2024-04-23T19:10:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.398+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94002887595344] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.398+0000 7fbfbdc3f800 20 remove_watcher() i=0 2024-04-23T19:10:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.398+0000 7fbfbdc3f800 2 removed watcher, disabling cache 2024-04-23T19:10:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.398+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94002887600192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b91550 con 0x557ec0b86e30 2024-04-23T19:10:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.399+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94002887600192] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.399+0000 7fbfbdc3f800 20 remove_watcher() i=1 2024-04-23T19:10:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.399+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94002887603744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b91550 con 0x557ec0b8d040 2024-04-23T19:10:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.400+0000 7fbfbbf3f640 1 -- 172.21.15.63:0/273663434 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94002887603744] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb4029070 con 0x557ec0b8d040 2024-04-23T19:10:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.400+0000 7fbfbdc3f800 20 remove_watcher() i=2 2024-04-23T19:10:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.400+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94002887606352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b91550 con 0x7fbfb403e110 2024-04-23T19:10:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.401+0000 7fbfbb73e640 1 -- 172.21.15.63:0/273663434 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94002887606352] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfac055040 con 0x7fbfb403e110 2024-04-23T19:10:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.401+0000 7fbfbdc3f800 20 remove_watcher() i=3 2024-04-23T19:10:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.401+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94002887611088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b94050 con 0x557ec0b86e30 2024-04-23T19:10:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.403+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94002887611088] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x557ec0b86e30 2024-04-23T19:10:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.403+0000 7fbfbdc3f800 20 remove_watcher() i=4 2024-04-23T19:10:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.403+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94002887608592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b91550 con 0x7fbfac017050 2024-04-23T19:10:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.404+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94002887608592] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.404+0000 7fbfbdc3f800 20 remove_watcher() i=5 2024-04-23T19:10:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.404+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94002887614864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b952d0 con 0x7fbfac017050 2024-04-23T19:10:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.405+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94002887614864] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.405+0000 7fbfbdc3f800 20 remove_watcher() i=6 2024-04-23T19:10:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.405+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94002887617552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557ec0b952d0 con 0x7fbfac017050 2024-04-23T19:10:25.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94002887617552] v30'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbfb005a040 con 0x7fbfac017050 2024-04-23T19:10:25.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 20 remove_watcher() i=7 2024-04-23T19:10:25.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbfac017050 msgr2=0x7fbfac04c430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbfac017050 0x7fbfac04c430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0b86e30 msgr2=0x557ec0b892f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0b86e30 0x557ec0b892f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbfb403e110 msgr2=0x7fbfb404e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbfb403e110 0x7fbfb404e9b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557ec0b8d040 msgr2=0x557ec0b8f430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557ec0b8d040 0x557ec0b8f430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe84036eb0 msgr2=0x7fbe84039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe84036eb0 0x7fbe84039370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfac04b0b0 tx=0x7fbfac030000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 msgr2=0x557ec0b7b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.406+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/273663434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b73d30 0x557ec0b7b140 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfb404cb80 tx=0x7fbfb404cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 reap_dead start 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbaf3d640 1 -- 172.21.15.63:0/273663434 reap_dead start 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 shutdown_connections 2024-04-23T19:10:25.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 >> 172.21.15.63:0/273663434 conn(0x557ec0b70000 msgr2=0x557ec0b6d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 shutdown_connections 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/273663434 wait complete. 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe94036e60 msgr2=0x7fbe94039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe94036e60 0x7fbe94039320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x557ec0b53e20 tx=0x7fbfac04af80 comp rx=0 tx=0).stop 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b669a0 msgr2=0x557ec0b66dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b669a0 0x557ec0b66dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x557ec0b52d00 tx=0x7fbfb40178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 shutdown_connections 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe94036e60 0x7fbe94039320 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1458585473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0b669a0 0x557ec0b66dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 >> 172.21.15.63:0/1458585473 conn(0x557ec0b5aa40 msgr2=0x557ec0b5ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 shutdown_connections 2024-04-23T19:10:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1458585473 wait complete. 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557ec0b55a50 msgr2=0x557ec0b57e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557ec0b55a50 0x557ec0b57e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0a54620 msgr2=0x557ec0b52350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0a54620 0x557ec0b52350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbea0036eb0 msgr2=0x7fbea0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.407+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbea0036eb0 0x7fbea0039370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0054000 tx=0x7fbfb004e000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 msgr2=0x557ec0a50d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0a50d30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c010 tx=0x7fbfac000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 shutdown_connections 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557ec0b55a50 0x557ec0b57e40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557ec0a54620 0x557ec0b52350 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbea0036eb0 0x7fbea0039370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 --2- 172.21.15.63:0/1047490904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ec0a4e400 0x557ec0a50d30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 >> 172.21.15.63:0/1047490904 conn(0x557ec0a49db0 msgr2=0x557ec0b3b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 shutdown_connections 2024-04-23T19:10:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.408+0000 7fbfbdc3f800 1 -- 172.21.15.63:0/1047490904 wait complete. 2024-04-23T19:10:25.416 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-23T19:10:25.416 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-23T19:10:25.416 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-23T19:10:25.417 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-23T19:10:25.503 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:25.503 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:25.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.503+0000 7f5f35546800 1 Processor -- start 2024-04-23T19:10:25.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f35546800 1 -- start start 2024-04-23T19:10:25.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f35546800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5576196e1740 0x5576196e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f35546800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55761958f7f0 con 0x5576196ddc30 2024-04-23T19:10:25.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f35546800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55761957ee20 con 0x5576196e1740 2024-04-23T19:10:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f3284b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5576196e1740 0x5576196e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f3304c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5576196ddc30 0x5576196e0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52622/0 (socket says 172.21.15.63:52622) 2024-04-23T19:10:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f3304c640 1 -- 172.21.15.63:0/1710872950 learned_addr learned my addr 172.21.15.63:0/1710872950 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.504+0000 7f5f3284b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5576196e1740 0x5576196e3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47930/0 (socket says 172.21.15.63:47930) 2024-04-23T19:10:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.505+0000 7f5f3284b640 1 -- 172.21.15.63:0/1710872950 >> v1:172.21.15.16:6789/0 conn(0x5576196ddc30 legacy=0x5576196e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.505+0000 7f5f3204a640 1 -- 172.21.15.63:0/1710872950 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765036238 0 0) 0x55761958f7f0 con 0x5576196ddc30 2024-04-23T19:10:25.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.505+0000 7f5f3284b640 1 -- 172.21.15.63:0/1710872950 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761959db50 con 0x5576196e1740 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f3284b640 1 --2- 172.21.15.63:0/1710872950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196e1740 0x5576196e3b30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800a470 tx=0x7f5f2801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28775840c4a930f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f3204a640 1 -- 172.21.15.63:0/1710872950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f28057070 con 0x5576196e1740 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f3204a640 1 -- 172.21.15.63:0/1710872950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f28052570 con 0x5576196e1740 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f3204a640 1 -- 172.21.15.63:0/1710872950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f28052870 con 0x5576196e1740 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f35546800 1 -- 172.21.15.63:0/1710872950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196e1740 msgr2=0x5576196e3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f35546800 1 --2- 172.21.15.63:0/1710872950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196e1740 0x5576196e3b30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800a470 tx=0x7f5f2801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f35546800 1 -- 172.21.15.63:0/1710872950 shutdown_connections 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f35546800 1 --2- 172.21.15.63:0/1710872950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196e1740 0x5576196e3b30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f35546800 1 -- 172.21.15.63:0/1710872950 >> 172.21.15.63:0/1710872950 conn(0x5576196d7600 msgr2=0x5576196d9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f35546800 1 -- 172.21.15.63:0/1710872950 shutdown_connections 2024-04-23T19:10:25.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.507+0000 7f5f35546800 1 -- 172.21.15.63:0/1710872950 wait complete. 2024-04-23T19:10:25.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.522+0000 7f5f35546800 1 Processor -- start 2024-04-23T19:10:25.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.522+0000 7f5f35546800 1 -- start start 2024-04-23T19:10:25.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.522+0000 7f5f35546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576196dd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.522+0000 7f5f35546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576196286f0 con 0x5576196ddc30 2024-04-23T19:10:25.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5f3304c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576196dd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5f3304c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576196dd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47942/0 (socket says 172.21.15.63:47942) 2024-04-23T19:10:25.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5f3304c640 1 -- 172.21.15.63:0/148549716 learned_addr learned my addr 172.21.15.63:0/148549716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5f3304c640 1 -- 172.21.15.63:0/148549716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576196dbb30 con 0x5576196ddc30 2024-04-23T19:10:25.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5f3304c640 1 --2- 172.21.15.63:0/148549716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576196dd570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b210 tx=0x7f5f2400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab79b38e58128fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5e2160a640 1 -- 172.21.15.63:0/148549716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f24008070 con 0x5576196ddc30 2024-04-23T19:10:25.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5e2160a640 1 -- 172.21.15.63:0/148549716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2400d500 con 0x5576196ddc30 2024-04-23T19:10:25.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.523+0000 7f5e2160a640 1 -- 172.21.15.63:0/148549716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2400d840 con 0x5576196ddc30 2024-04-23T19:10:25.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 -- 172.21.15.63:0/148549716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 msgr2=0x5576196dd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 --2- 172.21.15.63:0/148549716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576196dd570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b210 tx=0x7f5f2400b6d0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 -- 172.21.15.63:0/148549716 shutdown_connections 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 --2- 172.21.15.63:0/148549716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576196dd570 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 -- 172.21.15.63:0/148549716 >> 172.21.15.63:0/148549716 conn(0x5576196d7600 msgr2=0x5576197c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 -- 172.21.15.63:0/148549716 shutdown_connections 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 -- 172.21.15.63:0/148549716 wait complete. 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 Processor -- start 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.524+0000 7f5f35546800 1 -- start start 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f35546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576197d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f35546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576196e6380 con 0x5576196ddc30 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f3304c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576197d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f3304c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576197d7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47944/0 (socket says 172.21.15.63:47944) 2024-04-23T19:10:25.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f3304c640 1 -- 172.21.15.63:0/3095997481 learned_addr learned my addr 172.21.15.63:0/3095997481 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f3304c640 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576197d8460 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f3304c640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576197d7f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b210 tx=0x7f5f24000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5e1fe07640 1 -- 172.21.15.63:0/3095997481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2400f950 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5e1fe07640 1 -- 172.21.15.63:0/3095997481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2400faf0 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5e1fe07640 1 -- 172.21.15.63:0/3095997481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2400fdd0 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.525+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576197d86f0 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.526+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576197db3a0 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.526+0000 7f5e1fe07640 1 -- 172.21.15.63:0/3095997481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5f24017040 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.526+0000 7f5e1fe07640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036e60 0x7f5e18039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.527+0000 7f5e1fe07640 1 -- 172.21.15.63:0/3095997481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f5f24015020 con 0x5576196ddc30 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.527+0000 7f5f3284b640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036e60 0x7f5e18039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.527+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5576196e1740 0x5576197df160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.527+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5576197df6a0 con 0x5576196e1740 2024-04-23T19:10:25.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.527+0000 7f5f3384d640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5576196e1740 0x5576197df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.527+0000 7f5f3284b640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036e60 0x7f5e18039320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f28054000 tx=0x7f5f2804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.527+0000 7f5f3384d640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5576196e1740 0x5576197df160 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.528+0000 7f5f3384d640 1 -- 172.21.15.63:0/3095997481 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c00a4a0 con 0x5576196e1740 2024-04-23T19:10:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.528+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5576197dc730 con 0x5576196e1740 2024-04-23T19:10:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.528+0000 7f5f3384d640 1 -- 172.21.15.63:0/3095997481 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5f2c00a4a0 con 0x5576196e1740 2024-04-23T19:10:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.528+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5576197e1510 con 0x5576196e1740 2024-04-23T19:10:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.529+0000 7f5f3384d640 1 -- 172.21.15.63:0/3095997481 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5f2c00a4a0 con 0x5576196e1740 2024-04-23T19:10:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.529+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5576197e0850 con 0x5576196e1740 2024-04-23T19:10:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.529+0000 7f5f3384d640 1 -- 172.21.15.63:0/3095997481 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5f2c00a4a0 con 0x5576196e1740 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.529+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5576197e28c0 0x5576197e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.529+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5576197e5270 con 0x5576197e28c0 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.529+0000 7f5f3304c640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5576197e28c0 0x5576197e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.529+0000 7f5f3304c640 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5576197e28c0 0x5576197e4cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.530+0000 7f5f3304c640 1 -- 172.21.15.63:0/3095997481 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5f2402d070 con 0x5576197e28c0 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.530+0000 7f5f35546800 1 Processor -- start 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.530+0000 7f5f35546800 1 -- start start 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f35546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576197f3600 0x5576197f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f35546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576195cee30 con 0x5576197f3600 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f3384d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576197f3600 0x5576197f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f3384d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576197f3600 0x5576197f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47954/0 (socket says 172.21.15.63:47954) 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f3384d640 1 -- 172.21.15.63:0/4035551915 learned_addr learned my addr 172.21.15.63:0/4035551915 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f3384d640 1 -- 172.21.15.63:0/4035551915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576197f3f60 con 0x5576197f3600 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f3384d640 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576197f3600 0x5576197f3a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5576197dfa80 tx=0x7f5f2c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5e1d602640 1 -- 172.21.15.63:0/4035551915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c013440 con 0x5576197f3600 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5e1d602640 1 -- 172.21.15.63:0/4035551915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c0135e0 con 0x5576197f3600 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576197f49d0 con 0x5576197f3600 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.531+0000 7f5e1d602640 1 -- 172.21.15.63:0/4035551915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c0138c0 con 0x5576197f3600 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.532+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576197f4ed0 con 0x5576197f3600 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.532+0000 7f5e1d602640 1 -- 172.21.15.63:0/4035551915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5f2c011040 con 0x5576197f3600 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.532+0000 7f5e1d602640 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036eb0 0x7f5e0c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.532+0000 7f5e1d602640 1 -- 172.21.15.63:0/4035551915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f5f2c01d070 con 0x5576197f3600 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f3304c640 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036eb0 0x7f5e0c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f35546800 1 Processor -- start 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f35546800 1 -- start start 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f3304c640 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036eb0 0x7f5e0c039370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f24028000 tx=0x7f5f2404ba00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f35546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x5576198076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f35546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576196dbeb0 con 0x5576198052d0 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f3384d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x5576198076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f3384d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x5576198076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47966/0 (socket says 172.21.15.63:47966) 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f3384d640 1 -- 172.21.15.63:0/3557735733 learned_addr learned my addr 172.21.15.63:0/3557735733 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.533+0000 7f5f3384d640 1 -- 172.21.15.63:0/3557735733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557619807c00 con 0x5576198052d0 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f3384d640 1 --2- 172.21.15.63:0/3557735733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x5576198076c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c04cc70 tx=0x7f5f2c02afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a422e46bccfbbf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5e16ffd640 1 -- 172.21.15.63:0/3557735733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c042070 con 0x5576198052d0 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5e16ffd640 1 -- 172.21.15.63:0/3557735733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c02b3a0 con 0x5576198052d0 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5e16ffd640 1 -- 172.21.15.63:0/3557735733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c012ab0 con 0x5576198052d0 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f35546800 1 -- 172.21.15.63:0/3557735733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 msgr2=0x5576198076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f35546800 1 --2- 172.21.15.63:0/3557735733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x5576198076c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c04cc70 tx=0x7f5f2c02afb0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f35546800 1 -- 172.21.15.63:0/3557735733 shutdown_connections 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f35546800 1 --2- 172.21.15.63:0/3557735733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x5576198076c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f35546800 1 -- 172.21.15.63:0/3557735733 >> 172.21.15.63:0/3557735733 conn(0x5576197cfd90 msgr2=0x5576197cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f35546800 1 -- 172.21.15.63:0/3557735733 shutdown_connections 2024-04-23T19:10:25.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.534+0000 7f5f35546800 1 -- 172.21.15.63:0/3557735733 wait complete. 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f35546800 1 Processor -- start 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f35546800 1 -- start start 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f35546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x55761980c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f35546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557619808010 con 0x5576198052d0 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f3384d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x55761980c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f3384d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x55761980c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:47972/0 (socket says 172.21.15.63:47972) 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 learned_addr learned my addr 172.21.15.63:0/1866766273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761980c750 con 0x5576198052d0 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.535+0000 7f5f3384d640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x55761980c210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c04bb90 tx=0x7f5f2c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.536+0000 7f5e157fa640 1 -- 172.21.15.63:0/1866766273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c021c60 con 0x5576198052d0 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.536+0000 7f5e157fa640 1 -- 172.21.15.63:0/1866766273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c021e00 con 0x5576198052d0 2024-04-23T19:10:25.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.536+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55761980c9e0 con 0x5576198052d0 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.536+0000 7f5e157fa640 1 -- 172.21.15.63:0/1866766273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c02a480 con 0x5576198052d0 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.536+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761980f690 con 0x5576198052d0 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5e157fa640 1 -- 172.21.15.63:0/1866766273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5f2c02a620 con 0x5576198052d0 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5e157fa640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5dfc036eb0 0x7f5dfc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5e157fa640 1 -- 172.21.15.63:0/1866766273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f5f2c032030 con 0x5576198052d0 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5f3304c640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5dfc036eb0 0x7f5dfc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557619817c90 0x55761981a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981a8a0 con 0x557619817c90 2024-04-23T19:10:25.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5f3304c640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5dfc036eb0 0x7f5dfc039370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400f640 tx=0x7f5f24042000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.537+0000 7f5f3284b640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557619817c90 0x55761981a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f3284b640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557619817c90 0x55761981a150 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 20 realm 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981ac70 con 0x557619817c90 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.538+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981b540 con 0x557619817c90 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981b8e0 con 0x557619817c90 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981c130 con 0x557619817c90 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 20 searching for the correct realm 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55761981dea0 0x557619820290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.539+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.540+0000 7f5f3384d640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55761981dea0 0x557619820290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.540+0000 7f5f3384d640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55761981dea0 0x557619820290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.540+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.540+0000 7f5f3384d640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5f2c04eb60 0x7f5f2c02c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.540+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2c04eb60 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.540+0000 7f5f3304c640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5f2c04eb60 0x7f5f2c02c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.541+0000 7f5f3304c640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5f2c04eb60 0x7f5f2c02c650 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.541+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.541+0000 7f5f3304c640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5f2404d7c0 0x7f5f2401c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.541+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.541+0000 7f5f3284b640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5f2404d7c0 0x7f5f2401c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.542+0000 7f5f3284b640 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5f2404d7c0 0x7f5f2401c5e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.542+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.542+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.542+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.542+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.543+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.543+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.543+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.543+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.543+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.543+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2c04eb60 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.544+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.544+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.544+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.544+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.544+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.544+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2c04eb60 2024-04-23T19:10:25.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.545+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.546+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.547+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.547+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.547+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.547+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.547+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.547+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.548+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.548+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2c04eb60 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.548+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.548+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.548+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v23'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.548+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2c04eb60 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.549+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x55761981dea0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.550+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.550+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2404d7c0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.550+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.550+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x557619817c90 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.550+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.550+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557619820820 con 0x7f5f2c04eb60 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981cae0 con 0x557619817c90 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981d380 con 0x557619817c90 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.551+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981d380 con 0x7f5f2404d7c0 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 20 zone default found 2024-04-23T19:10:25.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 4 Realm: () 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 20 zonegroup default 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.552+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557619820ca0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55761981d150 con 0x55761981dea0 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557619822c20 con 0x55761981dea0 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 20 started sync module instance, tier type = 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.553+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619821fc0 con 0x7f5f2c04eb60 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.554+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.554+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93965722461104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619821c20 con 0x7f5f2c04eb60 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.554+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619823b50 con 0x557619817c90 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.558+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93965722461104] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.558+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.558+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93965722465952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619823ac0 con 0x557619817c90 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.558+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619824a10 con 0x55761981dea0 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.560+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.560+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93965722469728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5576198249c0 con 0x55761981dea0 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.560+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619825990 con 0x7f5f2c04eb60 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.560+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93965722465952] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.563+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93965722469728] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.563+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.563+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93965722473600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619824e00 con 0x7f5f2c04eb60 2024-04-23T19:10:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.563+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5576198263d0 con 0x557619817c90 2024-04-23T19:10:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.566+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.566+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93965722476592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619826ac0 con 0x557619817c90 2024-04-23T19:10:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.566+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5576198275b0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.566+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93965722473600] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.567+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93965722476592] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.567+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.567+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93965722480768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619827570 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.567+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619827e50 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.568+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93965722480768] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.569+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.569+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93965722468160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619827570 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.569+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619828420 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93965722484592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619828840 con 0x7f5f2404d7c0 2024-04-23T19:10:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 20 add_watcher() i=0 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 20 add_watcher() i=1 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 20 add_watcher() i=2 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 20 add_watcher() i=3 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 20 add_watcher() i=4 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 20 add_watcher() i=5 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93965722468160] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.570+0000 7f5f35546800 20 add_watcher() i=6 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.571+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93965722484592] v30'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.571+0000 7f5f35546800 20 add_watcher() i=7 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.571+0000 7f5f35546800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.571+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557619828840 con 0x557619817c90 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.571+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.571+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93965722485648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5576198287b0 con 0x557619817c90 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.573+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93965722485648] v30'36 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.574+0000 7f5de9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.574+0000 7f5f35546800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.574+0000 7f5f35546800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.574+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619834140 con 0x557619817c90 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.575+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.575+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619834790 con 0x7f5f2404d7c0 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.576+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.576+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619834c10 con 0x55761981dea0 2024-04-23T19:10:25.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.579+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.579+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619834b60 con 0x557619817c90 2024-04-23T19:10:25.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.580+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'37 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.580+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835190 con 0x7f5f2c04eb60 2024-04-23T19:10:25.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.583+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'26 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.583+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835700 con 0x55761981dea0 2024-04-23T19:10:25.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.584+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'17 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.584+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835700 con 0x557619817c90 2024-04-23T19:10:25.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.585+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.585+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835700 con 0x7f5f2404d7c0 2024-04-23T19:10:25.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.586+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'17 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.586+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835700 con 0x7f5f2c04eb60 2024-04-23T19:10:25.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.588+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.588+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.589+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'18 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.589+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2c04eb60 2024-04-23T19:10:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.591+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'27 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.591+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.592+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.592+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2c04eb60 2024-04-23T19:10:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.594+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.594+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.595+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'23 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.595+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x557619817c90 2024-04-23T19:10:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.596+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'38 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.596+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.597+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'20 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.597+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2c04eb60 2024-04-23T19:10:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.598+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'28 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.598+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.600+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.600+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x557619817c90 2024-04-23T19:10:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.601+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.601+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x557619817c90 2024-04-23T19:10:25.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.602+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'39 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.602+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.604+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.604+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2c04eb60 2024-04-23T19:10:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.606+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.606+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.607+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.607+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.608+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.608+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.609+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'8 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.609+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.610+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'18 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.610+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.612+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.612+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.613+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.613+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.614+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'18 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.614+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x55761981dea0 2024-04-23T19:10:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.615+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'22 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.615+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.616+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'24 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.616+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619835bb0 con 0x7f5f2c04eb60 2024-04-23T19:10:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.617+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5de8fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f35546800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f35546800 20 Filter name: none 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5d627fc640 20 reqs_thread_entry: start 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f35546800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f35546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557619860b20 con 0x7f5f2404d7c0 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(94 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f35546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.618+0000 7f5f35546800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr:could not remove user: unable to remove user, user does not exist2024-04-23T19:10:25.618+0000 7f5f35546800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:25.619 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-04-23T19:10:25.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.619+0000 7f5de9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.619+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93965722485648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619832b60 con 0x557619817c90 2024-04-23T19:10:25.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.621+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 93965722485648] v30'40 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.621+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93965722461104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619828390 con 0x7f5f2c04eb60 2024-04-23T19:10:25.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.622+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 93965722461104] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.622+0000 7f5f35546800 20 remove_watcher() i=0 2024-04-23T19:10:25.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.622+0000 7f5f35546800 2 removed watcher, disabling cache 2024-04-23T19:10:25.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.622+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93965722465952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619828390 con 0x557619817c90 2024-04-23T19:10:25.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.623+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 93965722465952] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.623+0000 7f5f35546800 20 remove_watcher() i=1 2024-04-23T19:10:25.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.623+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93965722469728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5576198236a0 con 0x55761981dea0 2024-04-23T19:10:25.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.624+0000 7f5f3384d640 1 -- 172.21.15.63:0/1866766273 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 93965722469728] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2c044070 con 0x55761981dea0 2024-04-23T19:10:25.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.624+0000 7f5f35546800 20 remove_watcher() i=2 2024-04-23T19:10:25.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.624+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93965722473600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619828390 con 0x7f5f2c04eb60 2024-04-23T19:10:25.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.626+0000 7f5f3304c640 1 -- 172.21.15.63:0/1866766273 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 93965722473600] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2402d070 con 0x7f5f2c04eb60 2024-04-23T19:10:25.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.626+0000 7f5f35546800 20 remove_watcher() i=3 2024-04-23T19:10:25.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.626+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93965722476592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619824560 con 0x557619817c90 2024-04-23T19:10:25.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.627+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 93965722476592] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x557619817c90 2024-04-23T19:10:25.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.627+0000 7f5f35546800 20 remove_watcher() i=4 2024-04-23T19:10:25.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.627+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93965722480768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5576198236a0 con 0x7f5f2404d7c0 2024-04-23T19:10:25.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.628+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 93965722480768] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.628+0000 7f5f35546800 20 remove_watcher() i=5 2024-04-23T19:10:25.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.628+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93965722468160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619828390 con 0x7f5f2404d7c0 2024-04-23T19:10:25.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.629+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 93965722468160] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.629+0000 7f5f35546800 20 remove_watcher() i=6 2024-04-23T19:10:25.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.629+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93965722484592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557619826030 con 0x7f5f2404d7c0 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 93965722484592] v30'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5f2805a040 con 0x7f5f2404d7c0 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 20 remove_watcher() i=7 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5f2404d7c0 msgr2=0x7f5f2401c5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5f2404d7c0 0x7f5f2401c5e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557619817c90 msgr2=0x55761981a150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557619817c90 0x55761981a150 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5f2c04eb60 msgr2=0x7f5f2c02c650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5f2c04eb60 0x7f5f2c02c650 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55761981dea0 msgr2=0x557619820290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55761981dea0 0x557619820290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5dfc036eb0 msgr2=0x7f5dfc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5dfc036eb0 0x7f5dfc039370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400f640 tx=0x7f5f24042000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 msgr2=0x55761980c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x55761980c210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c04bb90 tx=0x7f5f2c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f3284b640 1 -- 172.21.15.63:0/1866766273 reap_dead start 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 shutdown_connections 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 --2- 172.21.15.63:0/1866766273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576198052d0 0x55761980c210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 >> 172.21.15.63:0/1866766273 conn(0x5576197cfd90 msgr2=0x5576197fa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 shutdown_connections 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.630+0000 7f5f35546800 1 -- 172.21.15.63:0/1866766273 wait complete. 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036eb0 msgr2=0x7f5e0c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036eb0 0x7f5e0c039370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f24028000 tx=0x7f5f2404ba00 comp rx=0 tx=0).stop 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576197f3600 msgr2=0x5576197f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576197f3600 0x5576197f3a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5576197dfa80 tx=0x7f5f2c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 shutdown_connections 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036eb0 0x7f5e0c039370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/4035551915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576197f3600 0x5576197f3a20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 >> 172.21.15.63:0/4035551915 conn(0x5576197e76a0 msgr2=0x5576197e7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 shutdown_connections 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/4035551915 wait complete. 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5576197e28c0 msgr2=0x5576197e4cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5576197e28c0 0x5576197e4cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5576196e1740 msgr2=0x5576197df160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5576196e1740 0x5576197df160 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036e60 msgr2=0x7f5e18039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036e60 0x7f5e18039320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f28054000 tx=0x7f5f2804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 msgr2=0x5576197d7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576197d7f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b210 tx=0x7f5f24000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 shutdown_connections 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5576197e28c0 0x5576197e4cb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5576196e1740 0x5576197df160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036e60 0x7f5e18039320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 --2- 172.21.15.63:0/3095997481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576196ddc30 0x5576197d7f20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 >> 172.21.15.63:0/3095997481 conn(0x5576196d7600 msgr2=0x5576196dc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 shutdown_connections 2024-04-23T19:10:25.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.631+0000 7f5f35546800 1 -- 172.21.15.63:0/3095997481 wait complete. 2024-04-23T19:10:25.640 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-23T19:10:25.640 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-23T19:10:25.640 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-23T19:10:25.640 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-23T19:10:25.684 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:25.684 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.684+0000 7fda3eb0c800 1 Processor -- start 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.684+0000 7fda3eb0c800 1 -- start start 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.684+0000 7fda3eb0c800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55dc59cd8c30 0x55dc59cdb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.685+0000 7fda3eb0c800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55dc59b8a7f0 con 0x55dc59cdb560 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.685+0000 7fda3eb0c800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55dc59b79e20 con 0x55dc59cd8c30 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.685+0000 7fda3c612640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55dc59cd8c30 0x55dc59cdb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.685+0000 7fda3be11640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55dc59cdb560 0x55dc59cdd950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52632/0 (socket says 172.21.15.63:52632) 2024-04-23T19:10:25.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.685+0000 7fda3be11640 1 -- 172.21.15.63:0/1244154240 learned_addr learned my addr 172.21.15.63:0/1244154240 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:25.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.685+0000 7fda3c612640 1 -- 172.21.15.63:0/1244154240 >> v1:172.21.15.16:6789/0 conn(0x55dc59cdb560 legacy=0x55dc59cdd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.685+0000 7fda3c612640 1 -- 172.21.15.63:0/1244154240 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc59b98b50 con 0x55dc59cd8c30 2024-04-23T19:10:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.687+0000 7fda3c612640 1 --2- 172.21.15.63:0/1244154240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59cdb020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00a470 tx=0x7fda2c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1497c5a7a1a379 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.687+0000 7fda3b610640 1 -- 172.21.15.63:0/1244154240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda2c057070 con 0x55dc59cd8c30 2024-04-23T19:10:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.687+0000 7fda3b610640 1 -- 172.21.15.63:0/1244154240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c052570 con 0x55dc59cd8c30 2024-04-23T19:10:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.687+0000 7fda3b610640 1 -- 172.21.15.63:0/1244154240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda2c052870 con 0x55dc59cd8c30 2024-04-23T19:10:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.688+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1244154240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 msgr2=0x55dc59cdb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.688+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1244154240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59cdb020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00a470 tx=0x7fda2c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.688+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1244154240 shutdown_connections 2024-04-23T19:10:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.688+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1244154240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59cdb020 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.688+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1244154240 >> 172.21.15.63:0/1244154240 conn(0x55dc59cd2600 msgr2=0x55dc59cd4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.688+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1244154240 shutdown_connections 2024-04-23T19:10:25.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.688+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1244154240 wait complete. 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3eb0c800 1 Processor -- start 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3eb0c800 1 -- start start 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3eb0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dc9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3eb0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc59c236f0 con 0x55dc59cd8c30 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3c612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dc9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3c612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dc9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48002/0 (socket says 172.21.15.63:48002) 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3c612640 1 -- 172.21.15.63:0/3955417417 learned_addr learned my addr 172.21.15.63:0/3955417417 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.701+0000 7fda3c612640 1 -- 172.21.15.63:0/3955417417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc59dc9940 con 0x55dc59cd8c30 2024-04-23T19:10:25.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.702+0000 7fda3c612640 1 --2- 172.21.15.63:0/3955417417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dc9370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda2c055040 tx=0x7fda2c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4bbee7909969ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.702+0000 7fd92a60c640 1 -- 172.21.15.63:0/3955417417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda2c057040 con 0x55dc59cd8c30 2024-04-23T19:10:25.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.702+0000 7fd92a60c640 1 -- 172.21.15.63:0/3955417417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c052990 con 0x55dc59cd8c30 2024-04-23T19:10:25.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.702+0000 7fd92a60c640 1 -- 172.21.15.63:0/3955417417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda2c052c70 con 0x55dc59cd8c30 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3955417417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 msgr2=0x55dc59dc9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/3955417417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dc9370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda2c055040 tx=0x7fda2c04c000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3955417417 shutdown_connections 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/3955417417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dc9370 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3955417417 >> 172.21.15.63:0/3955417417 conn(0x55dc59cd2600 msgr2=0x55dc59dc4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3955417417 shutdown_connections 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3955417417 wait complete. 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 Processor -- start 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 -- start start 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3eb0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc59ce1380 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3c612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3c612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dd0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48016/0 (socket says 172.21.15.63:48016) 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.703+0000 7fda3c612640 1 -- 172.21.15.63:0/1596199944 learned_addr learned my addr 172.21.15.63:0/1596199944 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.704+0000 7fda3c612640 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc59dd1350 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.704+0000 7fda3c612640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dd0e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00a470 tx=0x7fda2c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.704+0000 7fd928e09640 1 -- 172.21.15.63:0/1596199944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda2c073070 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.704+0000 7fd928e09640 1 -- 172.21.15.63:0/1596199944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c05d650 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.704+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc59dd15e0 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.704+0000 7fd928e09640 1 -- 172.21.15.63:0/1596199944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda2c057070 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.704+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc59cd6a20 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.705+0000 7fd928e09640 1 -- 172.21.15.63:0/1596199944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fda2c076070 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.705+0000 7fd928e09640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd92403b2c0 0x7fd92403d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.705+0000 7fd928e09640 1 -- 172.21.15.63:0/1596199944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fda2c08e690 con 0x55dc59cd8c30 2024-04-23T19:10:25.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.705+0000 7fda3be11640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd92403b2c0 0x7fd92403d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.705+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59cdb560 0x55dc59dd62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.705+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59cd8830 con 0x55dc59cdb560 2024-04-23T19:10:25.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.705+0000 7fda3ce13640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59cdb560 0x55dc59dd62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.706+0000 7fda3be11640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd92403b2c0 0x7fd92403d780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda30000f60 tx=0x7fda300012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.706+0000 7fda3ce13640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59cdb560 0x55dc59dd62a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.706+0000 7fda3ce13640 1 -- 172.21.15.63:0/1596199944 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda3400a4a0 con 0x55dc59cdb560 2024-04-23T19:10:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.706+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59cd7e60 con 0x55dc59cdb560 2024-04-23T19:10:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.706+0000 7fda3ce13640 1 -- 172.21.15.63:0/1596199944 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fda3400a4a0 con 0x55dc59cdb560 2024-04-23T19:10:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.706+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59dd8220 con 0x55dc59cdb560 2024-04-23T19:10:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.707+0000 7fda3ce13640 1 -- 172.21.15.63:0/1596199944 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fda3400a4a0 con 0x55dc59cdb560 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.707+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59dd7560 con 0x55dc59cdb560 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.707+0000 7fda3ce13640 1 -- 172.21.15.63:0/1596199944 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fda3400a4a0 con 0x55dc59cdb560 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.707+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc59dd95d0 0x55dc59ddb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.707+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59ddbf40 con 0x55dc59dd95d0 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.707+0000 7fda3c612640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc59dd95d0 0x55dc59ddb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.708+0000 7fda3c612640 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc59dd95d0 0x55dc59ddb9c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.708+0000 7fda3c612640 1 -- 172.21.15.63:0/1596199944 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fda2c075030 con 0x55dc59dd95d0 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3eb0c800 1 Processor -- start 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3eb0c800 1 -- start start 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3eb0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59dea320 0x55dc59dea740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3eb0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc59bc9e30 con 0x55dc59dea320 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3ce13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59dea320 0x55dc59dea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3ce13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59dea320 0x55dc59dea740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48030/0 (socket says 172.21.15.63:48030) 2024-04-23T19:10:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3ce13640 1 -- 172.21.15.63:0/993433135 learned_addr learned my addr 172.21.15.63:0/993433135 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3ce13640 1 -- 172.21.15.63:0/993433135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc59deac80 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.709+0000 7fda3ce13640 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59dea320 0x55dc59dea740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55dc59dd2210 tx=0x7fda340178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.710+0000 7fd921ffb640 1 -- 172.21.15.63:0/993433135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda34013440 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.710+0000 7fd921ffb640 1 -- 172.21.15.63:0/993433135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda340135e0 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.710+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc59deb6f0 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.710+0000 7fd921ffb640 1 -- 172.21.15.63:0/993433135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda340138c0 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.710+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc59debbf0 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.710+0000 7fd921ffb640 1 -- 172.21.15.63:0/993433135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fda34011040 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fd921ffb640 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd914036e60 0x7fd914039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fd921ffb640 1 -- 172.21.15.63:0/993433135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fda3401d070 con 0x55dc59dea320 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3c612640 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd914036e60 0x7fd914039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3eb0c800 1 Processor -- start 2024-04-23T19:10:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3eb0c800 1 -- start start 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3c612640 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd914036e60 0x7fd914039320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda2c08cac0 tx=0x7fda2c08c990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3eb0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59df7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3eb0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc59dc9c00 con 0x55dc59df76f0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3ce13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59df7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3ce13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59df7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48034/0 (socket says 172.21.15.63:48034) 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.711+0000 7fda3ce13640 1 -- 172.21.15.63:0/3258301797 learned_addr learned my addr 172.21.15.63:0/3258301797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3ce13640 1 -- 172.21.15.63:0/3258301797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc59df80e0 con 0x55dc59df76f0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3ce13640 1 --2- 172.21.15.63:0/3258301797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59df7b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fda3404ccb0 tx=0x7fda34012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b4d06c76ee1a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fd91b7fe640 1 -- 172.21.15.63:0/3258301797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda34012df0 con 0x55dc59df76f0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fd91b7fe640 1 -- 172.21.15.63:0/3258301797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda34001310 con 0x55dc59df76f0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fd91b7fe640 1 -- 172.21.15.63:0/3258301797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda34001630 con 0x55dc59df76f0 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3258301797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 msgr2=0x55dc59df7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/3258301797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59df7b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fda3404ccb0 tx=0x7fda34012910 comp rx=0 tx=0).stop 2024-04-23T19:10:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3258301797 shutdown_connections 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/3258301797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59df7b10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3258301797 >> 172.21.15.63:0/3258301797 conn(0x55dc59df39c0 msgr2=0x55dc59df1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3258301797 shutdown_connections 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.712+0000 7fda3eb0c800 1 -- 172.21.15.63:0/3258301797 wait complete. 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.713+0000 7fda3eb0c800 1 Processor -- start 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.713+0000 7fda3eb0c800 1 -- start start 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.713+0000 7fda3eb0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59dfec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.713+0000 7fda3eb0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc59df8870 con 0x55dc59df76f0 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.713+0000 7fda3ce13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59dfec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.713+0000 7fda3ce13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59dfec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48050/0 (socket says 172.21.15.63:48050) 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.713+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 learned_addr learned my addr 172.21.15.63:0/1282496178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.714+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc59dff150 con 0x55dc59df76f0 2024-04-23T19:10:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.714+0000 7fda3ce13640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59dfec10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fda34015ac0 tx=0x7fda3404cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.714+0000 7fd919ffb640 1 -- 172.21.15.63:0/1282496178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda3402ae30 con 0x55dc59df76f0 2024-04-23T19:10:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.714+0000 7fd919ffb640 1 -- 172.21.15.63:0/1282496178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda34021c60 con 0x55dc59df76f0 2024-04-23T19:10:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.714+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc59dff3e0 con 0x55dc59df76f0 2024-04-23T19:10:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.714+0000 7fd919ffb640 1 -- 172.21.15.63:0/1282496178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda3402a350 con 0x55dc59df76f0 2024-04-23T19:10:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.714+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc59e02090 con 0x55dc59df76f0 2024-04-23T19:10:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fd919ffb640 1 -- 172.21.15.63:0/1282496178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fda34040040 con 0x55dc59df76f0 2024-04-23T19:10:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fd919ffb640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd90c036eb0 0x7fd90c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fd919ffb640 1 -- 172.21.15.63:0/1282496178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fda34032070 con 0x55dc59df76f0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fda3c612640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd90c036eb0 0x7fd90c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fda3c612640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd90c036eb0 0x7fd90c039370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55dc59ddc320 tx=0x7fda2c089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59e0a980 0x55dc59e0ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.715+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e0d590 con 0x55dc59e0a980 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3be11640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59e0a980 0x55dc59e0ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3be11640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59e0a980 0x55dc59e0ce40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda30009050 con 0x55dc59e0a980 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 20 realm 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e09dd0 con 0x55dc59e0a980 2024-04-23T19:10:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda30009050 con 0x55dc59e0a980 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.716+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e0e080 con 0x55dc59e0a980 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda30009050 con 0x55dc59e0a980 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e0e950 con 0x55dc59e0a980 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fda30009050 con 0x55dc59e0a980 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e0ecf0 con 0x55dc59e0a980 2024-04-23T19:10:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fda30009050 con 0x55dc59e0a980 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 20 searching for the correct realm 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc59e10bd0 0x55dc59e12fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x55dc59e10bd0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.717+0000 7fda3ce13640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc59e10bd0 0x55dc59e12fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.718+0000 7fda3ce13640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc59e10bd0 0x55dc59e12fc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.718+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.718+0000 7fda3ce13640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fda340123a0 0x7fda340468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.718+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x7fda340123a0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.718+0000 7fda3c612640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fda340123a0 0x7fda340468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.719+0000 7fda3c612640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fda340123a0 0x7fda340468c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.719+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.719+0000 7fda3c612640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fda2c085040 0x7fda2c0773b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.719+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x7fda2c085040 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.719+0000 7fda3be11640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fda2c085040 0x7fda2c0773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.720+0000 7fda3be11640 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fda2c085040 0x7fda2c0773b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.720+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda30009050 con 0x7fda2c085040 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.720+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x55dc59e0a980 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.720+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda30009050 con 0x55dc59e0a980 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.720+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x55dc59e0a980 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.720+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.720+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x7fda2c085040 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x7fda2c085040 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x7fda340123a0 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x55dc59e10bd0 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.721+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x55dc59e0a980 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.722+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.722+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x7fda2c085040 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.722+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.722+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x55dc59e0a980 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.722+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.722+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13670 con 0x7fda340123a0 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e10bd0 2024-04-23T19:10:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e10bd0 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda2c085040 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.723+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e10bd0 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e10bd0 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e0a980 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e0a980 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.724+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda2c085040 2024-04-23T19:10:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e10bd0 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda340123a0 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e0a980 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v23'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.725+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda2c085040 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e10bd0 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda2c085040 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda340123a0 2024-04-23T19:10:25.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.726+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e10bd0 2024-04-23T19:10:25.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda2c085040 2024-04-23T19:10:25.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x55dc59e0a980 2024-04-23T19:10:25.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fda2c0a7060 con 0x7fda340123a0 2024-04-23T19:10:25.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:25.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.727+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e0f670 con 0x55dc59e0a980 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e0ff10 con 0x55dc59e0a980 2024-04-23T19:10:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13af0 con 0x7fda2c085040 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 zone default found 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 4 Realm: () 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 zonegroup default 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.728+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e13fa0 con 0x7fda2c085040 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e15980 con 0x55dc59e10bd0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e15e70 con 0x55dc59e10bd0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 20 started sync module instance, tier type = 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.729+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e14dc0 con 0x7fda340123a0 2024-04-23T19:10:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.730+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.730+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94404889102608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e162d0 con 0x7fda340123a0 2024-04-23T19:10:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.730+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e16b20 con 0x55dc59e0a980 2024-04-23T19:10:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.731+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94404889102608] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.732+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.732+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94404889110128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e16f40 con 0x55dc59e0a980 2024-04-23T19:10:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.732+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e177f0 con 0x55dc59e10bd0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.733+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94404889110128] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.733+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.733+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94404889111184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e17360 con 0x55dc59e10bd0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.733+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e182b0 con 0x7fda340123a0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.734+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94404889111184] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.734+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.734+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94404889112240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e172d0 con 0x7fda340123a0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.734+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e19170 con 0x55dc59e0a980 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.735+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94404889112240] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.738+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.738+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94404889119936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e19590 con 0x55dc59e0a980 2024-04-23T19:10:25.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.738+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e19f50 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.739+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.739+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94404889123488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e19ec0 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.739+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1aec0 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.741+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(56 notify.6 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.741+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94404889127440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1b2e0 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.741+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1be00 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.741+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94404889123488] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.741+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94404889119936] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94404889127440] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94404889120992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1a2f0 con 0x7fda2c085040 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 20 add_watcher() i=0 2024-04-23T19:10:25.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 20 add_watcher() i=1 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 20 add_watcher() i=2 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 20 add_watcher() i=3 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 20 add_watcher() i=4 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 20 add_watcher() i=5 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.742+0000 7fda3eb0c800 20 add_watcher() i=6 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.744+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94404889120992] v30'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.744+0000 7fda3eb0c800 20 add_watcher() i=7 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.744+0000 7fda3eb0c800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.744+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1ae30 con 0x55dc59e0a980 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.744+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.744+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94404889128496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1b280 con 0x55dc59e0a980 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.747+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94404889128496] v30'41 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.747+0000 7fd8f27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.747+0000 7fda3eb0c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:25.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.747+0000 7fda3eb0c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:25.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.747+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e27110 con 0x55dc59e0a980 2024-04-23T19:10:25.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.748+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.748+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e27760 con 0x7fda2c085040 2024-04-23T19:10:25.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.749+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.749+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e27bc0 con 0x55dc59e10bd0 2024-04-23T19:10:25.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.751+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.751+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e27af0 con 0x55dc59e0a980 2024-04-23T19:10:25.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.752+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'42 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.752+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e280c0 con 0x7fda340123a0 2024-04-23T19:10:25.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.753+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'29 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.753+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e28570 con 0x55dc59e10bd0 2024-04-23T19:10:25.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.754+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'19 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.754+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e28570 con 0x55dc59e0a980 2024-04-23T19:10:25.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.756+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.756+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e28570 con 0x7fda2c085040 2024-04-23T19:10:25.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.757+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'19 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.757+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e28570 con 0x7fda340123a0 2024-04-23T19:10:25.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.758+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.758+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e24ea0 con 0x55dc59e10bd0 2024-04-23T19:10:25.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.759+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'19 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.759+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e24ea0 con 0x7fda340123a0 2024-04-23T19:10:25.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.761+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'30 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.761+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e24fd0 con 0x7fda2c085040 2024-04-23T19:10:25.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.762+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'15 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.762+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e24fd0 con 0x7fda340123a0 2024-04-23T19:10:25.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.764+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.764+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e24fd0 con 0x7fda2c085040 2024-04-23T19:10:25.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.767+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'25 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.767+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e24fd0 con 0x55dc59e0a980 2024-04-23T19:10:25.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.770+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'43 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.770+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e24fd0 con 0x55dc59e10bd0 2024-04-23T19:10:25.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.773+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'23 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.773+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x7fda340123a0 2024-04-23T19:10:25.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.775+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'31 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.775+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.778+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.778+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e0a980 2024-04-23T19:10:25.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.779+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.779+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e0a980 2024-04-23T19:10:25.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.781+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'44 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.781+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.784+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.784+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x7fda340123a0 2024-04-23T19:10:25.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.785+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.785+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.786+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.786+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.789+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.789+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.791+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'9 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.791+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.792+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'20 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.792+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x7fda2c085040 2024-04-23T19:10:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.794+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.794+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.795+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.795+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x7fda2c085040 2024-04-23T19:10:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.796+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'20 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.796+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x55dc59e10bd0 2024-04-23T19:10:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.796+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'25 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.796+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x7fda2c085040 2024-04-23T19:10:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.799+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'26 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.799+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e251a0 con 0x7fda340123a0 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fd8f17e2640 20 reqs_thread_entry: start 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fda3eb0c800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fda3eb0c800 20 Filter name: none 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fd862ffd640 20 reqs_thread_entry: start 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fda3eb0c800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fda3eb0c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.800+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e53db0 con 0x55dc59e0a980 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.801+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(94 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.801+0000 7fda3eb0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:25.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.801+0000 7fda3eb0c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-23T19:10:25.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.801+0000 7fda3eb0c800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:25.802 INFO:teuthology.orchestra.run.smithi063.stderr:could not remove user: unable to remove user, user does not exist 2024-04-23T19:10:25.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.802+0000 7fd8f27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:25.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.802+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94404889128496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59dc0c90 con 0x55dc59e0a980 2024-04-23T19:10:25.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.803+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94404889128496] v30'45 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.803+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94404889102608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1ae30 con 0x7fda340123a0 2024-04-23T19:10:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.806+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94404889102608] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.806+0000 7fda3eb0c800 20 remove_watcher() i=0 2024-04-23T19:10:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.806+0000 7fda3eb0c800 2 removed watcher, disabling cache 2024-04-23T19:10:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.806+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94404889110128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e14910 con 0x55dc59e0a980 2024-04-23T19:10:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.807+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94404889110128] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.807+0000 7fda3eb0c800 20 remove_watcher() i=1 2024-04-23T19:10:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.807+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94404889111184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e14910 con 0x55dc59e10bd0 2024-04-23T19:10:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.808+0000 7fda3ce13640 1 -- 172.21.15.63:0/1282496178 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94404889111184] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda34029070 con 0x55dc59e10bd0 2024-04-23T19:10:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.808+0000 7fda3eb0c800 20 remove_watcher() i=2 2024-04-23T19:10:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.808+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94404889112240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e14910 con 0x7fda340123a0 2024-04-23T19:10:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.809+0000 7fda3c612640 1 -- 172.21.15.63:0/1282496178 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94404889112240] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda2c097070 con 0x7fda340123a0 2024-04-23T19:10:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.809+0000 7fda3eb0c800 20 remove_watcher() i=3 2024-04-23T19:10:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.809+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94404889119936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e14910 con 0x55dc59e0a980 2024-04-23T19:10:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.811+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94404889119936] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x55dc59e0a980 2024-04-23T19:10:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.811+0000 7fda3eb0c800 20 remove_watcher() i=4 2024-04-23T19:10:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.811+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94404889123488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e14910 con 0x7fda2c085040 2024-04-23T19:10:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.812+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94404889123488] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.812+0000 7fda3eb0c800 20 remove_watcher() i=5 2024-04-23T19:10:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.812+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94404889127440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e19aa0 con 0x7fda2c085040 2024-04-23T19:10:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.815+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94404889127440] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.815+0000 7fda3eb0c800 20 remove_watcher() i=6 2024-04-23T19:10:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.815+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94404889120992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc59e1aec0 con 0x7fda2c085040 2024-04-23T19:10:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94404889120992] v30'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda3002a040 con 0x7fda2c085040 2024-04-23T19:10:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 20 remove_watcher() i=7 2024-04-23T19:10:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fda2c085040 msgr2=0x7fda2c0773b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fda2c085040 0x7fda2c0773b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59e0a980 msgr2=0x55dc59e0ce40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59e0a980 0x55dc59e0ce40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fda340123a0 msgr2=0x7fda340468c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fda340123a0 0x7fda340468c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc59e10bd0 msgr2=0x55dc59e12fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc59e10bd0 0x55dc59e12fc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd90c036eb0 msgr2=0x7fd90c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd90c036eb0 0x7fd90c039370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55dc59ddc320 tx=0x7fda2c089000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 msgr2=0x55dc59dfec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1282496178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59df76f0 0x55dc59dfec10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fda34015ac0 tx=0x7fda3404cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 reap_dead start 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.816+0000 7fda3be11640 1 -- 172.21.15.63:0/1282496178 reap_dead start 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 shutdown_connections 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 >> 172.21.15.63:0/1282496178 conn(0x55dc59df39c0 msgr2=0x55dc59df1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 shutdown_connections 2024-04-23T19:10:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1282496178 wait complete. 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd914036e60 msgr2=0x7fd914039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd914036e60 0x7fd914039320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda2c08cac0 tx=0x7fda2c08c990 comp rx=0 tx=0).stop 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59dea320 msgr2=0x55dc59dea740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59dea320 0x55dc59dea740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55dc59dd2210 tx=0x7fda340178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 shutdown_connections 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd914036e60 0x7fd914039320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/993433135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59dea320 0x55dc59dea740 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 >> 172.21.15.63:0/993433135 conn(0x55dc59dde3c0 msgr2=0x55dc59dde7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 shutdown_connections 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/993433135 wait complete. 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc59dd95d0 msgr2=0x55dc59ddb9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc59dd95d0 0x55dc59ddb9c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59cdb560 msgr2=0x55dc59dd62a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59cdb560 0x55dc59dd62a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd92403b2c0 msgr2=0x7fd92403d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd92403b2c0 0x7fd92403d780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda30000f60 tx=0x7fda300012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 msgr2=0x55dc59dd0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.817+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dd0e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00a470 tx=0x7fda2c003020 comp rx=0 tx=0).stop 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 shutdown_connections 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc59dd95d0 0x55dc59ddb9c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc59cdb560 0x55dc59dd62a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd92403b2c0 0x7fd92403d780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 --2- 172.21.15.63:0/1596199944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc59cd8c30 0x55dc59dd0e10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 >> 172.21.15.63:0/1596199944 conn(0x55dc59cd2600 msgr2=0x55dc59dc3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 shutdown_connections 2024-04-23T19:10:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.818+0000 7fda3eb0c800 1 -- 172.21.15.63:0/1596199944 wait complete. 2024-04-23T19:10:25.826 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-23T19:10:25.826 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-23T19:10:25.826 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-23T19:10:25.826 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-23T19:10:25.872 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:25.873 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.873+0000 7f619e862800 1 Processor -- start 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.873+0000 7f619e862800 1 -- start start 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.873+0000 7f619e862800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55727dcca4a0 0x55727dcca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.873+0000 7f619e862800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55727db7e7f0 con 0x55727dccd490 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.873+0000 7f619e862800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55727db6de20 con 0x55727dcca4a0 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.874+0000 7f619c361640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55727dcca4a0 0x55727dcca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.874+0000 7f619bb60640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55727dccd490 0x55727dccd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52640/0 (socket says 172.21.15.63:52640) 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.874+0000 7f619bb60640 1 -- 172.21.15.63:0/2193199444 learned_addr learned my addr 172.21.15.63:0/2193199444 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:25.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.874+0000 7f619c361640 1 -- 172.21.15.63:0/2193199444 >> v1:172.21.15.16:6789/0 conn(0x55727dccd490 legacy=0x55727dccd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.874+0000 7f619c361640 1 -- 172.21.15.63:0/2193199444 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727db8cb50 con 0x55727dcca4a0 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619c361640 1 --2- 172.21.15.63:0/2193199444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcca8c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f618c00a470 tx=0x7f618c01ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1fc072e0abec86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619b35f640 1 -- 172.21.15.63:0/2193199444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c057070 con 0x55727dcca4a0 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619b35f640 1 -- 172.21.15.63:0/2193199444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c04ad80 con 0x55727dcca4a0 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619b35f640 1 -- 172.21.15.63:0/2193199444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c052590 con 0x55727dcca4a0 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619e862800 1 -- 172.21.15.63:0/2193199444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 msgr2=0x55727dcca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619e862800 1 --2- 172.21.15.63:0/2193199444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcca8c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f618c00a470 tx=0x7f618c01ce50 comp rx=0 tx=0).stop 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619e862800 1 -- 172.21.15.63:0/2193199444 shutdown_connections 2024-04-23T19:10:25.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619e862800 1 --2- 172.21.15.63:0/2193199444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcca8c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619e862800 1 -- 172.21.15.63:0/2193199444 >> 172.21.15.63:0/2193199444 conn(0x55727dcc5e30 msgr2=0x55727dcc8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619e862800 1 -- 172.21.15.63:0/2193199444 shutdown_connections 2024-04-23T19:10:25.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.876+0000 7f619e862800 1 -- 172.21.15.63:0/2193199444 wait complete. 2024-04-23T19:10:25.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.889+0000 7f619e862800 1 Processor -- start 2024-04-23T19:10:25.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.889+0000 7f619e862800 1 -- start start 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.889+0000 7f619e862800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.889+0000 7f619e862800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55727dc17710 con 0x55727dcca4a0 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f619c361640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f619c361640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48064/0 (socket says 172.21.15.63:48064) 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f619c361640 1 -- 172.21.15.63:0/2550153804 learned_addr learned my addr 172.21.15.63:0/2550153804 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f619c361640 1 -- 172.21.15.63:0/2550153804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727dccb390 con 0x55727dcca4a0 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f619c361640 1 --2- 172.21.15.63:0/2550153804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcccdd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f618c055040 tx=0x7f618c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b7498f11a2dc0b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f608a60c640 1 -- 172.21.15.63:0/2550153804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c057040 con 0x55727dcca4a0 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f608a60c640 1 -- 172.21.15.63:0/2550153804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c063070 con 0x55727dcca4a0 2024-04-23T19:10:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.890+0000 7f608a60c640 1 -- 172.21.15.63:0/2550153804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c0527e0 con 0x55727dcca4a0 2024-04-23T19:10:25.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 -- 172.21.15.63:0/2550153804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 msgr2=0x55727dcccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 --2- 172.21.15.63:0/2550153804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcccdd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f618c055040 tx=0x7f618c04c000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 -- 172.21.15.63:0/2550153804 shutdown_connections 2024-04-23T19:10:25.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 --2- 172.21.15.63:0/2550153804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727dcccdd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 -- 172.21.15.63:0/2550153804 >> 172.21.15.63:0/2550153804 conn(0x55727dcc5e30 msgr2=0x55727ddb92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 -- 172.21.15.63:0/2550153804 shutdown_connections 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 -- 172.21.15.63:0/2550153804 wait complete. 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 Processor -- start 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.891+0000 7f619e862800 1 -- start start 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619e862800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727ddc7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619e862800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55727dcd5990 con 0x55727dcca4a0 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619c361640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727ddc7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619c361640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727ddc7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48080/0 (socket says 172.21.15.63:48080) 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619c361640 1 -- 172.21.15.63:0/3068909525 learned_addr learned my addr 172.21.15.63:0/3068909525 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619c361640 1 -- 172.21.15.63:0/3068909525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727ddc7b50 con 0x55727dcca4a0 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619c361640 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727ddc7610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f618c060040 tx=0x7f618c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f6088e09640 1 -- 172.21.15.63:0/3068909525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c057050 con 0x55727dcca4a0 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55727ddc7de0 con 0x55727dcca4a0 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f6088e09640 1 -- 172.21.15.63:0/3068909525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c052ce0 con 0x55727dcca4a0 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f6088e09640 1 -- 172.21.15.63:0/3068909525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c05d470 con 0x55727dcca4a0 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.892+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727ddcaa90 con 0x55727dcca4a0 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f619e862800 1 Processor -- start 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f619e862800 1 -- start start 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f619e862800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727ddd5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f6088e09640 1 -- 172.21.15.63:0/3068909525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f618c05d660 con 0x55727dcca4a0 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f619e862800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55727dbbde30 con 0x55727ddd4f20 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f6088e09640 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6084036e60 0x7f6084039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f619cb62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727ddd5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f619cb62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727ddd5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48082/0 (socket says 172.21.15.63:48082) 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f619cb62640 1 -- 172.21.15.63:0/2944357979 learned_addr learned my addr 172.21.15.63:0/2944357979 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.893+0000 7f6088e09640 1 -- 172.21.15.63:0/3068909525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f618c08e2d0 con 0x55727dcca4a0 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f619c361640 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6084036e60 0x7f6084039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f619cb62640 1 -- 172.21.15.63:0/2944357979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727ddd5880 con 0x55727ddd4f20 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f619cb62640 1 --2- 172.21.15.63:0/2944357979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727ddd5340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f619400a430 tx=0x7f619400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9b3f480c42d2d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f619c361640 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6084036e60 0x7f6084039320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f618c066fd0 tx=0x7f618c08e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f60827fc640 1 -- 172.21.15.63:0/2944357979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6194015070 con 0x55727ddd4f20 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f60827fc640 1 -- 172.21.15.63:0/2944357979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6194001670 con 0x55727ddd4f20 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f60827fc640 1 -- 172.21.15.63:0/2944357979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6194001990 con 0x55727ddd4f20 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f619e862800 1 -- 172.21.15.63:0/2944357979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 msgr2=0x55727ddd5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.894+0000 7f619e862800 1 --2- 172.21.15.63:0/2944357979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727ddd5340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f619400a430 tx=0x7f619400a8f0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 -- 172.21.15.63:0/2944357979 shutdown_connections 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 --2- 172.21.15.63:0/2944357979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727ddd5340 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 -- 172.21.15.63:0/2944357979 >> 172.21.15.63:0/2944357979 conn(0x55727dccd490 msgr2=0x55727ddd19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 -- 172.21.15.63:0/2944357979 shutdown_connections 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 -- 172.21.15.63:0/2944357979 wait complete. 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 Processor -- start 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 -- start start 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727dddc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619e862800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55727ddd6210 con 0x55727ddd4f20 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619cb62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727dddc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619cb62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727dddc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48086/0 (socket says 172.21.15.63:48086) 2024-04-23T19:10:25.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.895+0000 7f619cb62640 1 -- 172.21.15.63:0/146358169 learned_addr learned my addr 172.21.15.63:0/146358169 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.896+0000 7f619cb62640 1 -- 172.21.15.63:0/146358169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727dddc970 con 0x55727ddd4f20 2024-04-23T19:10:25.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.896+0000 7f619cb62640 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727dddc430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f619400a930 tx=0x7f6194003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.896+0000 7f6080ff9640 1 -- 172.21.15.63:0/146358169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6194020020 con 0x55727ddd4f20 2024-04-23T19:10:25.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.896+0000 7f6080ff9640 1 -- 172.21.15.63:0/146358169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61940108b0 con 0x55727ddd4f20 2024-04-23T19:10:25.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.896+0000 7f6080ff9640 1 -- 172.21.15.63:0/146358169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6194015040 con 0x55727ddd4f20 2024-04-23T19:10:25.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.896+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55727dddcc00 con 0x55727ddd4f20 2024-04-23T19:10:25.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.896+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727dddf8b0 con 0x55727ddd4f20 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.897+0000 7f6080ff9640 1 -- 172.21.15.63:0/146358169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6194023020 con 0x55727ddd4f20 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.897+0000 7f6080ff9640 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6074036e60 0x7f6074039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.897+0000 7f6080ff9640 1 -- 172.21.15.63:0/146358169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f619404c6f0 con 0x55727ddd4f20 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.897+0000 7f619bb60640 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6074036e60 0x7f6074039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.897+0000 7f619e862800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.897+0000 7f619e862800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.897+0000 7f619e862800 1 Processor -- start 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619bb60640 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6074036e60 0x7f6074039320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55727ddd0e40 tx=0x7f6190001270 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619e862800 1 -- start start 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619e862800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddf6b40 0x55727ddf6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619e862800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55727dccb650 con 0x55727ddf6b40 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619c361640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddf6b40 0x55727ddf6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619c361640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddf6b40 0x55727ddf6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48100/0 (socket says 172.21.15.63:48100) 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619c361640 1 -- 172.21.15.63:0/4148744019 learned_addr learned my addr 172.21.15.63:0/4148744019 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619c361640 1 -- 172.21.15.63:0/4148744019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727ddf74a0 con 0x55727ddf6b40 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.898+0000 7f619c361640 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddf6b40 0x55727ddf6f60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f618c00a470 tx=0x7f618c05ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.899+0000 7f6060fe1640 1 -- 172.21.15.63:0/4148744019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c08a020 con 0x55727ddf6b40 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.899+0000 7f6060fe1640 1 -- 172.21.15.63:0/4148744019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c080040 con 0x55727ddf6b40 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.899+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55727ddf7f10 con 0x55727ddf6b40 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.899+0000 7f6060fe1640 1 -- 172.21.15.63:0/4148744019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c07f070 con 0x55727ddf6b40 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.899+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727ddf8410 con 0x55727ddf6b40 2024-04-23T19:10:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.899+0000 7f6060fe1640 1 -- 172.21.15.63:0/4148744019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f618c05c3d0 con 0x55727ddf6b40 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f6060fe1640 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5fdc036eb0 0x7f5fdc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f6060fe1640 1 -- 172.21.15.63:0/4148744019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f618c0a3170 con 0x55727ddf6b40 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f619bb60640 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5fdc036eb0 0x7f5fdc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f600a7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f619e862800 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55727de04600 0x55727de06aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55727de07170 con 0x55727de04600 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f619bb60640 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5fdc036eb0 0x7f5fdc039370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6190004170 tx=0x7f619000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f619cb62640 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55727de04600 0x55727de06aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.900+0000 7f619cb62640 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55727de04600 0x55727de06aa0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.901+0000 7f619cb62640 1 -- 172.21.15.63:0/3068909525 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f619400e070 con 0x55727de04600 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.901+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55727de08ad0 con 0x55727de04600 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.901+0000 7f619cb62640 1 -- 172.21.15.63:0/3068909525 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f619400e070 con 0x55727de04600 2024-04-23T19:10:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.901+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55727de0d640 con 0x55727de04600 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.902+0000 7f619cb62640 1 -- 172.21.15.63:0/3068909525 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [stat,call,call,writefull 0~993] v30'2 uv2 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7f619400e070 con 0x55727de04600 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6074036e60 msgr2=0x7f6074039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6074036e60 0x7f6074039320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55727ddd0e40 tx=0x7f6190001270 comp rx=0 tx=0).stop 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 msgr2=0x55727dddc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727dddc430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f619400a930 tx=0x7f6194003370 comp rx=0 tx=0).stop 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 shutdown_connections 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6074036e60 0x7f6074039320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 --2- 172.21.15.63:0/146358169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddd4f20 0x55727dddc430 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 >> 172.21.15.63:0/146358169 conn(0x55727dccd490 msgr2=0x55727ddcf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 shutdown_connections 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.903+0000 7f619e862800 1 -- 172.21.15.63:0/146358169 wait complete. 2024-04-23T19:10:25.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5fdc036eb0 msgr2=0x7f5fdc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5fdc036eb0 0x7f5fdc039370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6190004170 tx=0x7f619000f000 comp rx=0 tx=0).stop 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddf6b40 msgr2=0x55727ddf6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddf6b40 0x55727ddf6f60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f618c00a470 tx=0x7f618c05ee90 comp rx=0 tx=0).stop 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 shutdown_connections 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5fdc036eb0 0x7f5fdc039370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/4148744019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727ddf6b40 0x55727ddf6f60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 >> 172.21.15.63:0/4148744019 conn(0x55727ddeb5d0 msgr2=0x55727ddebff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 shutdown_connections 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/4148744019 wait complete. 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55727de04600 msgr2=0x55727de06aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55727de04600 0x55727de06aa0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6084036e60 msgr2=0x7f6084039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6084036e60 0x7f6084039320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f618c066fd0 tx=0x7f618c08e040 comp rx=0 tx=0).stop 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 msgr2=0x55727ddc7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727ddc7610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f618c060040 tx=0x7f618c002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 shutdown_connections 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55727de04600 0x55727de06aa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6084036e60 0x7f6084039320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 --2- 172.21.15.63:0/3068909525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55727dcca4a0 0x55727ddc7610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 >> 172.21.15.63:0/3068909525 conn(0x55727dcc5e30 msgr2=0x55727dccbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 shutdown_connections 2024-04-23T19:10:25.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.904+0000 7f619e862800 1 -- 172.21.15.63:0/3068909525 wait complete. 2024-04-23T19:10:25.913 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-23T19:10:25.913 DEBUG:tasks.util.rgw: json result: {'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-23T19:10:25.915 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-23T19:10:25.915 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-23T19:10:25.915 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-23T19:10:25.957 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:25.957 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.957+0000 7f7796919800 1 Processor -- start 2024-04-23T19:10:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.957+0000 7f7796919800 1 -- start start 2024-04-23T19:10:25.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f7796919800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55dc80791880 0x55dc80825ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f7796919800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55dc806cf7f0 con 0x55dc80792c10 2024-04-23T19:10:25.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f7796919800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55dc806bee20 con 0x55dc80791880 2024-04-23T19:10:25.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f7793c1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55dc80791880 0x55dc80825ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f779441f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55dc80792c10 0x55dc80791170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52652/0 (socket says 172.21.15.63:52652) 2024-04-23T19:10:25.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f779441f640 1 -- 172.21.15.63:0/1270583216 learned_addr learned my addr 172.21.15.63:0/1270583216 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:25.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f7793c1e640 1 -- 172.21.15.63:0/1270583216 >> v1:172.21.15.16:6789/0 conn(0x55dc80792c10 legacy=0x55dc80791170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.958+0000 7f7793c1e640 1 -- 172.21.15.63:0/1270583216 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc806ddb50 con 0x55dc80791880 2024-04-23T19:10:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.960+0000 7f7793c1e640 1 --2- 172.21.15.63:0/1270583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80825ff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f778800a470 tx=0x7f778801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a7073510f8dd37 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.960+0000 7f779341d640 1 -- 172.21.15.63:0/1270583216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7788057070 con 0x55dc80791880 2024-04-23T19:10:25.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.960+0000 7f779341d640 1 -- 172.21.15.63:0/1270583216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778804adc0 con 0x55dc80791880 2024-04-23T19:10:25.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.960+0000 7f779341d640 1 -- 172.21.15.63:0/1270583216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7788052590 con 0x55dc80791880 2024-04-23T19:10:25.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.961+0000 7f7796919800 1 -- 172.21.15.63:0/1270583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 msgr2=0x55dc80825ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.961+0000 7f7796919800 1 --2- 172.21.15.63:0/1270583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80825ff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f778800a470 tx=0x7f778801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:25.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.961+0000 7f7796919800 1 -- 172.21.15.63:0/1270583216 shutdown_connections 2024-04-23T19:10:25.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.961+0000 7f7796919800 1 --2- 172.21.15.63:0/1270583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80825ff0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.961+0000 7f7796919800 1 -- 172.21.15.63:0/1270583216 >> 172.21.15.63:0/1270583216 conn(0x55dc808187d0 msgr2=0x55dc8081abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.961+0000 7f7796919800 1 -- 172.21.15.63:0/1270583216 shutdown_connections 2024-04-23T19:10:25.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.961+0000 7f7796919800 1 -- 172.21.15.63:0/1270583216 wait complete. 2024-04-23T19:10:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f7796919800 1 Processor -- start 2024-04-23T19:10:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f7796919800 1 -- start start 2024-04-23T19:10:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f7796919800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80910810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f7796919800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc807686e0 con 0x55dc80791880 2024-04-23T19:10:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f779441f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80910810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f779441f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80910810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48126/0 (socket says 172.21.15.63:48126) 2024-04-23T19:10:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f779441f640 1 -- 172.21.15.63:0/1648514896 learned_addr learned my addr 172.21.15.63:0/1648514896 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.974+0000 7f779441f640 1 -- 172.21.15.63:0/1648514896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc80910de0 con 0x55dc80791880 2024-04-23T19:10:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f779441f640 1 --2- 172.21.15.63:0/1648514896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80910810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7784001dd0 tx=0x7f778400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b470a9e7cf8ed6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f768260c640 1 -- 172.21.15.63:0/1648514896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7784012020 con 0x55dc80791880 2024-04-23T19:10:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f768260c640 1 -- 172.21.15.63:0/1648514896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778400e5a0 con 0x55dc80791880 2024-04-23T19:10:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f768260c640 1 -- 172.21.15.63:0/1648514896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778400e900 con 0x55dc80791880 2024-04-23T19:10:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f7796919800 1 -- 172.21.15.63:0/1648514896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 msgr2=0x55dc80910810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f7796919800 1 --2- 172.21.15.63:0/1648514896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80910810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7784001dd0 tx=0x7f778400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f7796919800 1 -- 172.21.15.63:0/1648514896 shutdown_connections 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f7796919800 1 --2- 172.21.15.63:0/1648514896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80910810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f7796919800 1 -- 172.21.15.63:0/1648514896 >> 172.21.15.63:0/1648514896 conn(0x55dc808187d0 msgr2=0x55dc8090c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f7796919800 1 -- 172.21.15.63:0/1648514896 shutdown_connections 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.975+0000 7f7796919800 1 -- 172.21.15.63:0/1648514896 wait complete. 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f7796919800 1 Processor -- start 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f7796919800 1 -- start start 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f7796919800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80918380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f7796919800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc80828840 con 0x55dc80791880 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f779441f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80918380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f779441f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80918380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48142/0 (socket says 172.21.15.63:48142) 2024-04-23T19:10:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f779441f640 1 -- 172.21.15.63:0/3916437119 learned_addr learned my addr 172.21.15.63:0/3916437119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.976+0000 7f779441f640 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc809188c0 con 0x55dc80791880 2024-04-23T19:10:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.977+0000 7f779441f640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80918380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7784001dd0 tx=0x7f7784000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.977+0000 7f7680e09640 1 -- 172.21.15.63:0/3916437119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7784012030 con 0x55dc80791880 2024-04-23T19:10:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.977+0000 7f7680e09640 1 -- 172.21.15.63:0/3916437119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778401a7a0 con 0x55dc80791880 2024-04-23T19:10:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.977+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc80918b50 con 0x55dc80791880 2024-04-23T19:10:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.977+0000 7f7680e09640 1 -- 172.21.15.63:0/3916437119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778401a990 con 0x55dc80791880 2024-04-23T19:10:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.977+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8091b800 con 0x55dc80791880 2024-04-23T19:10:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7680e09640 1 -- 172.21.15.63:0/3916437119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7784015020 con 0x55dc80791880 2024-04-23T19:10:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7680e09640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f767c036eb0 0x7f767c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7680e09640 1 -- 172.21.15.63:0/3916437119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7784020030 con 0x55dc80791880 2024-04-23T19:10:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7793c1e640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f767c036eb0 0x7f767c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80792c10 0x55dc8091f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8091fb00 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7794c20640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80792c10 0x55dc8091f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7793c1e640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f767c036eb0 0x7f767c039370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f778800a470 tx=0x7f778804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.978+0000 7f7794c20640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80792c10 0x55dc8091f5c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.979+0000 7f7794c20640 1 -- 172.21.15.63:0/3916437119 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f778c00a4a0 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.979+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8091cb90 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.979+0000 7f7794c20640 1 -- 172.21.15.63:0/3916437119 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f778c00a4a0 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.979+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80921920 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.979+0000 7f7794c20640 1 -- 172.21.15.63:0/3916437119 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f778c00a4a0 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.979+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80920c60 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.980+0000 7f7794c20640 1 -- 172.21.15.63:0/3916437119 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f778c00a4a0 con 0x55dc80792c10 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.980+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc80922cd0 0x55dc809250c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.980+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80925680 con 0x55dc80922cd0 2024-04-23T19:10:25.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.980+0000 7f779441f640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc80922cd0 0x55dc809250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.980+0000 7f779441f640 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc80922cd0 0x55dc809250c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.980+0000 7f779441f640 1 -- 172.21.15.63:0/3916437119 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f778400d3d0 con 0x55dc80922cd0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7796919800 1 Processor -- start 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7796919800 1 -- start start 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7796919800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80933af0 0x55dc80933f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7796919800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc8070ee30 con 0x55dc80933af0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7794c20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80933af0 0x55dc80933f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7794c20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80933af0 0x55dc80933f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48146/0 (socket says 172.21.15.63:48146) 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7794c20640 1 -- 172.21.15.63:0/3562104958 learned_addr learned my addr 172.21.15.63:0/3562104958 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.981+0000 7f7794c20640 1 -- 172.21.15.63:0/3562104958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc80934450 con 0x55dc80933af0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.982+0000 7f7794c20640 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80933af0 0x55dc80933f10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55dc80919780 tx=0x7f778c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.982+0000 7f7679ffb640 1 -- 172.21.15.63:0/3562104958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778c013440 con 0x55dc80933af0 2024-04-23T19:10:25.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.982+0000 7f7679ffb640 1 -- 172.21.15.63:0/3562104958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778c0135e0 con 0x55dc80933af0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.982+0000 7f7679ffb640 1 -- 172.21.15.63:0/3562104958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778c0138c0 con 0x55dc80933af0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.982+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc80934ec0 con 0x55dc80933af0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.982+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc809353c0 con 0x55dc80933af0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7679ffb640 1 -- 172.21.15.63:0/3562104958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f778c011040 con 0x55dc80933af0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7679ffb640 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f766c036e60 0x7f766c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7679ffb640 1 -- 172.21.15.63:0/3562104958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f778c01d070 con 0x55dc80933af0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f779441f640 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f766c036e60 0x7f766c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7796919800 1 Processor -- start 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7796919800 1 -- start start 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f779441f640 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f766c036e60 0x7f766c039320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f778400d6e0 tx=0x7f778400dc60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7796919800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc809411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7796919800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc80911160 con 0x55dc80940dd0 2024-04-23T19:10:25.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7794c20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc809411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7794c20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc809411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48152/0 (socket says 172.21.15.63:48152) 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.983+0000 7f7794c20640 1 -- 172.21.15.63:0/2513536760 learned_addr learned my addr 172.21.15.63:0/2513536760 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f7794c20640 1 -- 172.21.15.63:0/2513536760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc809417c0 con 0x55dc80940dd0 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f7794c20640 1 --2- 172.21.15.63:0/2513536760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc809411f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f778c04cca0 tx=0x7f778c012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383ecfc5667adc95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f76737fe640 1 -- 172.21.15.63:0/2513536760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778c042030 con 0x55dc80940dd0 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f76737fe640 1 -- 172.21.15.63:0/2513536760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778c001310 con 0x55dc80940dd0 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f76737fe640 1 -- 172.21.15.63:0/2513536760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778c027040 con 0x55dc80940dd0 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f7796919800 1 -- 172.21.15.63:0/2513536760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 msgr2=0x55dc809411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f7796919800 1 --2- 172.21.15.63:0/2513536760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc809411f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f778c04cca0 tx=0x7f778c012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f7796919800 1 -- 172.21.15.63:0/2513536760 shutdown_connections 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f7796919800 1 --2- 172.21.15.63:0/2513536760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc809411f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.984+0000 7f7796919800 1 -- 172.21.15.63:0/2513536760 >> 172.21.15.63:0/2513536760 conn(0x55dc8093d130 msgr2=0x55dc8093b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:25.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7796919800 1 -- 172.21.15.63:0/2513536760 shutdown_connections 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7796919800 1 -- 172.21.15.63:0/2513536760 wait complete. 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7796919800 1 Processor -- start 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7796919800 1 -- start start 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7796919800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc80948330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7796919800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc80941ed0 con 0x55dc80940dd0 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7794c20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc80948330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7794c20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc80948330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48166/0 (socket says 172.21.15.63:48166) 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 learned_addr learned my addr 172.21.15.63:0/2792573326 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.985+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc80948870 con 0x55dc80940dd0 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.986+0000 7f7794c20640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc80948330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f778c001df0 tx=0x7f778c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.986+0000 7f7671ffb640 1 -- 172.21.15.63:0/2792573326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778c027030 con 0x55dc80940dd0 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.986+0000 7f7671ffb640 1 -- 172.21.15.63:0/2792573326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778c02a350 con 0x55dc80940dd0 2024-04-23T19:10:25.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.986+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc80948b00 con 0x55dc80940dd0 2024-04-23T19:10:25.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.986+0000 7f7671ffb640 1 -- 172.21.15.63:0/2792573326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f778c02a650 con 0x55dc80940dd0 2024-04-23T19:10:25.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.986+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8094b7b0 con 0x55dc80940dd0 2024-04-23T19:10:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7671ffb640 1 -- 172.21.15.63:0/2792573326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f778c045070 con 0x55dc80940dd0 2024-04-23T19:10:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7671ffb640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7664036eb0 0x7f7664039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7671ffb640 1 -- 172.21.15.63:0/2792573326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f778c043070 con 0x55dc80940dd0 2024-04-23T19:10:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f779441f640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7664036eb0 0x7f7664039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80953fc0 0x55dc80956480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80956bd0 con 0x55dc80953fc0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7793c1e640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80953fc0 0x55dc80956480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f779441f640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7664036eb0 0x7f7664039370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f778400ddd0 tx=0x7f778402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.987+0000 7f7793c1e640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80953fc0 0x55dc80956480 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 20 realm 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80957000 con 0x55dc80953fc0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc809578d0 con 0x55dc80953fc0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80957c70 con 0x55dc80953fc0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.988+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7796919800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7796919800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7796919800 20 searching for the correct realm 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc8095a0d0 0x55dc8095c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095cc40 con 0x55dc8095a0d0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7794c20640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc8095a0d0 0x55dc8095c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.989+0000 7f7794c20640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc8095a0d0 0x55dc8095c570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f7794c20640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f778c0123a0 0x7f778c04ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095cc40 con 0x7f778c0123a0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f779441f640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f778c0123a0 0x7f778c04ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f779441f640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f778c0123a0 0x7f778c04ea00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f779441f640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7784037070 0x7f77840165e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.990+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095cc40 con 0x7f7784037070 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.991+0000 7f7793c1e640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7784037070 0x7f77840165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.991+0000 7f7793c1e640 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7784037070 0x7f77840165e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.991+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.991+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.992+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f778c0123a0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.993+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f778c0123a0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.994+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.995+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.995+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.995+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.995+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.995+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.995+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.996+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f778c0123a0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.997+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.997+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.997+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.997+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.997+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.997+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.997+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.998+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.998+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.998+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f778c0123a0 2024-04-23T19:10:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.998+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.998+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc8095a0d0 2024-04-23T19:10:26.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.998+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778c029070 con 0x55dc8095a0d0 2024-04-23T19:10:26.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.998+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x7f7784037070 2024-04-23T19:10:26.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f778808a040 con 0x7f778c0123a0 2024-04-23T19:10:26.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc809574e0 con 0x55dc80953fc0 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:25.999+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc809589f0 con 0x55dc80953fc0 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc809589f0 con 0x7f7784037070 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 zone default found 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 4 Realm: () 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 zonegroup default 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80958e10 con 0x7f7784037070 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.000+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80959230 con 0x55dc8095a0d0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7796919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7796919800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095eaa0 con 0x55dc8095a0d0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7796919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7796919800 20 started sync module instance, tier type = 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7796919800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.001+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095df40 con 0x7f778c0123a0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.002+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.002+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94405538470544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095ee40 con 0x7f778c0123a0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.002+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095f690 con 0x55dc80953fc0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.003+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.003+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94405538476784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095fde0 con 0x55dc80953fc0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.003+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80960900 con 0x55dc8095a0d0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.005+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94405538470544] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.006+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.006+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94405538476784] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.006+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94405538481104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8095f830 con 0x55dc8095a0d0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.006+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc809611a0 con 0x7f778c0123a0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.009+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94405538481104] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.009+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.009+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94405538483440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80961590 con 0x7f778c0123a0 2024-04-23T19:10:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.009+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80961fb0 con 0x55dc80953fc0 2024-04-23T19:10:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.012+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.012+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94405538486944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80962880 con 0x55dc80953fc0 2024-04-23T19:10:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.012+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80963350 con 0x7f7784037070 2024-04-23T19:10:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.013+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.013+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94405538491968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc809637c0 con 0x7f7784037070 2024-04-23T19:10:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.013+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc809642e0 con 0x7f7784037070 2024-04-23T19:10:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.014+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94405538491968] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.015+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94405538483440] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.017+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.017+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94405538472800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc809637c0 con 0x7f7784037070 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.017+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80964750 con 0x7f7784037070 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.018+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.018+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94405538497184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80964b70 con 0x7f7784037070 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.018+0000 7f7796919800 20 add_watcher() i=0 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.018+0000 7f7796919800 20 add_watcher() i=1 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.018+0000 7f7796919800 20 add_watcher() i=2 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.018+0000 7f7796919800 20 add_watcher() i=3 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.019+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94405538486944] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.019+0000 7f7796919800 20 add_watcher() i=4 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.019+0000 7f7796919800 20 add_watcher() i=5 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.019+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94405538472800] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.020+0000 7f7796919800 20 add_watcher() i=6 2024-04-23T19:10:26.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.021+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94405538497184] v30'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.021+0000 7f7796919800 20 add_watcher() i=7 2024-04-23T19:10:26.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.021+0000 7f7796919800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.021+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc80963310 con 0x55dc80953fc0 2024-04-23T19:10:26.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.021+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.021+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94405538498240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc80963310 con 0x55dc80953fc0 2024-04-23T19:10:26.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.023+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94405538498240] v30'46 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.023+0000 7f764a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7796919800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7796919800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7796919800 5 note: GC not initialized 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f76497e2640 20 reqs_thread_entry: start 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7796919800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7796919800 20 Filter name: none 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f75baffd640 20 reqs_thread_entry: start 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b050 con 0x55dc80953fc0 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.024+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b050 con 0x7f778c0123a0 2024-04-23T19:10:26.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.025+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.026+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.027+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.027+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.027+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.027+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f778c0123a0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.027+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.027+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.028+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.029+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f778c0123a0 2024-04-23T19:10:26.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.030+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.031+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.032+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.032+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.032+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.033+0000 7f764a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.033+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94405538498240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.034+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94405538498240] v30'47 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.034+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94405538470544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f778c0123a0 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.035+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94405538470544] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.035+0000 7f7796919800 20 remove_watcher() i=0 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.035+0000 7f7796919800 2 removed watcher, disabling cache 2024-04-23T19:10:26.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.035+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94405538476784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.036+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94405538476784] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.036+0000 7f7796919800 20 remove_watcher() i=1 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.036+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94405538481104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc8095a0d0 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.037+0000 7f7794c20640 1 -- 172.21.15.63:0/2792573326 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94405538481104] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778c068040 con 0x55dc8095a0d0 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.037+0000 7f7796919800 20 remove_watcher() i=2 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.037+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94405538483440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f778c0123a0 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.038+0000 7f779441f640 1 -- 172.21.15.63:0/2792573326 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94405538483440] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778400d3d0 con 0x7f778c0123a0 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.038+0000 7f7796919800 20 remove_watcher() i=3 2024-04-23T19:10:26.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.038+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94405538486944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x55dc80953fc0 2024-04-23T19:10:26.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.039+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94405538486944] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x55dc80953fc0 2024-04-23T19:10:26.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.039+0000 7f7796919800 20 remove_watcher() i=4 2024-04-23T19:10:26.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.039+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94405538491968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.040+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94405538491968] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.040+0000 7f7796919800 20 remove_watcher() i=5 2024-04-23T19:10:26.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.040+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94405538472800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.041+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94405538472800] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.041+0000 7f7796919800 20 remove_watcher() i=6 2024-04-23T19:10:26.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.041+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94405538497184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55dc8099b4d0 con 0x7f7784037070 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94405538497184] v30'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f778805a040 con 0x7f7784037070 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 20 remove_watcher() i=7 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7784037070 msgr2=0x7f77840165e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7784037070 0x7f77840165e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80953fc0 msgr2=0x55dc80956480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80953fc0 0x55dc80956480 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f778c0123a0 msgr2=0x7f778c04ea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f778c0123a0 0x7f778c04ea00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc8095a0d0 msgr2=0x55dc8095c570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dc8095a0d0 0x55dc8095c570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7664036eb0 msgr2=0x7f7664039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7664036eb0 0x7f7664039370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f778400ddd0 tx=0x7f778402e000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 msgr2=0x55dc80948330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7796919800 1 --2- 172.21.15.63:0/2792573326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80940dd0 0x55dc80948330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f778c001df0 tx=0x7f778c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 reap_dead start 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.042+0000 7f7793c1e640 1 -- 172.21.15.63:0/2792573326 reap_dead start 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 shutdown_connections 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 >> 172.21.15.63:0/2792573326 conn(0x55dc8093d130 msgr2=0x55dc8093a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 shutdown_connections 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/2792573326 wait complete. 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f766c036e60 msgr2=0x7f766c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f766c036e60 0x7f766c039320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f778400d6e0 tx=0x7f778400dc60 comp rx=0 tx=0).stop 2024-04-23T19:10:26.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80933af0 msgr2=0x55dc80933f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80933af0 0x55dc80933f10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55dc80919780 tx=0x7f778c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 shutdown_connections 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f766c036e60 0x7f766c039320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3562104958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80933af0 0x55dc80933f10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 >> 172.21.15.63:0/3562104958 conn(0x55dc80927b90 msgr2=0x55dc80927f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 shutdown_connections 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3562104958 wait complete. 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc80922cd0 msgr2=0x55dc809250c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc80922cd0 0x55dc809250c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80792c10 msgr2=0x55dc8091f5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80792c10 0x55dc8091f5c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f767c036eb0 msgr2=0x7f767c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f767c036eb0 0x7f767c039370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f778800a470 tx=0x7f778804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 msgr2=0x55dc80918380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80918380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7784001dd0 tx=0x7f7784000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 shutdown_connections 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dc80922cd0 0x55dc809250c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dc80792c10 0x55dc8091f5c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f767c036eb0 0x7f767c039370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 --2- 172.21.15.63:0/3916437119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc80791880 0x55dc80918380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.043+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 >> 172.21.15.63:0/3916437119 conn(0x55dc808187d0 msgr2=0x55dc8090a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.044+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 shutdown_connections 2024-04-23T19:10:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.044+0000 7f7796919800 1 -- 172.21.15.63:0/3916437119 wait complete. 2024-04-23T19:10:26.052 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[],"summary":[]} 2024-04-23T19:10:26.052 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-23T19:10:26.052 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:26.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:26.053 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b5b75800 1 Processor -- start 2024-04-23T19:10:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b5b75800 1 -- start start 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b5b75800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55c339363c80 0x55c339366070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b5b75800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c3392157f0 con 0x55c3393665b0 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b5b75800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55c339204e20 con 0x55c339363c80 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b3674640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55c339363c80 0x55c339366070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b2e73640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55c3393665b0 0x55c3393689a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52662/0 (socket says 172.21.15.63:52662) 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b2e73640 1 -- 172.21.15.63:0/2209741150 learned_addr learned my addr 172.21.15.63:0/2209741150 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b3674640 1 -- 172.21.15.63:0/2209741150 >> v1:172.21.15.16:6789/0 conn(0x55c3393665b0 legacy=0x55c3393689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.137+0000 7f69b3674640 1 -- 172.21.15.63:0/2209741150 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c339223b50 con 0x55c339363c80 2024-04-23T19:10:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.139+0000 7f69b3674640 1 --2- 172.21.15.63:0/2209741150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339366070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69a4005fd0 tx=0x7f69a401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd57e2595465567 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.139+0000 7f69b2672640 1 -- 172.21.15.63:0/2209741150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a4052560 con 0x55c339363c80 2024-04-23T19:10:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b2672640 1 -- 172.21.15.63:0/2209741150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a4052700 con 0x55c339363c80 2024-04-23T19:10:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b2672640 1 -- 172.21.15.63:0/2209741150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a4052a60 con 0x55c339363c80 2024-04-23T19:10:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b5b75800 1 -- 172.21.15.63:0/2209741150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 msgr2=0x55c339366070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b5b75800 1 --2- 172.21.15.63:0/2209741150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339366070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69a4005fd0 tx=0x7f69a401ccc0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b5b75800 1 -- 172.21.15.63:0/2209741150 shutdown_connections 2024-04-23T19:10:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b5b75800 1 --2- 172.21.15.63:0/2209741150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339366070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b5b75800 1 -- 172.21.15.63:0/2209741150 >> 172.21.15.63:0/2209741150 conn(0x55c33935d630 msgr2=0x55c33935fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b5b75800 1 -- 172.21.15.63:0/2209741150 shutdown_connections 2024-04-23T19:10:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.140+0000 7f69b5b75800 1 -- 172.21.15.63:0/2209741150 wait complete. 2024-04-23T19:10:26.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.153+0000 7f69b5b75800 1 Processor -- start 2024-04-23T19:10:26.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.153+0000 7f69b5b75800 1 -- start start 2024-04-23T19:10:26.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.153+0000 7f69b5b75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339454350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.153+0000 7f69b5b75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c3392ae6f0 con 0x55c339363c80 2024-04-23T19:10:26.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.153+0000 7f69b3674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339454350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.153+0000 7f69b3674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339454350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48188/0 (socket says 172.21.15.63:48188) 2024-04-23T19:10:26.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.153+0000 7f69b3674640 1 -- 172.21.15.63:0/1755817938 learned_addr learned my addr 172.21.15.63:0/1755817938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.154+0000 7f69b3674640 1 -- 172.21.15.63:0/1755817938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c339454920 con 0x55c339363c80 2024-04-23T19:10:26.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.154+0000 7f69b3674640 1 --2- 172.21.15.63:0/1755817938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339454350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007000 tx=0x7f69a404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358a011698738e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.154+0000 7f68a160a640 1 -- 172.21.15.63:0/1755817938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a40529e0 con 0x55c339363c80 2024-04-23T19:10:26.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.154+0000 7f68a160a640 1 -- 172.21.15.63:0/1755817938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a4052b80 con 0x55c339363c80 2024-04-23T19:10:26.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.154+0000 7f68a160a640 1 -- 172.21.15.63:0/1755817938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a405c430 con 0x55c339363c80 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 -- 172.21.15.63:0/1755817938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 msgr2=0x55c339454350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1755817938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339454350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007000 tx=0x7f69a404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 -- 172.21.15.63:0/1755817938 shutdown_connections 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1755817938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c339454350 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 -- 172.21.15.63:0/1755817938 >> 172.21.15.63:0/1755817938 conn(0x55c33935d630 msgr2=0x55c33944fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 -- 172.21.15.63:0/1755817938 shutdown_connections 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 -- 172.21.15.63:0/1755817938 wait complete. 2024-04-23T19:10:26.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 Processor -- start 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 -- start start 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c33945be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b5b75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c33936c370 con 0x55c339363c80 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b3674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c33945be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b3674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c33945be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48192/0 (socket says 172.21.15.63:48192) 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.155+0000 7f69b3674640 1 -- 172.21.15.63:0/1055472988 learned_addr learned my addr 172.21.15.63:0/1055472988 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.156+0000 7f69b3674640 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33945c3c0 con 0x55c339363c80 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.156+0000 7f69b3674640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c33945be80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f69a40526d0 tx=0x7f69a4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.156+0000 7f689fe07640 1 -- 172.21.15.63:0/1055472988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a4075070 con 0x55c339363c80 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.156+0000 7f689fe07640 1 -- 172.21.15.63:0/1055472988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a4070410 con 0x55c339363c80 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.156+0000 7f689fe07640 1 -- 172.21.15.63:0/1055472988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a40529e0 con 0x55c339363c80 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.156+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c33945c650 con 0x55c339363c80 2024-04-23T19:10:26.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.156+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c33945f300 con 0x55c339363c80 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.157+0000 7f689fe07640 1 -- 172.21.15.63:0/1055472988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f69a4078070 con 0x55c339363c80 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.157+0000 7f689fe07640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6898036eb0 0x7f6898039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.157+0000 7f689fe07640 1 -- 172.21.15.63:0/1055472988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f69a406d070 con 0x55c339363c80 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.157+0000 7f69b2e73640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6898036eb0 0x7f6898039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.157+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c3393665b0 0x55c339463120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.157+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c339463660 con 0x55c3393665b0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.157+0000 7f69b3e75640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c3393665b0 0x55c339463120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.158+0000 7f69b2e73640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6898036eb0 0x7f6898039370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f699c000f60 tx=0x7f699c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.158+0000 7f69b3e75640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c3393665b0 0x55c339463120 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.158+0000 7f69b3e75640 1 -- 172.21.15.63:0/1055472988 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f69ac0083e0 con 0x55c3393665b0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.158+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394606d0 con 0x55c3393665b0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.158+0000 7f69b3e75640 1 -- 172.21.15.63:0/1055472988 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f69ac0083e0 con 0x55c3393665b0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.158+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c339465420 con 0x55c3393665b0 2024-04-23T19:10:26.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.159+0000 7f69b3e75640 1 -- 172.21.15.63:0/1055472988 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f69ac0083e0 con 0x55c3393665b0 2024-04-23T19:10:26.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.159+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c339464760 con 0x55c3393665b0 2024-04-23T19:10:26.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.159+0000 7f69b3e75640 1 -- 172.21.15.63:0/1055472988 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f69ac0083e0 con 0x55c3393665b0 2024-04-23T19:10:26.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.159+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c3394667d0 0x55c339468bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.159+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c339469140 con 0x55c3394667d0 2024-04-23T19:10:26.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.159+0000 7f69b3674640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c3394667d0 0x55c339468bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.159+0000 7f69b3674640 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c3394667d0 0x55c339468bc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b3674640 1 -- 172.21.15.63:0/1055472988 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f69a4077030 con 0x55c3394667d0 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b5b75800 1 Processor -- start 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b5b75800 1 -- start start 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b5b75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339477580 0x55c3394779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b5b75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c339254e30 con 0x55c339477580 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b3e75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339477580 0x55c3394779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b3e75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339477580 0x55c3394779a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48196/0 (socket says 172.21.15.63:48196) 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.160+0000 7f69b3e75640 1 -- 172.21.15.63:0/832939823 learned_addr learned my addr 172.21.15.63:0/832939823 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.161+0000 7f69b3e75640 1 -- 172.21.15.63:0/832939823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c339477ee0 con 0x55c339477580 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.161+0000 7f69b3e75640 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339477580 0x55c3394779a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c33945d280 tx=0x7f69ac00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.161+0000 7f689d602640 1 -- 172.21.15.63:0/832939823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac01c070 con 0x55c339477580 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.161+0000 7f689d602640 1 -- 172.21.15.63:0/832939823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ac00bdc0 con 0x55c339477580 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.161+0000 7f689d602640 1 -- 172.21.15.63:0/832939823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac018540 con 0x55c339477580 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.161+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c339478950 con 0x55c339477580 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.161+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c339478e50 con 0x55c339477580 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f689d602640 1 -- 172.21.15.63:0/832939823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f69ac01f070 con 0x55c339477580 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f689d602640 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f688c036eb0 0x7f688c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f689d602640 1 -- 172.21.15.63:0/832939823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f69ac04e880 con 0x55c339477580 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f69b3674640 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f688c036eb0 0x7f688c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f69b5b75800 1 Processor -- start 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f69b3674640 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f688c036eb0 0x7f688c039370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f69a405cd30 tx=0x7f69a4087040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f69b5b75800 1 -- start start 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.162+0000 7f69b5b75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c33948da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f69b5b75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c339454c40 con 0x55c33948b680 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f69b3e75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c33948da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f69b3e75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c33948da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48206/0 (socket says 172.21.15.63:48206) 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f69b3e75640 1 -- 172.21.15.63:0/3793433275 learned_addr learned my addr 172.21.15.63:0/3793433275 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f69b3e75640 1 -- 172.21.15.63:0/3793433275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c339482bd0 con 0x55c33948b680 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f69b3e75640 1 --2- 172.21.15.63:0/3793433275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c33948da70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69ac04ccc0 tx=0x7f69ac014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a2dfa687882287 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f6896ffd640 1 -- 172.21.15.63:0/3793433275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac029c70 con 0x55c33948b680 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f6896ffd640 1 -- 172.21.15.63:0/3793433275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ac04a020 con 0x55c33948b680 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.163+0000 7f6896ffd640 1 -- 172.21.15.63:0/3793433275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac014bb0 con 0x55c33948b680 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 -- 172.21.15.63:0/3793433275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 msgr2=0x55c33948da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 --2- 172.21.15.63:0/3793433275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c33948da70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f69ac04ccc0 tx=0x7f69ac014690 comp rx=0 tx=0).stop 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 -- 172.21.15.63:0/3793433275 shutdown_connections 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 --2- 172.21.15.63:0/3793433275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c33948da70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 -- 172.21.15.63:0/3793433275 >> 172.21.15.63:0/3793433275 conn(0x55c339361db0 msgr2=0x55c339362190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 -- 172.21.15.63:0/3793433275 shutdown_connections 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 -- 172.21.15.63:0/3793433275 wait complete. 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 Processor -- start 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 -- start start 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c3394923e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b5b75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c33948e1f0 con 0x55c33948b680 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b3e75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c3394923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b3e75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c3394923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48220/0 (socket says 172.21.15.63:48220) 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.164+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 learned_addr learned my addr 172.21.15.63:0/1535277119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.165+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c339492920 con 0x55c33948b680 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.165+0000 7f69b3e75640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c3394923e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69ac04bbb0 tx=0x7f69ac04cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.165+0000 7f68957fa640 1 -- 172.21.15.63:0/1535277119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac0186e0 con 0x55c33948b680 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.165+0000 7f68957fa640 1 -- 172.21.15.63:0/1535277119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ac018880 con 0x55c33948b680 2024-04-23T19:10:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.165+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c339492bb0 con 0x55c33948b680 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.165+0000 7f68957fa640 1 -- 172.21.15.63:0/1535277119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac018b60 con 0x55c33948b680 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.165+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c339495860 con 0x55c33948b680 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.166+0000 7f68957fa640 1 -- 172.21.15.63:0/1535277119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f69ac04e0d0 con 0x55c33948b680 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.166+0000 7f68957fa640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f687c036e40 0x7f687c039300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.166+0000 7f68957fa640 1 -- 172.21.15.63:0/1535277119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f69ac04b030 con 0x55c33948b680 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.166+0000 7f69b3674640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f687c036e40 0x7f687c039300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.166+0000 7f69b3674640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f687c036e40 0x7f687c039300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c339469540 tx=0x7f69a4091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c33949dfd0 0x55c3394a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a0be0 con 0x55c33949dfd0 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b2e73640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c33949dfd0 0x55c3394a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b2e73640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c33949dfd0 0x55c3394a0490 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f699c009050 con 0x55c33949dfd0 2024-04-23T19:10:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b5b75800 20 realm 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.167+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c33949d420 con 0x55c33949dfd0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f699c009050 con 0x55c33949dfd0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a16c0 con 0x55c33949dfd0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f699c009050 con 0x55c33949dfd0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a1f90 con 0x55c33949dfd0 2024-04-23T19:10:26.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f699c009050 con 0x55c33949dfd0 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a2330 con 0x55c33949dfd0 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f699c009050 con 0x55c33949dfd0 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 20 searching for the correct realm 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c3394a4210 0x55c3394a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.168+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.169+0000 7f69b3e75640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c3394a4210 0x55c3394a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.169+0000 7f69b3e75640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c3394a4210 0x55c3394a6600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.169+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.169+0000 7f69b3e75640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69ac0163a0 0x7f69ac02b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.169+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69ac0163a0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.169+0000 7f69b3674640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69ac0163a0 0x7f69ac02b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.170+0000 7f69b3674640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69ac0163a0 0x7f69ac02b790 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.170+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.170+0000 7f69b3674640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69a407a060 0x7f69a409b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.170+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.170+0000 7f69b2e73640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69a407a060 0x7f69a409b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.171+0000 7f69b2e73640 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69a407a060 0x7f69a409b970 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.171+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c009050 con 0x7f69a407a060 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.171+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.171+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c009050 con 0x55c33949dfd0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.171+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.171+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.171+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69ac0163a0 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.172+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69ac0163a0 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.173+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.174+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.174+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.174+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.174+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.174+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.174+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.175+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69ac0163a0 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.176+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69ac0163a0 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.177+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c3394a4210 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.178+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.178+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69a407a060 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.178+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.178+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x55c33949dfd0 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.178+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.178+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a6cd0 con 0x7f69ac0163a0 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a2ce0 con 0x55c33949dfd0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a3580 con 0x55c33949dfd0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a2920 con 0x7f69a407a060 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 zone default found 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 4 Realm: () 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 zonegroup default 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.179+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a7150 con 0x7f69a407a060 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a8fe0 con 0x55c3394a4210 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a9380 con 0x55c3394a4210 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 20 started sync module instance, tier type = 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.180+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a8420 con 0x7f69ac0163a0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.181+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.181+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94296968169536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394a9830 con 0x7f69ac0163a0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.181+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394aa080 con 0x55c33949dfd0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.183+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.183+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94296968174544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394aa600 con 0x55c33949dfd0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.183+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94296968169536] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.183+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ab100 con 0x55c3394a4210 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.184+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94296968174544] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.184+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.184+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94296968178672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ab0c0 con 0x55c3394a4210 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.184+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ab9d0 con 0x7f69ac0163a0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.185+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.185+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94296968181024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394abdf0 con 0x7f69ac0163a0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.185+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ac640 con 0x55c33949dfd0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.185+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94296968178672] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.186+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.186+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94296968183280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394aca30 con 0x55c33949dfd0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.186+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ad280 con 0x7f69a407a060 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.187+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94296968181024] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.187+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94296968183280] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.187+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.187+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94296968182080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ac320 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.187+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ad580 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.188+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94296968182080] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.189+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.189+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94296968187984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ad540 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.189+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394adf40 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94296968190544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394ae360 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94296968187984] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 20 add_watcher() i=0 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 20 add_watcher() i=1 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 20 add_watcher() i=2 2024-04-23T19:10:26.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 20 add_watcher() i=3 2024-04-23T19:10:26.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 20 add_watcher() i=4 2024-04-23T19:10:26.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 20 add_watcher() i=5 2024-04-23T19:10:26.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.190+0000 7f69b5b75800 20 add_watcher() i=6 2024-04-23T19:10:26.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.191+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94296968190544] v30'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.191+0000 7f69b5b75800 20 add_watcher() i=7 2024-04-23T19:10:26.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.191+0000 7f69b5b75800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.191+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394aca40 con 0x55c33949dfd0 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.191+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.191+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94296968191600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394aec30 con 0x55c33949dfd0 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.192+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94296968191600] v30'48 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f6869fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 5 note: GC not initialized 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f6868fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 20 Filter name: none 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f67e27fc640 20 reqs_thread_entry: start 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e43a0 con 0x55c33949dfd0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.193+0000 7f69b5b75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:could not fetch user info: no user info saved 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.194+0000 7f6869fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.194+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94296968191600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x55c33949dfd0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.195+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94296968191600] v30'49 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.195+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94296968169536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x7f69ac0163a0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.196+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94296968169536] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.196+0000 7f69b5b75800 20 remove_watcher() i=0 2024-04-23T19:10:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.196+0000 7f69b5b75800 2 removed watcher, disabling cache 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.196+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94296968174544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x55c33949dfd0 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.197+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94296968174544] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.197+0000 7f69b5b75800 20 remove_watcher() i=1 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.197+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94296968178672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x55c3394a4210 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.199+0000 7f69b3e75640 1 -- 172.21.15.63:0/1535277119 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94296968178672] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69ac028070 con 0x55c3394a4210 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.199+0000 7f69b5b75800 20 remove_watcher() i=2 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.199+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94296968181024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x7f69ac0163a0 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.200+0000 7f69b3674640 1 -- 172.21.15.63:0/1535277119 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94296968181024] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69a4077030 con 0x7f69ac0163a0 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.200+0000 7f69b5b75800 20 remove_watcher() i=3 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.200+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94296968183280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x55c33949dfd0 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.201+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94296968183280] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x55c33949dfd0 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.201+0000 7f69b5b75800 20 remove_watcher() i=4 2024-04-23T19:10:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.201+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94296968182080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x7f69a407a060 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.202+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94296968182080] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.202+0000 7f69b5b75800 20 remove_watcher() i=5 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.202+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94296968187984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x7f69a407a060 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.203+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94296968187984] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.203+0000 7f69b5b75800 20 remove_watcher() i=6 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.203+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94296968190544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c3394e4360 con 0x7f69a407a060 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94296968190544] v30'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f699c02a040 con 0x7f69a407a060 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 20 remove_watcher() i=7 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69a407a060 msgr2=0x7f69a409b970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69a407a060 0x7f69a409b970 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c33949dfd0 msgr2=0x55c3394a0490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c33949dfd0 0x55c3394a0490 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69ac0163a0 msgr2=0x7f69ac02b790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69ac0163a0 0x7f69ac02b790 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c3394a4210 msgr2=0x55c3394a6600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c3394a4210 0x55c3394a6600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f687c036e40 msgr2=0x7f687c039300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f687c036e40 0x7f687c039300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c339469540 tx=0x7f69a4091000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 msgr2=0x55c3394923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b2e73640 1 -- 172.21.15.63:0/1535277119 reap_dead start 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c3394923e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69ac04bbb0 tx=0x7f69ac04cc20 comp rx=0 tx=0).stop 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 shutdown_connections 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1535277119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c33948b680 0x55c3394923e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 >> 172.21.15.63:0/1535277119 conn(0x55c339361db0 msgr2=0x55c339362190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 shutdown_connections 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/1535277119 wait complete. 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f688c036eb0 msgr2=0x7f688c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f688c036eb0 0x7f688c039370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f69a405cd30 tx=0x7f69a4087040 comp rx=0 tx=0).stop 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339477580 msgr2=0x55c3394779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339477580 0x55c3394779a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c33945d280 tx=0x7f69ac00ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 shutdown_connections 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f688c036eb0 0x7f688c039370 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 --2- 172.21.15.63:0/832939823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339477580 0x55c3394779a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 >> 172.21.15.63:0/832939823 conn(0x55c33946b620 msgr2=0x55c33946ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 shutdown_connections 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.204+0000 7f69b5b75800 1 -- 172.21.15.63:0/832939823 wait complete. 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c3394667d0 msgr2=0x55c339468bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c3394667d0 0x55c339468bc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c3393665b0 msgr2=0x55c339463120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c3393665b0 0x55c339463120 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6898036eb0 msgr2=0x7f6898039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6898036eb0 0x7f6898039370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f699c000f60 tx=0x7f699c0012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 msgr2=0x55c33945be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c33945be80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f69a40526d0 tx=0x7f69a4003330 comp rx=0 tx=0).stop 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 shutdown_connections 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c3394667d0 0x55c339468bc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c3393665b0 0x55c339463120 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6898036eb0 0x7f6898039370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 --2- 172.21.15.63:0/1055472988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c339363c80 0x55c33945be80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 >> 172.21.15.63:0/1055472988 conn(0x55c33935d630 msgr2=0x55c33944e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 shutdown_connections 2024-04-23T19:10:26.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.205+0000 7f69b5b75800 1 -- 172.21.15.63:0/1055472988 wait complete. 2024-04-23T19:10:26.214 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T19:10:26.214 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-23T19:10:26.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-23T19:10:26.215 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-04-23T19:10:26.269 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:26.269 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:26.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.269+0000 7fcb5031f800 1 Processor -- start 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.269+0000 7fcb5031f800 1 -- start start 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb5031f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55b811e45500 0x55b811e4e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb5031f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b811cf77f0 con 0x55b811e439c0 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb5031f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55b811ce6e20 con 0x55b811e45500 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb4b7c2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55b811e45500 0x55b811e4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb4bfc3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55b811e439c0 0x55b811e43de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52664/0 (socket says 172.21.15.63:52664) 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/1279215803 learned_addr learned my addr 172.21.15.63:0/1279215803 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb4afc1640 1 -- 172.21.15.63:0/1279215803 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3676514099 0 0) 0x55b811cf77f0 con 0x55b811e439c0 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb4afc1640 1 -- 172.21.15.63:0/1279215803 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb34001e10 con 0x55b811e439c0 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/1279215803 >> v1:172.21.15.16:6789/0 conn(0x55b811e439c0 legacy=0x55b811e43de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.270+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/1279215803 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811d05b50 con 0x55b811e45500 2024-04-23T19:10:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.272+0000 7fcb4b7c2640 1 --2- 172.21.15.63:0/1279215803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e45500 0x55b811e4e000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000a470 tx=0x7fcb4001cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0dd2e7e2ee6eca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.272+0000 7fcb4afc1640 1 -- 172.21.15.63:0/1279215803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb40057070 con 0x55b811e45500 2024-04-23T19:10:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.272+0000 7fcb4afc1640 1 -- 172.21.15.63:0/1279215803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4004acd0 con 0x55b811e45500 2024-04-23T19:10:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.272+0000 7fcb4afc1640 1 -- 172.21.15.63:0/1279215803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb40052440 con 0x55b811e45500 2024-04-23T19:10:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.273+0000 7fcb5031f800 1 -- 172.21.15.63:0/1279215803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e45500 msgr2=0x55b811e4e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.273+0000 7fcb5031f800 1 --2- 172.21.15.63:0/1279215803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e45500 0x55b811e4e000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000a470 tx=0x7fcb4001cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.273+0000 7fcb5031f800 1 -- 172.21.15.63:0/1279215803 shutdown_connections 2024-04-23T19:10:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.273+0000 7fcb5031f800 1 --2- 172.21.15.63:0/1279215803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e45500 0x55b811e4e000 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.273+0000 7fcb5031f800 1 -- 172.21.15.63:0/1279215803 >> 172.21.15.63:0/1279215803 conn(0x55b811e3f5a0 msgr2=0x55b811e419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.273+0000 7fcb5031f800 1 -- 172.21.15.63:0/1279215803 shutdown_connections 2024-04-23T19:10:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.273+0000 7fcb5031f800 1 -- 172.21.15.63:0/1279215803 wait complete. 2024-04-23T19:10:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.286+0000 7fcb5031f800 1 Processor -- start 2024-04-23T19:10:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.286+0000 7fcb5031f800 1 -- start start 2024-04-23T19:10:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.286+0000 7fcb5031f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.286+0000 7fcb5031f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b811d90690 con 0x55b811e439c0 2024-04-23T19:10:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.286+0000 7fcb4bfc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.286+0000 7fcb4bfc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48236/0 (socket says 172.21.15.63:48236) 2024-04-23T19:10:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.286+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/916417023 learned_addr learned my addr 172.21.15.63:0/916417023 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.287+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/916417023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811f38c70 con 0x55b811e439c0 2024-04-23T19:10:26.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.287+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/916417023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f386a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001de0 tx=0x7fcb3c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4685e93ee94cf35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.287+0000 7fca39e0b640 1 -- 172.21.15.63:0/916417023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c012020 con 0x55b811e439c0 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.287+0000 7fca39e0b640 1 -- 172.21.15.63:0/916417023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c00e5a0 con 0x55b811e439c0 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.287+0000 7fca39e0b640 1 -- 172.21.15.63:0/916417023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c00e900 con 0x55b811e439c0 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 -- 172.21.15.63:0/916417023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 msgr2=0x55b811f386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 --2- 172.21.15.63:0/916417023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f386a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001de0 tx=0x7fcb3c00b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 -- 172.21.15.63:0/916417023 shutdown_connections 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 --2- 172.21.15.63:0/916417023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f386a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 -- 172.21.15.63:0/916417023 >> 172.21.15.63:0/916417023 conn(0x55b811e3f5a0 msgr2=0x55b811f33fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 -- 172.21.15.63:0/916417023 shutdown_connections 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 -- 172.21.15.63:0/916417023 wait complete. 2024-04-23T19:10:26.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 Processor -- start 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 -- start start 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb5031f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b811e50820 con 0x55b811e439c0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb4bfc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb4bfc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48250/0 (socket says 172.21.15.63:48250) 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.288+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/2519691642 learned_addr learned my addr 172.21.15.63:0/2519691642 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.289+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811f40700 con 0x55b811e439c0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.289+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f401c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c010 tx=0x7fcb3c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.289+0000 7fca33fff640 1 -- 172.21.15.63:0/2519691642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c012030 con 0x55b811e439c0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.289+0000 7fca33fff640 1 -- 172.21.15.63:0/2519691642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c01a7a0 con 0x55b811e439c0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.289+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b811f40990 con 0x55b811e439c0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.289+0000 7fca33fff640 1 -- 172.21.15.63:0/2519691642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c01a950 con 0x55b811e439c0 2024-04-23T19:10:26.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.289+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b811f43640 con 0x55b811e439c0 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fca33fff640 1 -- 172.21.15.63:0/2519691642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcb3c015020 con 0x55b811e439c0 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fca33fff640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca34036e60 0x7fca34039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fca33fff640 1 -- 172.21.15.63:0/2519691642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fcb3c020030 con 0x55b811e439c0 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fcb4b7c2640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca34036e60 0x7fca34039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811e45500 0x55b811f47400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f47990 con 0x55b811e45500 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811e45500 0x55b811f47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.290+0000 7fcb4b7c2640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca34036e60 0x7fca34039320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb40054000 tx=0x7fcb4004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.291+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811e45500 0x55b811f47400 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.291+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/2519691642 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb4400a4a0 con 0x55b811e45500 2024-04-23T19:10:26.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.291+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f449d0 con 0x55b811e45500 2024-04-23T19:10:26.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.291+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/2519691642 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcb4400a4a0 con 0x55b811e45500 2024-04-23T19:10:26.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.291+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f497d0 con 0x55b811e45500 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.291+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/2519691642 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fcb4400a4a0 con 0x55b811e45500 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.292+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f48b10 con 0x55b811e45500 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.292+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/2519691642 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcb4400a4a0 con 0x55b811e45500 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.292+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b811f4ab80 0x55b811f4cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.292+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f4d4f0 con 0x55b811f4ab80 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.292+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b811f4ab80 0x55b811f4cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.292+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b811f4ab80 0x55b811f4cf70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/2519691642 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fcb3c00dcd0 con 0x55b811f4ab80 2024-04-23T19:10:26.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb5031f800 1 Processor -- start 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb5031f800 1 -- start start 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb5031f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f5b910 0x55b811f5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb5031f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b811d36e30 con 0x55b811f5b910 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb4c7c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f5b910 0x55b811f5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb4c7c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f5b910 0x55b811f5bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48262/0 (socket says 172.21.15.63:48262) 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.293+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/2519724219 learned_addr learned my addr 172.21.15.63:0/2519724219 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.294+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/2519724219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811f5c270 con 0x55b811f5b910 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.294+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f5b910 0x55b811f5bd30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55b811f415c0 tx=0x7fcb440178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.294+0000 7fca317fa640 1 -- 172.21.15.63:0/2519724219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb44013440 con 0x55b811f5b910 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.294+0000 7fca317fa640 1 -- 172.21.15.63:0/2519724219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb440135e0 con 0x55b811f5b910 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.294+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b811f5cce0 con 0x55b811f5b910 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.294+0000 7fca317fa640 1 -- 172.21.15.63:0/2519724219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb440138c0 con 0x55b811f5b910 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.294+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b811f5d1e0 con 0x55b811f5b910 2024-04-23T19:10:26.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fca317fa640 1 -- 172.21.15.63:0/2519724219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcb44011040 con 0x55b811f5b910 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fca317fa640 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca24036eb0 0x7fca24039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fca317fa640 1 -- 172.21.15.63:0/2519724219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fcb4401d070 con 0x55b811f5b910 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca24036eb0 0x7fca24039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fcb5031f800 1 Processor -- start 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fcb5031f800 1 -- start start 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca24036eb0 0x7fca24039370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c02d040 tx=0x7fcb3c04b320 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.295+0000 7fcb5031f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f69010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fcb5031f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b811f38ff0 con 0x55b811f68bf0 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fcb4c7c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f69010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fcb4c7c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f69010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48274/0 (socket says 172.21.15.63:48274) 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3831261678 learned_addr learned my addr 172.21.15.63:0/3831261678 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3831261678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811f695e0 con 0x55b811f68bf0 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/3831261678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f69010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb4404cca0 tx=0x7fcb44012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35caae7443dcc5a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fca2affd640 1 -- 172.21.15.63:0/3831261678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb44042030 con 0x55b811f68bf0 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fca2affd640 1 -- 172.21.15.63:0/3831261678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb44001310 con 0x55b811f68bf0 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.296+0000 7fca2affd640 1 -- 172.21.15.63:0/3831261678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb44027040 con 0x55b811f68bf0 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 -- 172.21.15.63:0/3831261678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 msgr2=0x55b811f69010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3831261678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f69010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb4404cca0 tx=0x7fcb44012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 -- 172.21.15.63:0/3831261678 shutdown_connections 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3831261678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f69010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 -- 172.21.15.63:0/3831261678 >> 172.21.15.63:0/3831261678 conn(0x55b811f64f50 msgr2=0x55b811f62fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 -- 172.21.15.63:0/3831261678 shutdown_connections 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 -- 172.21.15.63:0/3831261678 wait complete. 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 Processor -- start 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 -- start start 2024-04-23T19:10:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.297+0000 7fcb5031f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b811f69cf0 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fcb4c7c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fcb4c7c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48280/0 (socket says 172.21.15.63:48280) 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 learned_addr learned my addr 172.21.15.63:0/3713072090 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811f706e0 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f701a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001df0 tx=0x7fcb4404cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fca297fa640 1 -- 172.21.15.63:0/3713072090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb44027030 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fca297fa640 1 -- 172.21.15.63:0/3713072090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4402a350 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b811f70970 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fca297fa640 1 -- 172.21.15.63:0/3713072090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb4402a650 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.298+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b811f73620 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.299+0000 7fca297fa640 1 -- 172.21.15.63:0/3713072090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcb44045070 con 0x55b811f68bf0 2024-04-23T19:10:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.299+0000 7fca297fa640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca14036eb0 0x7fca14039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.299+0000 7fca297fa640 1 -- 172.21.15.63:0/3713072090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fcb44043070 con 0x55b811f68bf0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.299+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca14036eb0 0x7fca14039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.300+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca14036eb0 0x7fca14039370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b811f4d8d0 tx=0x7fcb3c033000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.300+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.300+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811f7be80 0x55b811f7e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.300+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f7ea90 con 0x55b811f7be80 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.300+0000 7fcb4b7c2640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811f7be80 0x55b811f7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.300+0000 7fcb4b7c2640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811f7be80 0x55b811f7e340 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 realm 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f7eec0 con 0x55b811f7be80 2024-04-23T19:10:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f7f790 con 0x55b811f7be80 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f7fb30 con 0x55b811f7be80 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.301+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.302+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.302+0000 7fcb5031f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.302+0000 7fcb5031f800 20 searching for the correct realm 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.302+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b811f820f0 0x55b811f844e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.302+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f84b10 con 0x55b811f820f0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.302+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b811f820f0 0x55b811f844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.302+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b811f820f0 0x55b811f844e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4c7c4640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcb440123a0 0x7fcb4402c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f84b10 con 0x7fcb440123a0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcb440123a0 0x7fcb4402c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcb440123a0 0x7fcb4402c790 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4bfc3640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcb3c03a040 0x7fcb3c03f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f84b10 con 0x7fcb3c03a040 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.303+0000 7fcb4b7c2640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcb3c03a040 0x7fcb3c03f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.304+0000 7fcb4b7c2640 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcb3c03a040 0x7fcb3c03f890 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.304+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.304+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.304+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.304+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb440123a0 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.305+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.306+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb440123a0 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.307+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.308+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.309+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.309+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.309+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.309+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb440123a0 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.309+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.309+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.310+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.310+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.310+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.310+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.310+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.310+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.311+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.311+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb440123a0 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.311+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.311+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f820f0 2024-04-23T19:10:26.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.311+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.311+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x7fcb3c03a040 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80380 con 0x55b811f7be80 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fcb4008a040 con 0x7fcb440123a0 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.312+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80830 con 0x55b811f7be80 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80c20 con 0x55b811f7be80 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f80c20 con 0x7fcb3c03a040 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 zone default found 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 4 Realm: () 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 zonegroup default 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f814f0 con 0x7fcb3c03a040 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.313+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f814f0 con 0x55b811f820f0 2024-04-23T19:10:26.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f86970 con 0x55b811f820f0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb5031f800 20 started sync module instance, tier type = 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb5031f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.314+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f85e10 con 0x7fcb440123a0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.315+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.315+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94249063831904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f86d10 con 0x7fcb440123a0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.315+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f875c0 con 0x55b811f7be80 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.316+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94249063831904] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.316+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.317+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94249063837872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f879b0 con 0x55b811f7be80 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.317+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f883f0 con 0x55b811f820f0 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.317+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94249063837872] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.318+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.318+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94249063841472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f887e0 con 0x55b811f820f0 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.318+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f89240 con 0x7fcb440123a0 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.319+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94249063841472] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.319+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.319+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94249063834112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f89630 con 0x7fcb440123a0 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.319+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8a070 con 0x55b811f7be80 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.320+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.320+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94249063848768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8a910 con 0x55b811f7be80 2024-04-23T19:10:26.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.320+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94249063834112] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.320+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f88d60 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.321+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.321+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94249063853728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f88d60 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.321+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8bd90 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.321+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94249063848768] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.322+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94249063853728] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.323+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.323+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94249063854784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8b8e0 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.323+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8c2a0 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94249063857648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8c6c0 con 0x7fcb3c03a040 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 20 add_watcher() i=0 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 20 add_watcher() i=1 2024-04-23T19:10:26.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 20 add_watcher() i=2 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 20 add_watcher() i=3 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 20 add_watcher() i=4 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 20 add_watcher() i=5 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94249063854784] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb5031f800 20 add_watcher() i=6 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.324+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94249063857648] v30'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.325+0000 7fcb5031f800 20 add_watcher() i=7 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.325+0000 7fcb5031f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.325+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8a460 con 0x55b811f7be80 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.325+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.325+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94249063858704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811f8a460 con 0x55b811f7be80 2024-04-23T19:10:26.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94249063858704] v30'50 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fca01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fcb5031f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fcb5031f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fcb5031f800 5 note: GC not initialized 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fca00fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fcb5031f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.326+0000 7fcb5031f800 20 Filter name: none 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fc97a7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc2b60 con 0x55b811f7be80 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc1300 con 0x55b811f7be80 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc1300 con 0x7fcb3c03a040 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.327+0000 7fcb5031f800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.328+0000 7fcb5031f800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.328+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc1bf0 con 0x55b811f7be80 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.328+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~391] v30'1 uv1 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb5031f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb5031f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb5031f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94249064077120 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc6450 con 0x55b811f7be80 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify (1) cookie 94249063848768 notify 128849018880 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55b811fc1f90 con 0x55b811f7be80 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fca2affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018880 cookie 94249063848768 notifier 4289 bl.length()=567 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fca2affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fca2affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fca2affd640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fca10003ea0 con 0x55b811f7be80 2024-04-23T19:10:26.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 notify.4 [notify cookie 94249064077120 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== watch-notify(notify_complete (2) cookie 94249064077120 notify 128849018880 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55b811fc25b0 con 0x55b811f7be80 2024-04-23T19:10:26.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.329+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc3f80 con 0x55b811f7be80 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v30'1 uv1 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb5031f800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb5031f800 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb5031f800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94249064076928 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc6450 con 0x7fcb440123a0 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== watch-notify(notify (1) cookie 94249063834112 notify 128849018880 ret 0) ==== 211+0+0 (crc 0 0 0) 0x7fcb3c05b040 con 0x7fcb440123a0 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fca2a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018880 cookie 94249063834112 notifier 4289 bl.length()=169 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fca2a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-23T19:10:26.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fca2a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(69 notify.3 [notify cookie 94249064076928 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fca2a7fc640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fc980003400 con 0x7fcb440123a0 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== watch-notify(notify_complete (2) cookie 94249064076928 notify 128849018880 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fcb3c033070 con 0x7fcb440123a0 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.331+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc1680 con 0x7fcb3c03a040 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.332+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v30'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.332+0000 7fcb5031f800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.332+0000 7fcb5031f800 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.332+0000 7fcb5031f800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94249064076928 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x7fcb440123a0 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== watch-notify(notify (1) cookie 94249063834112 notify 128849018881 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7fcb3c033070 con 0x7fcb440123a0 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fca2a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018881 cookie 94249063834112 notifier 4289 bl.length()=177 2024-04-23T19:10:26.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fca2a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-23T19:10:26.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fca2a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-23T19:10:26.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(72 notify.3 [notify cookie 94249064076928 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fca2a7fc640 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fc980003430 con 0x7fcb440123a0 2024-04-23T19:10:26.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== watch-notify(notify_complete (2) cookie 94249064076928 notify 128849018881 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55b811f43640 con 0x7fcb440123a0 2024-04-23T19:10:26.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.333+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.346 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:26.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.334+0000 7fca01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.334+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94249063858704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x55b811f7be80 2024-04-23T19:10:26.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.335+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94249063858704] v30'51 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.335+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94249063831904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x7fcb440123a0 2024-04-23T19:10:26.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.336+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94249063831904] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.336+0000 7fcb5031f800 20 remove_watcher() i=0 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.336+0000 7fcb5031f800 2 removed watcher, disabling cache 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.336+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94249063837872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x55b811f7be80 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.337+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94249063837872] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.337+0000 7fcb5031f800 20 remove_watcher() i=1 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.337+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94249063841472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x55b811f820f0 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.338+0000 7fcb4c7c4640 1 -- 172.21.15.63:0/3713072090 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94249063841472] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb44029070 con 0x55b811f820f0 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.338+0000 7fcb5031f800 20 remove_watcher() i=2 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.338+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94249063834112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x7fcb440123a0 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.339+0000 7fcb4bfc3640 1 -- 172.21.15.63:0/3713072090 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94249063834112] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb3c042040 con 0x7fcb440123a0 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.339+0000 7fcb5031f800 20 remove_watcher() i=3 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.339+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94249063848768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x55b811f7be80 2024-04-23T19:10:26.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.340+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94249063848768] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x55b811f7be80 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.340+0000 7fcb5031f800 20 remove_watcher() i=4 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.340+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94249063853728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x7fcb3c03a040 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.341+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94249063853728] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.341+0000 7fcb5031f800 20 remove_watcher() i=5 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.341+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94249063854784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x7fcb3c03a040 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.342+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94249063854784] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.342+0000 7fcb5031f800 20 remove_watcher() i=6 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.342+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94249063857648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b811fc55f0 con 0x7fcb3c03a040 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.343+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94249063857648] v30'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb4005a040 con 0x7fcb3c03a040 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.343+0000 7fcb5031f800 20 remove_watcher() i=7 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcb3c03a040 msgr2=0x7fcb3c03f890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcb3c03a040 0x7fcb3c03f890 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811f7be80 msgr2=0x55b811f7e340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811f7be80 0x55b811f7e340 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcb440123a0 msgr2=0x7fcb4402c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcb440123a0 0x7fcb4402c790 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b811f820f0 msgr2=0x55b811f844e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b811f820f0 0x55b811f844e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca14036eb0 msgr2=0x7fca14039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca14036eb0 0x7fca14039370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b811f4d8d0 tx=0x7fcb3c033000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb4b7c2640 1 -- 172.21.15.63:0/3713072090 reap_dead start 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 msgr2=0x55b811f701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f701a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001df0 tx=0x7fcb4404cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 shutdown_connections 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/3713072090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f68bf0 0x55b811f701a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 >> 172.21.15.63:0/3713072090 conn(0x55b811f64f50 msgr2=0x55b811f625b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 shutdown_connections 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/3713072090 wait complete. 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca24036eb0 msgr2=0x7fca24039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca24036eb0 0x7fca24039370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c02d040 tx=0x7fcb3c04b320 comp rx=0 tx=0).stop 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f5b910 msgr2=0x55b811f5bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f5b910 0x55b811f5bd30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55b811f415c0 tx=0x7fcb440178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 shutdown_connections 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca24036eb0 0x7fca24039370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519724219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811f5b910 0x55b811f5bd30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 >> 172.21.15.63:0/2519724219 conn(0x55b811f4f9b0 msgr2=0x55b811f4fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 shutdown_connections 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.344+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519724219 wait complete. 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b811f4ab80 msgr2=0x55b811f4cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b811f4ab80 0x55b811f4cf70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811e45500 msgr2=0x55b811f47400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811e45500 0x55b811f47400 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca34036e60 msgr2=0x7fca34039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca34036e60 0x7fca34039320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb40054000 tx=0x7fcb4004e000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 msgr2=0x55b811f401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f401c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c010 tx=0x7fcb3c000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 shutdown_connections 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b811f4ab80 0x55b811f4cf70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b811e45500 0x55b811f47400 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fca34036e60 0x7fca34039320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 --2- 172.21.15.63:0/2519691642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b811e439c0 0x55b811f401c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 >> 172.21.15.63:0/2519691642 conn(0x55b811e3f5a0 msgr2=0x55b811f327c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 shutdown_connections 2024-04-23T19:10:26.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.345+0000 7fcb5031f800 1 -- 172.21.15.63:0/2519691642 wait complete. 2024-04-23T19:10:26.357 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:26.358 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-23T19:10:26.358 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-23T19:10:26.358 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-04-23T19:10:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.406+0000 7f047eb2f800 1 Processor -- start 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.406+0000 7f047eb2f800 1 -- start start 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.406+0000 7f047eb2f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x562c6952ca70 0x562c6952ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.406+0000 7f047eb2f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562c693e07f0 con 0x562c6952d3d0 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.407+0000 7f047eb2f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x562c693cfe20 con 0x562c6952ca70 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.407+0000 7f047a7d1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x562c6952ca70 0x562c6952ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.407+0000 7f0479fd0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x562c6952d3d0 0x562c6952d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52676/0 (socket says 172.21.15.63:52676) 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.407+0000 7f0479fd0640 1 -- 172.21.15.63:0/1495742415 learned_addr learned my addr 172.21.15.63:0/1495742415 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.407+0000 7f047a7d1640 1 -- 172.21.15.63:0/1495742415 >> v1:172.21.15.16:6789/0 conn(0x562c6952d3d0 legacy=0x562c6952d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.407+0000 7f047a7d1640 1 -- 172.21.15.63:0/1495742415 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c693eeb50 con 0x562c6952ca70 2024-04-23T19:10:26.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.409+0000 7f047a7d1640 1 --2- 172.21.15.63:0/1495742415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c6952ce90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f046c00a470 tx=0x7f046c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303a004c1eebaebc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.409+0000 7f04797cf640 1 -- 172.21.15.63:0/1495742415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c057070 con 0x562c6952ca70 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.409+0000 7f04797cf640 1 -- 172.21.15.63:0/1495742415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c04adc0 con 0x562c6952ca70 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.409+0000 7f04797cf640 1 -- 172.21.15.63:0/1495742415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c052590 con 0x562c6952ca70 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.410+0000 7f047eb2f800 1 -- 172.21.15.63:0/1495742415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 msgr2=0x562c6952ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.410+0000 7f047eb2f800 1 --2- 172.21.15.63:0/1495742415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c6952ce90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f046c00a470 tx=0x7f046c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.410+0000 7f047eb2f800 1 -- 172.21.15.63:0/1495742415 shutdown_connections 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.410+0000 7f047eb2f800 1 --2- 172.21.15.63:0/1495742415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c6952ce90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.410+0000 7f047eb2f800 1 -- 172.21.15.63:0/1495742415 >> 172.21.15.63:0/1495742415 conn(0x562c69528650 msgr2=0x562c6952aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.410+0000 7f047eb2f800 1 -- 172.21.15.63:0/1495742415 shutdown_connections 2024-04-23T19:10:26.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.410+0000 7f047eb2f800 1 -- 172.21.15.63:0/1495742415 wait complete. 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.422+0000 7f047eb2f800 1 Processor -- start 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.422+0000 7f047eb2f800 1 -- start start 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.422+0000 7f047eb2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c696216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.422+0000 7f047eb2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c69479710 con 0x562c6952ca70 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f047a7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c696216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f047a7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c696216b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48300/0 (socket says 172.21.15.63:48300) 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f047a7d1640 1 -- 172.21.15.63:0/156277255 learned_addr learned my addr 172.21.15.63:0/156277255 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f047a7d1640 1 -- 172.21.15.63:0/156277255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c69621c80 con 0x562c6952ca70 2024-04-23T19:10:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f047a7d1640 1 --2- 172.21.15.63:0/156277255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c696216b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f046c055040 tx=0x7f046c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ef135cb60540ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f0368608640 1 -- 172.21.15.63:0/156277255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c057040 con 0x562c6952ca70 2024-04-23T19:10:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f0368608640 1 -- 172.21.15.63:0/156277255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c063070 con 0x562c6952ca70 2024-04-23T19:10:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.423+0000 7f0368608640 1 -- 172.21.15.63:0/156277255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c05d790 con 0x562c6952ca70 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 -- 172.21.15.63:0/156277255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 msgr2=0x562c696216b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 --2- 172.21.15.63:0/156277255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c696216b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f046c055040 tx=0x7f046c04c000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 -- 172.21.15.63:0/156277255 shutdown_connections 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 --2- 172.21.15.63:0/156277255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c696216b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 -- 172.21.15.63:0/156277255 >> 172.21.15.63:0/156277255 conn(0x562c69528650 msgr2=0x562c6961cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 -- 172.21.15.63:0/156277255 shutdown_connections 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 -- 172.21.15.63:0/156277255 wait complete. 2024-04-23T19:10:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.424+0000 7f047eb2f800 1 Processor -- start 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047eb2f800 1 -- start start 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047eb2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c69629160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047eb2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c6952e230 con 0x562c6952ca70 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047a7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c69629160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047a7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c69629160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48308/0 (socket says 172.21.15.63:48308) 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047a7d1640 1 -- 172.21.15.63:0/3995232372 learned_addr learned my addr 172.21.15.63:0/3995232372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047a7d1640 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c696296a0 con 0x562c6952ca70 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047a7d1640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c69629160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f046c060040 tx=0x7f046c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f0366e05640 1 -- 172.21.15.63:0/3995232372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c057050 con 0x562c6952ca70 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f0366e05640 1 -- 172.21.15.63:0/3995232372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c05dcd0 con 0x562c6952ca70 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f0366e05640 1 -- 172.21.15.63:0/3995232372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c052440 con 0x562c6952ca70 2024-04-23T19:10:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c69629930 con 0x562c6952ca70 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.425+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6962c5e0 con 0x562c6952ca70 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.426+0000 7f0366e05640 1 -- 172.21.15.63:0/3995232372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f046c052600 con 0x562c6952ca70 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.426+0000 7f0366e05640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036eb0 0x7f0360039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.426+0000 7f0366e05640 1 -- 172.21.15.63:0/3995232372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f046c08e350 con 0x562c6952ca70 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.426+0000 7f0479fd0640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036eb0 0x7f0360039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.427+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c6952d3d0 0x562c696303a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.427+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69630920 con 0x562c6952d3d0 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.427+0000 7f047afd2640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c6952d3d0 0x562c696303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.427+0000 7f0479fd0640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036eb0 0x7f0360039370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0470000f60 tx=0x7f04700012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.427+0000 7f047afd2640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c6952d3d0 0x562c696303a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.427+0000 7f047afd2640 1 -- 172.21.15.63:0/3995232372 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6952d3d0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.427+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c6962d970 con 0x562c6952d3d0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047afd2640 1 -- 172.21.15.63:0/3995232372 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6952d3d0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69632770 con 0x562c6952d3d0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047afd2640 1 -- 172.21.15.63:0/3995232372 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6952d3d0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69631ab0 con 0x562c6952d3d0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047afd2640 1 -- 172.21.15.63:0/3995232372 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6952d3d0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x562c69633b20 0x562c69635f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69636490 con 0x562c69633b20 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.428+0000 7f047a7d1640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x562c69633b20 0x562c69635f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.429+0000 7f047a7d1640 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x562c69633b20 0x562c69635f10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.429+0000 7f047a7d1640 1 -- 172.21.15.63:0/3995232372 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f046c078070 con 0x562c69633b20 2024-04-23T19:10:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.429+0000 7f047eb2f800 1 Processor -- start 2024-04-23T19:10:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.429+0000 7f047eb2f800 1 -- start start 2024-04-23T19:10:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047eb2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c696448b0 0x562c69644cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047eb2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c6941fe30 con 0x562c696448b0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047afd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c696448b0 0x562c69644cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047afd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c696448b0 0x562c69644cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48324/0 (socket says 172.21.15.63:48324) 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047afd2640 1 -- 172.21.15.63:0/810521119 learned_addr learned my addr 172.21.15.63:0/810521119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047afd2640 1 -- 172.21.15.63:0/810521119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c69645210 con 0x562c696448b0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047afd2640 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c696448b0 0x562c69644cd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x562c69630ce0 tx=0x7f04740178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f035ffff640 1 -- 172.21.15.63:0/810521119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474013440 con 0x562c696448b0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f035ffff640 1 -- 172.21.15.63:0/810521119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04740135e0 con 0x562c696448b0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f035ffff640 1 -- 172.21.15.63:0/810521119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04740138c0 con 0x562c696448b0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c69645c80 con 0x562c696448b0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.430+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c69646180 con 0x562c696448b0 2024-04-23T19:10:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.431+0000 7f035ffff640 1 -- 172.21.15.63:0/810521119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0474011040 con 0x562c696448b0 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.431+0000 7f035ffff640 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.431+0000 7f035ffff640 1 -- 172.21.15.63:0/810521119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f047401d070 con 0x562c696448b0 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.431+0000 7f047a7d1640 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047eb2f800 1 Processor -- start 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047a7d1640 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f046c05efd0 tx=0x7f046c08c630 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047eb2f800 1 -- start start 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047eb2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69652040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047eb2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c69621f40 con 0x562c69651c20 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047afd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69652040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047afd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69652040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48330/0 (socket says 172.21.15.63:48330) 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047afd2640 1 -- 172.21.15.63:0/4200940432 learned_addr learned my addr 172.21.15.63:0/4200940432 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047afd2640 1 -- 172.21.15.63:0/4200940432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c69652610 con 0x562c69651c20 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f047afd2640 1 --2- 172.21.15.63:0/4200940432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69652040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f047404cd50 tx=0x7f047404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2d9aae4f59d8cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.432+0000 7f035dffb640 1 -- 172.21.15.63:0/4200940432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474015560 con 0x562c69651c20 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f035dffb640 1 -- 172.21.15.63:0/4200940432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047402a350 con 0x562c69651c20 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f035dffb640 1 -- 172.21.15.63:0/4200940432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474012ca0 con 0x562c69651c20 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 -- 172.21.15.63:0/4200940432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 msgr2=0x562c69652040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 --2- 172.21.15.63:0/4200940432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69652040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f047404cd50 tx=0x7f047404d000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 -- 172.21.15.63:0/4200940432 shutdown_connections 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 --2- 172.21.15.63:0/4200940432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69652040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 -- 172.21.15.63:0/4200940432 >> 172.21.15.63:0/4200940432 conn(0x562c6964df30 msgr2=0x562c6964bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 -- 172.21.15.63:0/4200940432 shutdown_connections 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 -- 172.21.15.63:0/4200940432 wait complete. 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 Processor -- start 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 -- start start 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69659130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.433+0000 7f047eb2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c69652d60 con 0x562c69651c20 2024-04-23T19:10:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f047afd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69659130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f047afd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69659130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48332/0 (socket says 172.21.15.63:48332) 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 learned_addr learned my addr 172.21.15.63:0/2811962855 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c69659670 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f047afd2640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69659130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f047404bc20 tx=0x7f047404cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f034bfff640 1 -- 172.21.15.63:0/2811962855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474015950 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f034bfff640 1 -- 172.21.15.63:0/2811962855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047402a8e0 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f034bfff640 1 -- 172.21.15.63:0/2811962855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0474013c20 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c69659900 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.434+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6965c5b0 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.435+0000 7f034bfff640 1 -- 172.21.15.63:0/2811962855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f047404eb20 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.435+0000 7f034bfff640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.435+0000 7f034bfff640 1 -- 172.21.15.63:0/2811962855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f047400e040 con 0x562c69651c20 2024-04-23T19:10:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.435+0000 7f047a7d1640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047a7d1640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x562c69636870 tx=0x7f046c083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c69664c50 0x562c69667110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69667860 con 0x562c69664c50 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f0479fd0640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c69664c50 0x562c69667110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f0479fd0640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c69664c50 0x562c69667110 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0470009050 con 0x562c69664c50 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047eb2f800 20 realm 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.436+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c696640a0 con 0x562c69664c50 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0470009050 con 0x562c69664c50 2024-04-23T19:10:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69668350 con 0x562c69664c50 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0470009050 con 0x562c69664c50 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69668c20 con 0x562c69664c50 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0470009050 con 0x562c69664c50 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69668fc0 con 0x562c69664c50 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0470009050 con 0x562c69664c50 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 20 searching for the correct realm 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.437+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562c6966aea0 0x562c6966d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.438+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.438+0000 7f047afd2640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562c6966aea0 0x562c6966d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.438+0000 7f047afd2640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562c6966aea0 0x562c6966d290 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.438+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.438+0000 7f047afd2640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0474001c20 0x7f0474037450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.438+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f0474001c20 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.438+0000 7f047a7d1640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0474001c20 0x7f0474037450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.439+0000 7f047a7d1640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0474001c20 0x7f0474037450 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.439+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.439+0000 7f047a7d1640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c05e900 0x7f046c05b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.439+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.439+0000 7f0479fd0640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c05e900 0x7f046c05b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.440+0000 7f0479fd0640 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c05e900 0x7f046c05b370 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.440+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0470009050 con 0x7f046c05e900 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.440+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.440+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0470009050 con 0x562c69664c50 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.440+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.440+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.440+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f0474001c20 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.441+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.442+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.442+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.442+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.442+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.442+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.442+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f0474001c20 2024-04-23T19:10:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.443+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.443+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.443+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.443+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.443+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.443+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.443+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.444+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.445+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.445+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.445+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.445+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f0474001c20 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.445+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.445+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.446+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.446+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.446+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.446+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.446+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.446+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.447+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.447+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f0474001c20 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.447+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.447+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c6966aea0 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.447+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.447+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f046c05e900 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x562c69664c50 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562c6966d940 con 0x7f0474001c20 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69669940 con 0x562c69664c50 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.448+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c6966a1e0 con 0x562c69664c50 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c6966ddc0 con 0x7f046c05e900 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 zone default found 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 4 Realm: () 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 zonegroup default 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c6966e270 con 0x7f046c05e900 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.449+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c6966fc50 con 0x562c6966aea0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c69670100 con 0x562c6966aea0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047eb2f800 20 started sync module instance, tier type = 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047eb2f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.450+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c6966f0e0 con 0x7f0474001c20 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.451+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.451+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94748746904784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c6966ed40 con 0x7f0474001c20 2024-04-23T19:10:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.451+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69671030 con 0x562c69664c50 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.452+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94748746904784] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.452+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.452+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94748746910592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69671450 con 0x562c69664c50 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.452+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69671e10 con 0x562c6966aea0 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.453+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94748746910592] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.453+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.453+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94748746914144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696721b0 con 0x562c6966aea0 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.453+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69672c20 con 0x7f0474001c20 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.454+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94748746914144] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.455+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.455+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94748746917616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69673010 con 0x7f0474001c20 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.455+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69673a70 con 0x562c69664c50 2024-04-23T19:10:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.456+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.456+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94748746917616] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.456+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94748746921296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69674310 con 0x562c69664c50 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.456+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69670fa0 con 0x7f046c05e900 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.457+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.457+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94748746926208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69670fa0 con 0x7f046c05e900 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.457+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696757b0 con 0x7f046c05e900 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.457+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94748746921296] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.458+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94748746926208] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.459+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.459+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94748746927264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696752c0 con 0x7f046c05e900 2024-04-23T19:10:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.459+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69675c80 con 0x7f046c05e900 2024-04-23T19:10:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94748746930128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696760a0 con 0x7f046c05e900 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94748746927264] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 20 add_watcher() i=0 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 20 add_watcher() i=1 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 20 add_watcher() i=2 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 20 add_watcher() i=3 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 20 add_watcher() i=4 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 20 add_watcher() i=5 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.460+0000 7f047eb2f800 20 add_watcher() i=6 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.461+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94748746930128] v30'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.461+0000 7f047eb2f800 20 add_watcher() i=7 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.461+0000 7f047eb2f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.461+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c696790d0 con 0x562c69664c50 2024-04-23T19:10:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.461+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.461+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94748746931184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696764f0 con 0x562c69664c50 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.462+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94748746931184] v30'52 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.462+0000 7f0330fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.462+0000 7f047eb2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.462+0000 7f047eb2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.462+0000 7f047eb2f800 5 note: GC not initialized 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f02ab7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 20 Filter name: none 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f02a97fa640 20 reqs_thread_entry: start 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c696ac520 con 0x7f046c05e900 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.463+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c696aacc0 con 0x562c69664c50 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) ==== 197+0+23 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 20 rados_obj.operate() r=0 bl.length=7 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562c696ab0b0 con 0x562c69664c50 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.464+0000 7f047eb2f800 20 get_user_info_from_index found 0 xattrs 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.465+0000 7f0330fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.465+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94748746931184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696611f0 con 0x562c69664c50 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.466+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94748746931184] v30'53 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.466+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94748746904784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69675bf0 con 0x7f0474001c20 2024-04-23T19:10:26.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.467+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94748746904784] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.467+0000 7f047eb2f800 20 remove_watcher() i=0 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.467+0000 7f047eb2f800 2 removed watcher, disabling cache 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.467+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94748746910592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c6966f4d0 con 0x562c69664c50 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.468+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94748746910592] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.468+0000 7f047eb2f800 20 remove_watcher() i=1 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.468+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94748746914144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c6966f4d0 con 0x562c6966aea0 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.469+0000 7f047afd2640 1 -- 172.21.15.63:0/2811962855 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94748746914144] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047400a4a0 con 0x562c6966aea0 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.469+0000 7f047eb2f800 20 remove_watcher() i=2 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.469+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94748746917616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c69671960 con 0x7f0474001c20 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.470+0000 7f047a7d1640 1 -- 172.21.15.63:0/2811962855 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94748746917616] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f046c07e050 con 0x7f0474001c20 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.470+0000 7f047eb2f800 20 remove_watcher() i=3 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.470+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94748746921296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c6966f4d0 con 0x562c69664c50 2024-04-23T19:10:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.471+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94748746921296] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x562c69664c50 2024-04-23T19:10:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.471+0000 7f047eb2f800 20 remove_watcher() i=4 2024-04-23T19:10:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.471+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94748746926208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696726f0 con 0x7f046c05e900 2024-04-23T19:10:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.472+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94748746926208] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.472+0000 7f047eb2f800 20 remove_watcher() i=5 2024-04-23T19:10:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.472+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94748746927264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c6966f4d0 con 0x7f046c05e900 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.473+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94748746927264] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.473+0000 7f047eb2f800 20 remove_watcher() i=6 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.473+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94748746930128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562c696726f0 con 0x7f046c05e900 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94748746930128] v30'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f047002a040 con 0x7f046c05e900 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 20 remove_watcher() i=7 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c05e900 msgr2=0x7f046c05b370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f046c05e900 0x7f046c05b370 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c69664c50 msgr2=0x562c69667110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c69664c50 0x562c69667110 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0474001c20 msgr2=0x7f0474037450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0474001c20 0x7f0474037450 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562c6966aea0 msgr2=0x562c6966d290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562c6966aea0 0x562c6966d290 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 msgr2=0x7f0344039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0344036eb0 0x7f0344039370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x562c69636870 tx=0x7f046c083000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f0479fd0640 1 -- 172.21.15.63:0/2811962855 reap_dead start 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 msgr2=0x562c69659130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69659130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f047404bc20 tx=0x7f047404cce0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 shutdown_connections 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 --2- 172.21.15.63:0/2811962855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c69651c20 0x562c69659130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.474+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 >> 172.21.15.63:0/2811962855 conn(0x562c6964df30 msgr2=0x562c6964b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 shutdown_connections 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/2811962855 wait complete. 2024-04-23T19:10:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 msgr2=0x7f0354039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f046c05efd0 tx=0x7f046c08c630 comp rx=0 tx=0).stop 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c696448b0 msgr2=0x562c69644cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c696448b0 0x562c69644cd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x562c69630ce0 tx=0x7f04740178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 shutdown_connections 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0354036e60 0x7f0354039320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/810521119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c696448b0 0x562c69644cd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 >> 172.21.15.63:0/810521119 conn(0x562c69638950 msgr2=0x562c69638d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 shutdown_connections 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/810521119 wait complete. 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x562c69633b20 msgr2=0x562c69635f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x562c69633b20 0x562c69635f10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c6952d3d0 msgr2=0x562c696303a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c6952d3d0 0x562c696303a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036eb0 msgr2=0x7f0360039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036eb0 0x7f0360039370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0470000f60 tx=0x7f04700012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 msgr2=0x562c69629160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c69629160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f046c060040 tx=0x7f046c002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 shutdown_connections 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x562c69633b20 0x562c69635f10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562c6952d3d0 0x562c696303a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0360036eb0 0x7f0360039370 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 --2- 172.21.15.63:0/3995232372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c6952ca70 0x562c69629160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 >> 172.21.15.63:0/3995232372 conn(0x562c69528650 msgr2=0x562c6961b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 shutdown_connections 2024-04-23T19:10:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.475+0000 7f047eb2f800 1 -- 172.21.15.63:0/3995232372 wait complete. 2024-04-23T19:10:26.485 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-04-23T19:10:26.485 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:26.485 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:26.486 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:26.529 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:26.529 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:26.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96ca03f800 1 Processor -- start 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96ca03f800 1 -- start start 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96ca03f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5588e23eb6a0 0x5588e23eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96ca03f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5588e22997f0 con 0x5588e23e7b90 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96ca03f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5588e2288e20 con 0x5588e23eb6a0 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96c733d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5588e23eb6a0 0x5588e23eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96c7b3e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5588e23e7b90 0x5588e23e9f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52680/0 (socket says 172.21.15.63:52680) 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96c7b3e640 1 -- 172.21.15.63:0/2196786369 learned_addr learned my addr 172.21.15.63:0/2196786369 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96c733d640 1 -- 172.21.15.63:0/2196786369 >> v1:172.21.15.16:6789/0 conn(0x5588e23e7b90 legacy=0x5588e23e9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.530+0000 7f96c733d640 1 -- 172.21.15.63:0/2196786369 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e22a7b50 con 0x5588e23eb6a0 2024-04-23T19:10:26.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.532+0000 7f96c733d640 1 --2- 172.21.15.63:0/2196786369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23eb6a0 0x5588e23eda90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00a470 tx=0x7f96bc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3668d8bd616956 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.532+0000 7f96c6b3c640 1 -- 172.21.15.63:0/2196786369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96bc057070 con 0x5588e23eb6a0 2024-04-23T19:10:26.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.532+0000 7f96c6b3c640 1 -- 172.21.15.63:0/2196786369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc052570 con 0x5588e23eb6a0 2024-04-23T19:10:26.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.532+0000 7f96c6b3c640 1 -- 172.21.15.63:0/2196786369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96bc052870 con 0x5588e23eb6a0 2024-04-23T19:10:26.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.533+0000 7f96ca03f800 1 -- 172.21.15.63:0/2196786369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23eb6a0 msgr2=0x5588e23eda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.533+0000 7f96ca03f800 1 --2- 172.21.15.63:0/2196786369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23eb6a0 0x5588e23eda90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00a470 tx=0x7f96bc01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:26.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.533+0000 7f96ca03f800 1 -- 172.21.15.63:0/2196786369 shutdown_connections 2024-04-23T19:10:26.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.533+0000 7f96ca03f800 1 --2- 172.21.15.63:0/2196786369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23eb6a0 0x5588e23eda90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.533+0000 7f96ca03f800 1 -- 172.21.15.63:0/2196786369 >> 172.21.15.63:0/2196786369 conn(0x5588e23e1600 msgr2=0x5588e23e3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.533+0000 7f96ca03f800 1 -- 172.21.15.63:0/2196786369 shutdown_connections 2024-04-23T19:10:26.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.533+0000 7f96ca03f800 1 -- 172.21.15.63:0/2196786369 wait complete. 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.546+0000 7f96ca03f800 1 Processor -- start 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.546+0000 7f96ca03f800 1 -- start start 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.546+0000 7f96ca03f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24d8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.546+0000 7f96ca03f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5588e23326f0 con 0x5588e23e7b90 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f96c7b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24d8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f96c7b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24d8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48354/0 (socket says 172.21.15.63:48354) 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f96c7b3e640 1 -- 172.21.15.63:0/1782934584 learned_addr learned my addr 172.21.15.63:0/1782934584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f96c7b3e640 1 -- 172.21.15.63:0/1782934584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e24d8950 con 0x5588e23e7b90 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/1782934584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24d8380 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f96b8001de0 tx=0x7f96b800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cef46ea2ea89b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f95b5e0b640 1 -- 172.21.15.63:0/1782934584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b8012020 con 0x5588e23e7b90 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f95b5e0b640 1 -- 172.21.15.63:0/1782934584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b800e5a0 con 0x5588e23e7b90 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.547+0000 7f95b5e0b640 1 -- 172.21.15.63:0/1782934584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b800e8c0 con 0x5588e23e7b90 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 -- 172.21.15.63:0/1782934584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 msgr2=0x5588e24d8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1782934584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24d8380 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f96b8001de0 tx=0x7f96b800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 -- 172.21.15.63:0/1782934584 shutdown_connections 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1782934584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24d8380 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 -- 172.21.15.63:0/1782934584 >> 172.21.15.63:0/1782934584 conn(0x5588e23e1600 msgr2=0x5588e24d3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 -- 172.21.15.63:0/1782934584 shutdown_connections 2024-04-23T19:10:26.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 -- 172.21.15.63:0/1782934584 wait complete. 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 Processor -- start 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.548+0000 7f96ca03f800 1 -- start start 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96ca03f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96ca03f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5588e23ee2d0 con 0x5588e23e7b90 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96c7b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96c7b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24dfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48370/0 (socket says 172.21.15.63:48370) 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96c7b3e640 1 -- 172.21.15.63:0/1040088592 learned_addr learned my addr 172.21.15.63:0/1040088592 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96c7b3e640 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e24e0400 con 0x5588e23e7b90 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24dfec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c010 tx=0x7f96b8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f95affff640 1 -- 172.21.15.63:0/1040088592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b8012070 con 0x5588e23e7b90 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f95affff640 1 -- 172.21.15.63:0/1040088592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b801a7a0 con 0x5588e23e7b90 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588e24e0690 con 0x5588e23e7b90 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.549+0000 7f95affff640 1 -- 172.21.15.63:0/1040088592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b801aa80 con 0x5588e23e7b90 2024-04-23T19:10:26.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.550+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e24e3340 con 0x5588e23e7b90 2024-04-23T19:10:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.550+0000 7f95affff640 1 -- 172.21.15.63:0/1040088592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f96b8015020 con 0x5588e23e7b90 2024-04-23T19:10:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.550+0000 7f95affff640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95b0036e60 0x7f95b0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.550+0000 7f95affff640 1 -- 172.21.15.63:0/1040088592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f96b8020070 con 0x5588e23e7b90 2024-04-23T19:10:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96c733d640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95b0036e60 0x7f95b0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e23eb6a0 0x5588e24e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e24e7680 con 0x5588e23eb6a0 2024-04-23T19:10:26.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96c833f640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e23eb6a0 0x5588e24e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96c733d640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95b0036e60 0x7f95b0039320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f96bc054000 tx=0x7f96bc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96c833f640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e23eb6a0 0x5588e24e7100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96c833f640 1 -- 172.21.15.63:0/1040088592 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96c000a4a0 con 0x5588e23eb6a0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.551+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e24e46d0 con 0x5588e23eb6a0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.552+0000 7f96c833f640 1 -- 172.21.15.63:0/1040088592 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f96c000a4a0 con 0x5588e23eb6a0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.552+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e24e94d0 con 0x5588e23eb6a0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.552+0000 7f96c833f640 1 -- 172.21.15.63:0/1040088592 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f96c000a4a0 con 0x5588e23eb6a0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.552+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e24e8810 con 0x5588e23eb6a0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.552+0000 7f96c833f640 1 -- 172.21.15.63:0/1040088592 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f96c000a4a0 con 0x5588e23eb6a0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.552+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5588e24ea880 0x5588e24ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.552+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e24ed1f0 con 0x5588e24ea880 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.553+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5588e24ea880 0x5588e24ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.553+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5588e24ea880 0x5588e24ecc70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.553+0000 7f96c7b3e640 1 -- 172.21.15.63:0/1040088592 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f96b802b070 con 0x5588e24ea880 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96ca03f800 1 Processor -- start 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96ca03f800 1 -- start start 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96ca03f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e24fb610 0x5588e24fba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96ca03f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5588e22d8e30 con 0x5588e24fb610 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96c833f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e24fb610 0x5588e24fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96c833f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e24fb610 0x5588e24fba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48378/0 (socket says 172.21.15.63:48378) 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96c833f640 1 -- 172.21.15.63:0/816690616 learned_addr learned my addr 172.21.15.63:0/816690616 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.554+0000 7f96c833f640 1 -- 172.21.15.63:0/816690616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e24fbf70 con 0x5588e24fb610 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.555+0000 7f96c833f640 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e24fb610 0x5588e24fba30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5588e24e7a40 tx=0x7f96c00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.555+0000 7f95ad7fa640 1 -- 172.21.15.63:0/816690616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c0013440 con 0x5588e24fb610 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.555+0000 7f95ad7fa640 1 -- 172.21.15.63:0/816690616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c00135e0 con 0x5588e24fb610 2024-04-23T19:10:26.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.555+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588e24fc9e0 con 0x5588e24fb610 2024-04-23T19:10:26.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.555+0000 7f95ad7fa640 1 -- 172.21.15.63:0/816690616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c00138c0 con 0x5588e24fb610 2024-04-23T19:10:26.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.555+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e24fcee0 con 0x5588e24fb610 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.555+0000 7f95ad7fa640 1 -- 172.21.15.63:0/816690616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f96c0011040 con 0x5588e24fb610 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f95ad7fa640 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95a0036eb0 0x7f95a0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f95ad7fa640 1 -- 172.21.15.63:0/816690616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f96c001d070 con 0x5588e24fb610 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95a0036eb0 0x7f95a0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f96ca03f800 1 Processor -- start 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f96ca03f800 1 -- start start 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95a0036eb0 0x7f95a0039370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96b8000fd0 tx=0x7f96b804afc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f96ca03f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e2508d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f96ca03f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5588e24d8cd0 con 0x5588e2508920 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.556+0000 7f96c833f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e2508d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f96c833f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e2508d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48380/0 (socket says 172.21.15.63:48380) 2024-04-23T19:10:26.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f96c833f640 1 -- 172.21.15.63:0/602873476 learned_addr learned my addr 172.21.15.63:0/602873476 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f96c833f640 1 -- 172.21.15.63:0/602873476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e2509310 con 0x5588e2508920 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f96c833f640 1 --2- 172.21.15.63:0/602873476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e2508d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f96c004cc70 tx=0x7f96c0012860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e139cb11adb34d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f95a6ffd640 1 -- 172.21.15.63:0/602873476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c0042020 con 0x5588e2508920 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f95a6ffd640 1 -- 172.21.15.63:0/602873476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c0021a50 con 0x5588e2508920 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f95a6ffd640 1 -- 172.21.15.63:0/602873476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c0001310 con 0x5588e2508920 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f96ca03f800 1 -- 172.21.15.63:0/602873476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 msgr2=0x5588e2508d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.557+0000 7f96ca03f800 1 --2- 172.21.15.63:0/602873476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e2508d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f96c004cc70 tx=0x7f96c0012860 comp rx=0 tx=0).stop 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 -- 172.21.15.63:0/602873476 shutdown_connections 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 --2- 172.21.15.63:0/602873476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e2508d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 -- 172.21.15.63:0/602873476 >> 172.21.15.63:0/602873476 conn(0x5588e2504c30 msgr2=0x5588e2502c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 -- 172.21.15.63:0/602873476 shutdown_connections 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 -- 172.21.15.63:0/602873476 wait complete. 2024-04-23T19:10:26.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 Processor -- start 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 -- start start 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e23e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96ca03f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5588e2509a60 con 0x5588e2508920 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96c833f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e23e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96c833f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e23e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48390/0 (socket says 172.21.15.63:48390) 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.558+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 learned_addr learned my addr 172.21.15.63:0/259820364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.559+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e23e5ba0 con 0x5588e2508920 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.559+0000 7f96c833f640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e23e74b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96c0001bc0 tx=0x7f96c004cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.559+0000 7f95a57fa640 1 -- 172.21.15.63:0/259820364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c004e110 con 0x5588e2508920 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.559+0000 7f95a57fa640 1 -- 172.21.15.63:0/259820364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c0030030 con 0x5588e2508920 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.559+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588e23e5e30 con 0x5588e2508920 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.559+0000 7f95a57fa640 1 -- 172.21.15.63:0/259820364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c004e460 con 0x5588e2508920 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.559+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e23e62d0 con 0x5588e2508920 2024-04-23T19:10:26.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f95a57fa640 1 -- 172.21.15.63:0/259820364 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f96c002a350 con 0x5588e2508920 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f95a57fa640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9590036eb0 0x7f9590039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f95a57fa640 1 -- 172.21.15.63:0/259820364 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f96c000e040 con 0x5588e2508920 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9590036eb0 0x7f9590039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e251ddd0 0x5588e2520290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e25209e0 con 0x5588e251ddd0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f96c733d640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e251ddd0 0x5588e2520290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.560+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9590036eb0 0x7f9590039370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f96b8019f60 tx=0x7f96b8032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96c733d640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e251ddd0 0x5588e2520290 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 20 realm 2024-04-23T19:10:26.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2520db0 con 0x5588e251ddd0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2521650 con 0x5588e251ddd0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.561+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e25219f0 con 0x5588e251ddd0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96ca03f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e25212a0 con 0x5588e251ddd0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96ca03f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96ca03f800 20 searching for the correct realm 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5588e2523ea0 0x5588e2526290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96c833f640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5588e2523ea0 0x5588e2526290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.562+0000 7f96c833f640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5588e2523ea0 0x5588e2526290 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c833f640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f96c00123a0 0x7f96c004e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96c00123a0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f96c00123a0 0x7f96c004e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f96c00123a0 0x7f96c004e980 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c7b3e640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f96b8017050 0x7f96b800d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.563+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.564+0000 7f96c733d640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f96b8017050 0x7f96b800d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.564+0000 7f96c733d640 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f96b8017050 0x7f96b800d3d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.564+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.564+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.565+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96c00123a0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.566+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96c00123a0 2024-04-23T19:10:26.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.567+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.568+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.569+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.569+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.569+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.569+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96c00123a0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.570+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96c00123a0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e2523ea0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.571+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96b8017050 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.572+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.572+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x5588e251ddd0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.572+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.572+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526960 con 0x7f96c00123a0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2522940 con 0x5588e251ddd0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2522580 con 0x5588e251ddd0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e25232a0 con 0x7f96b8017050 2024-04-23T19:10:26.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 zone default found 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 4 Realm: () 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 zonegroup default 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.573+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2526e30 con 0x7f96b8017050 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2528cc0 con 0x5588e2523ea0 2024-04-23T19:10:26.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2529060 con 0x5588e2523ea0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 20 started sync module instance, tier type = 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.574+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e2528100 con 0x7f96c00123a0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.575+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.575+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94046400972064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e25298b0 con 0x7f96c00123a0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.575+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252a100 con 0x5588e251ddd0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.576+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94046400972064] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.576+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.576+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94046400975872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252a140 con 0x5588e251ddd0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.576+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252ac50 con 0x5588e2523ea0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.577+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94046400975872] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.578+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.578+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94046400980736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252b040 con 0x5588e2523ea0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.578+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252baa0 con 0x7f96c00123a0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.579+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.579+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94046400980736] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.579+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94046400984448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252c340 con 0x7f96c00123a0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.579+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e2529c90 con 0x5588e251ddd0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.580+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.580+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94046400989488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252d200 con 0x5588e251ddd0 2024-04-23T19:10:26.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.580+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252dc00 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.580+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94046400984448] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.581+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.581+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94046400986768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252dbc0 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.581+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94046400989488] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.581+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252cd50 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.582+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(56 notify.6 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.582+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94046400990544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252e610 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.582+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252f010 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.582+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94046400986768] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94046400993040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252efd0 con 0x7f96b8017050 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=0 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=1 2024-04-23T19:10:26.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=2 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=3 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=4 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=5 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94046400990544] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=6 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94046400993040] v30'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 20 add_watcher() i=7 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.583+0000 7f96ca03f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.584+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e252efd0 con 0x5588e251ddd0 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.584+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.584+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94046400998176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252ef40 con 0x5588e251ddd0 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94046400998176] v30'54 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f957dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f96ca03f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f96ca03f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f96ca03f800 5 note: GC not initialized 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f957cfe1640 20 reqs_thread_entry: start 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f96ca03f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.585+0000 7f96ca03f800 20 Filter name: none 2024-04-23T19:10:26.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.586+0000 7f94f67fc640 20 reqs_thread_entry: start 2024-04-23T19:10:26.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.586+0000 7f96ca03f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.586+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588e2564db0 con 0x5588e251ddd0 2024-04-23T19:10:26.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.586+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.586+0000 7f96ca03f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:26.596 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:26.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.587+0000 7f957dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.587+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94046400998176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e2563d00 con 0x5588e251ddd0 2024-04-23T19:10:26.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.588+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94046400998176] v30'55 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.588+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94046400972064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252eb20 con 0x7f96c00123a0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.589+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94046400972064] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.589+0000 7f96ca03f800 20 remove_watcher() i=0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.589+0000 7f96ca03f800 2 removed watcher, disabling cache 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.589+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94046400975872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252eb20 con 0x5588e251ddd0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.590+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94046400975872] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.590+0000 7f96ca03f800 20 remove_watcher() i=1 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.590+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94046400980736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252eb20 con 0x5588e2523ea0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.591+0000 7f96c833f640 1 -- 172.21.15.63:0/259820364 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94046400980736] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c0057040 con 0x5588e2523ea0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.591+0000 7f96ca03f800 20 remove_watcher() i=2 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.591+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94046400984448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252eb20 con 0x7f96c00123a0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.592+0000 7f96c7b3e640 1 -- 172.21.15.63:0/259820364 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94046400984448] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b8042040 con 0x7f96c00123a0 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.592+0000 7f96ca03f800 20 remove_watcher() i=3 2024-04-23T19:10:26.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.592+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94046400989488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252b580 con 0x5588e251ddd0 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.593+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94046400989488] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x5588e251ddd0 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.593+0000 7f96ca03f800 20 remove_watcher() i=4 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.593+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94046400986768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252eb20 con 0x7f96b8017050 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.594+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94046400986768] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.594+0000 7f96ca03f800 20 remove_watcher() i=5 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.594+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94046400990544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252c930 con 0x7f96b8017050 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.595+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94046400990544] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.595+0000 7f96ca03f800 20 remove_watcher() i=6 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.595+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94046400993040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588e252b580 con 0x7f96b8017050 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94046400993040] v30'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc05a040 con 0x7f96b8017050 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 20 remove_watcher() i=7 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f96b8017050 msgr2=0x7f96b800d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f96b8017050 0x7f96b800d3d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e251ddd0 msgr2=0x5588e2520290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e251ddd0 0x5588e2520290 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f96c00123a0 msgr2=0x7f96c004e980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f96c00123a0 0x7f96c004e980 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5588e2523ea0 msgr2=0x5588e2526290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5588e2523ea0 0x5588e2526290 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9590036eb0 msgr2=0x7f9590039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9590036eb0 0x7f9590039370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f96b8019f60 tx=0x7f96b8032000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 msgr2=0x5588e23e74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/259820364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e2508920 0x5588e23e74b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96c0001bc0 tx=0x7f96c004cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 reap_dead start 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96c733d640 1 -- 172.21.15.63:0/259820364 reap_dead start 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 shutdown_connections 2024-04-23T19:10:26.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 >> 172.21.15.63:0/259820364 conn(0x5588e2504c30 msgr2=0x5588e2502290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 shutdown_connections 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/259820364 wait complete. 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95a0036eb0 msgr2=0x7f95a0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95a0036eb0 0x7f95a0039370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96b8000fd0 tx=0x7f96b804afc0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e24fb610 msgr2=0x5588e24fba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e24fb610 0x5588e24fba30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5588e24e7a40 tx=0x7f96c00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 shutdown_connections 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95a0036eb0 0x7f95a0039370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 --2- 172.21.15.63:0/816690616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e24fb610 0x5588e24fba30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 >> 172.21.15.63:0/816690616 conn(0x5588e24ef6b0 msgr2=0x5588e24efab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 shutdown_connections 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.596+0000 7f96ca03f800 1 -- 172.21.15.63:0/816690616 wait complete. 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5588e24ea880 msgr2=0x5588e24ecc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5588e24ea880 0x5588e24ecc70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e23eb6a0 msgr2=0x5588e24e7100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e23eb6a0 0x5588e24e7100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95b0036e60 msgr2=0x7f95b0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95b0036e60 0x7f95b0039320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f96bc054000 tx=0x7f96bc04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 msgr2=0x5588e24dfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24dfec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c010 tx=0x7f96b8000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 shutdown_connections 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5588e24ea880 0x5588e24ecc70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588e23eb6a0 0x5588e24e7100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f95b0036e60 0x7f95b0039320 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 --2- 172.21.15.63:0/1040088592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e23e7b90 0x5588e24dfec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 >> 172.21.15.63:0/1040088592 conn(0x5588e23e1600 msgr2=0x5588e24d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 shutdown_connections 2024-04-23T19:10:26.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.597+0000 7f96ca03f800 1 -- 172.21.15.63:0/1040088592 wait complete. 2024-04-23T19:10:26.608 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:26.608 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-23T19:10:26.608 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-23T19:10:26.608 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-23T19:10:26.652 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:26.652 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.652+0000 7f5e44346800 1 Processor -- start 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e44346800 1 -- start start 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e44346800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561c819fd750 0x561c819fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e44346800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561c818ae7f0 con 0x561c819fe0b0 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e44346800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x561c8189de20 con 0x561c819fd750 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e41e4c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561c819fd750 0x561c819fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e4164b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x561c819fe0b0 0x561c819fa1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52694/0 (socket says 172.21.15.63:52694) 2024-04-23T19:10:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e4164b640 1 -- 172.21.15.63:0/2802037099 learned_addr learned my addr 172.21.15.63:0/2802037099 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e41e4c640 1 -- 172.21.15.63:0/2802037099 >> v1:172.21.15.16:6789/0 conn(0x561c819fe0b0 legacy=0x561c819fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.653+0000 7f5e41e4c640 1 -- 172.21.15.63:0/2802037099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c818bcb50 con 0x561c819fd750 2024-04-23T19:10:26.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.655+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/2802037099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c819fdb70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400a470 tx=0x7f5e3401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab24185af1048d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.655+0000 7f5e40e4a640 1 -- 172.21.15.63:0/2802037099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34057070 con 0x561c819fd750 2024-04-23T19:10:26.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.655+0000 7f5e40e4a640 1 -- 172.21.15.63:0/2802037099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34052570 con 0x561c819fd750 2024-04-23T19:10:26.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.655+0000 7f5e40e4a640 1 -- 172.21.15.63:0/2802037099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34052870 con 0x561c819fd750 2024-04-23T19:10:26.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.656+0000 7f5e44346800 1 -- 172.21.15.63:0/2802037099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 msgr2=0x561c819fdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.656+0000 7f5e44346800 1 --2- 172.21.15.63:0/2802037099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c819fdb70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400a470 tx=0x7f5e3401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:26.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.656+0000 7f5e44346800 1 -- 172.21.15.63:0/2802037099 shutdown_connections 2024-04-23T19:10:26.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.656+0000 7f5e44346800 1 --2- 172.21.15.63:0/2802037099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c819fdb70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.656+0000 7f5e44346800 1 -- 172.21.15.63:0/2802037099 >> 172.21.15.63:0/2802037099 conn(0x561c819f5db0 msgr2=0x561c819f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.656+0000 7f5e44346800 1 -- 172.21.15.63:0/2802037099 shutdown_connections 2024-04-23T19:10:26.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.656+0000 7f5e44346800 1 -- 172.21.15.63:0/2802037099 wait complete. 2024-04-23T19:10:26.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.669+0000 7f5e44346800 1 Processor -- start 2024-04-23T19:10:26.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.669+0000 7f5e44346800 1 -- start start 2024-04-23T19:10:26.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.669+0000 7f5e44346800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81aeda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.669+0000 7f5e44346800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c819476f0 con 0x561c819fd750 2024-04-23T19:10:26.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.669+0000 7f5e41e4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81aeda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.669+0000 7f5e41e4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81aeda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48408/0 (socket says 172.21.15.63:48408) 2024-04-23T19:10:26.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.669+0000 7f5e41e4c640 1 -- 172.21.15.63:0/888750517 learned_addr learned my addr 172.21.15.63:0/888750517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.670+0000 7f5e41e4c640 1 -- 172.21.15.63:0/888750517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c81aee020 con 0x561c819fd750 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.670+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/888750517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81aeda50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e34055040 tx=0x7f5e3404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3922000ae9da6215 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.670+0000 7f5d2fe07640 1 -- 172.21.15.63:0/888750517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34057040 con 0x561c819fd750 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.670+0000 7f5d2fe07640 1 -- 172.21.15.63:0/888750517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34052990 con 0x561c819fd750 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.670+0000 7f5d2fe07640 1 -- 172.21.15.63:0/888750517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34052c70 con 0x561c819fd750 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 -- 172.21.15.63:0/888750517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 msgr2=0x561c81aeda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 --2- 172.21.15.63:0/888750517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81aeda50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e34055040 tx=0x7f5e3404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 -- 172.21.15.63:0/888750517 shutdown_connections 2024-04-23T19:10:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 --2- 172.21.15.63:0/888750517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81aeda50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 -- 172.21.15.63:0/888750517 >> 172.21.15.63:0/888750517 conn(0x561c819f5db0 msgr2=0x561c81ae9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 -- 172.21.15.63:0/888750517 shutdown_connections 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 -- 172.21.15.63:0/888750517 wait complete. 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 Processor -- start 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 -- start start 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81af5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e44346800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c81a05a10 con 0x561c819fd750 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e41e4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81af5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e41e4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81af5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48418/0 (socket says 172.21.15.63:48418) 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.671+0000 7f5e41e4c640 1 -- 172.21.15.63:0/779654180 learned_addr learned my addr 172.21.15.63:0/779654180 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.672+0000 7f5e41e4c640 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c81af5a70 con 0x561c819fd750 2024-04-23T19:10:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.672+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81af5530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400a470 tx=0x7f5e34003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.672+0000 7f5d2e604640 1 -- 172.21.15.63:0/779654180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34073070 con 0x561c819fd750 2024-04-23T19:10:26.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.672+0000 7f5d2e604640 1 -- 172.21.15.63:0/779654180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3405d650 con 0x561c819fd750 2024-04-23T19:10:26.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.672+0000 7f5d2e604640 1 -- 172.21.15.63:0/779654180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34057070 con 0x561c819fd750 2024-04-23T19:10:26.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.672+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561c81af5d00 con 0x561c819fd750 2024-04-23T19:10:26.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.672+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c81af89b0 con 0x561c819fd750 2024-04-23T19:10:26.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.673+0000 7f5d2e604640 1 -- 172.21.15.63:0/779654180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5e34076070 con 0x561c819fd750 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.673+0000 7f5d2e604640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d28036e60 0x7f5d28039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.673+0000 7f5d2e604640 1 -- 172.21.15.63:0/779654180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f5e3408e670 con 0x561c819fd750 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.673+0000 7f5e4164b640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d28036e60 0x7f5d28039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.673+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c819fe0b0 0x561c81afc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.673+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81afcd40 con 0x561c819fe0b0 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.673+0000 7f5e4264d640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c819fe0b0 0x561c81afc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.674+0000 7f5e4164b640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d28036e60 0x7f5d28039320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000f60 tx=0x7f5e380012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.674+0000 7f5e4264d640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c819fe0b0 0x561c81afc770 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.674+0000 7f5e4264d640 1 -- 172.21.15.63:0/779654180 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5e3c00a4a0 con 0x561c819fe0b0 2024-04-23T19:10:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.674+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81af9d40 con 0x561c819fe0b0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.674+0000 7f5e4264d640 1 -- 172.21.15.63:0/779654180 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5e3c00a4a0 con 0x561c819fe0b0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.674+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81afeb90 con 0x561c819fe0b0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.675+0000 7f5e4264d640 1 -- 172.21.15.63:0/779654180 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5e3c00a4a0 con 0x561c819fe0b0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.675+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81afded0 con 0x561c819fe0b0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.675+0000 7f5e4264d640 1 -- 172.21.15.63:0/779654180 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5e3c00a4a0 con 0x561c819fe0b0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.675+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561c81afff40 0x561c81b02330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.675+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b028b0 con 0x561c81afff40 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.675+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561c81afff40 0x561c81b02330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.676+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561c81afff40 0x561c81b02330 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.676+0000 7f5e41e4c640 1 -- 172.21.15.63:0/779654180 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5e34075030 con 0x561c81afff40 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.676+0000 7f5e44346800 1 Processor -- start 2024-04-23T19:10:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.676+0000 7f5e44346800 1 -- start start 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e44346800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c81b10cd0 0x561c81b110f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e44346800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c818ede30 con 0x561c81b10cd0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e4264d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c81b10cd0 0x561c81b110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e4264d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c81b10cd0 0x561c81b110f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48420/0 (socket says 172.21.15.63:48420) 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e4264d640 1 -- 172.21.15.63:0/2794155230 learned_addr learned my addr 172.21.15.63:0/2794155230 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e4264d640 1 -- 172.21.15.63:0/2794155230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c81b11630 con 0x561c81b10cd0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e4264d640 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c81b10cd0 0x561c81b110f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x561c81af6240 tx=0x7f5e3c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5d277fe640 1 -- 172.21.15.63:0/2794155230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e3c013440 con 0x561c81b10cd0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5d277fe640 1 -- 172.21.15.63:0/2794155230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c0135e0 con 0x561c81b10cd0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561c81b120a0 con 0x561c81b10cd0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5d277fe640 1 -- 172.21.15.63:0/2794155230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e3c0138c0 con 0x561c81b10cd0 2024-04-23T19:10:26.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.677+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c81b125a0 con 0x561c81b10cd0 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.678+0000 7f5d277fe640 1 -- 172.21.15.63:0/2794155230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5e3c011040 con 0x561c81b10cd0 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.678+0000 7f5d277fe640 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d1c036eb0 0x7f5d1c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.678+0000 7f5d277fe640 1 -- 172.21.15.63:0/2794155230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f5e3c01d070 con 0x561c81b10cd0 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.678+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d1c036eb0 0x7f5d1c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e44346800 1 Processor -- start 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e44346800 1 -- start start 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d1c036eb0 0x7f5d1c039370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e3408cd20 tx=0x7f5e3405ee80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e44346800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c819fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e44346800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c81aee2e0 con 0x561c819fcaa0 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e4264d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c819fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e4264d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c819fcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48426/0 (socket says 172.21.15.63:48426) 2024-04-23T19:10:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e4264d640 1 -- 172.21.15.63:0/1728006827 learned_addr learned my addr 172.21.15.63:0/1728006827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.679+0000 7f5e4264d640 1 -- 172.21.15.63:0/1728006827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c819fb2a0 con 0x561c819fcaa0 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e4264d640 1 --2- 172.21.15.63:0/1728006827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c819fcec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c04cca0 tx=0x7f5e3c0127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607ade6843552d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5d257fa640 1 -- 172.21.15.63:0/1728006827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e3c042020 con 0x561c819fcaa0 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5d257fa640 1 -- 172.21.15.63:0/1728006827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c012cc0 con 0x561c819fcaa0 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5d257fa640 1 -- 172.21.15.63:0/1728006827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e3c027040 con 0x561c819fcaa0 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 -- 172.21.15.63:0/1728006827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 msgr2=0x561c819fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 --2- 172.21.15.63:0/1728006827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c819fcec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c04cca0 tx=0x7f5e3c0127a0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 -- 172.21.15.63:0/1728006827 shutdown_connections 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 --2- 172.21.15.63:0/1728006827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c819fcec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 -- 172.21.15.63:0/1728006827 >> 172.21.15.63:0/1728006827 conn(0x561c81b1a310 msgr2=0x561c81b18370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 -- 172.21.15.63:0/1728006827 shutdown_connections 2024-04-23T19:10:26.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 -- 172.21.15.63:0/1728006827 wait complete. 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.680+0000 7f5e44346800 1 Processor -- start 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e44346800 1 -- start start 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e44346800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c81b27740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e44346800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c819fb770 con 0x561c819fcaa0 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e4264d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c81b27740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e4264d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c81b27740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48438/0 (socket says 172.21.15.63:48438) 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 learned_addr learned my addr 172.21.15.63:0/3778156895 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c81b27c80 con 0x561c819fcaa0 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.681+0000 7f5e4264d640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c81b27740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c001df0 tx=0x7f5e3c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.682+0000 7f5d137fe640 1 -- 172.21.15.63:0/3778156895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e3c027040 con 0x561c819fcaa0 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.682+0000 7f5d137fe640 1 -- 172.21.15.63:0/3778156895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c021c60 con 0x561c819fcaa0 2024-04-23T19:10:26.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.682+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561c81b27f10 con 0x561c819fcaa0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.682+0000 7f5d137fe640 1 -- 172.21.15.63:0/3778156895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e3c02a350 con 0x561c819fcaa0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.682+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c81b2abc0 con 0x561c819fcaa0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.682+0000 7f5d137fe640 1 -- 172.21.15.63:0/3778156895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5e3c045020 con 0x561c819fcaa0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.682+0000 7f5d137fe640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d0c036eb0 0x7f5d0c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.683+0000 7f5d137fe640 1 -- 172.21.15.63:0/3778156895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f5e3c033030 con 0x561c819fcaa0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.683+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d0c036eb0 0x7f5d0c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.683+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d0c036eb0 0x7f5d0c039370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e3405cfd0 tx=0x7f5e34084000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.683+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.683+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c81b331f0 0x561c81b356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.683+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b35e00 con 0x561c81b331f0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.683+0000 7f5e4164b640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c81b331f0 0x561c81b356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e4164b640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c81b331f0 0x561c81b356b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 20 realm 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b32640 con 0x561c81b331f0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b368f0 con 0x561c81b331f0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.684+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b371c0 con 0x561c81b331f0 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e44346800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b37560 con 0x561c81b331f0 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e44346800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e44346800 20 searching for the correct realm 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561c81b39440 0x561c81b3b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x561c81b39440 2024-04-23T19:10:26.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.685+0000 7f5e4264d640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561c81b39440 0x561c81b3b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.686+0000 7f5e4264d640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561c81b39440 0x561c81b3b890 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.686+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c029070 con 0x561c81b39440 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.686+0000 7f5e4264d640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5e3c0123a0 0x7f5e3c03ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.686+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x7f5e3c0123a0 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.686+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5e3c0123a0 0x7f5e3c03ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.686+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5e3c0123a0 0x7f5e3c03ea40 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.687+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.687+0000 7f5e41e4c640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5e34085040 0x7f5e340773b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.687+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x7f5e34085040 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.687+0000 7f5e4164b640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5e34085040 0x7f5e340773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.687+0000 7f5e4164b640 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5e34085040 0x7f5e340773b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x561c81b331f0 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x561c81b331f0 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x7f5e34085040 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.688+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x7f5e34085040 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x7f5e3c0123a0 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x561c81b39440 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c029070 con 0x561c81b39440 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x561c81b331f0 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.689+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x7f5e34085040 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.690+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.690+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x561c81b331f0 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.690+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.690+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3bf60 con 0x7f5e3c0123a0 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.690+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.690+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b39440 2024-04-23T19:10:26.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.691+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c029070 con 0x561c81b39440 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.691+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b39440 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.691+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.691+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e34085040 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.691+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.691+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b39440 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b39440 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b331f0 2024-04-23T19:10:26.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b331f0 2024-04-23T19:10:26.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.692+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e34085040 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.693+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.693+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b39440 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.693+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.693+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e3c0123a0 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.693+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.694+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b331f0 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.694+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.694+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e34085040 2024-04-23T19:10:26.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.694+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.694+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b39440 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.694+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.694+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e34085040 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e3c0123a0 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b39440 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e34085040 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.695+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x561c81b331f0 2024-04-23T19:10:26.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f5e340a8060 con 0x7f5e3c0123a0 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b37f10 con 0x561c81b331f0 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.696+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b387b0 con 0x561c81b331f0 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3c3e0 con 0x7f5e34085040 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 zone default found 2024-04-23T19:10:26.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 4 Realm: () 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 zonegroup default 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3c890 con 0x7f5e34085040 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.697+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3e270 con 0x561c81b39440 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e44346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3e6d0 con 0x561c81b39440 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e44346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e44346800 20 started sync module instance, tier type = 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e44346800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.698+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3d700 con 0x7f5e3c0123a0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.699+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.699+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94680435120880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3d360 con 0x7f5e3c0123a0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.699+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3f600 con 0x561c81b331f0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.700+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94680435120880] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.700+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.700+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94680435126608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3fa20 con 0x561c81b331f0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.701+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b403e0 con 0x561c81b39440 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.702+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.702+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94680435130160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b40390 con 0x561c81b39440 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.702+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b41390 con 0x7f5e3c0123a0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.702+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94680435126608] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.703+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.703+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94680435127664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b41780 con 0x7f5e3c0123a0 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.703+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b421c0 con 0x561c81b331f0 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.703+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94680435130160] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.704+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.704+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94680435137680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b425b0 con 0x561c81b331f0 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.704+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b42fb0 con 0x7f5e34085040 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.704+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94680435127664] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.705+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.705+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94680435134112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b42f70 con 0x7f5e34085040 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.705+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b439f0 con 0x7f5e34085040 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.705+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94680435137680] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.706+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94680435134112] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.706+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.706+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94680435141312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b42ee0 con 0x7f5e34085040 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.706+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b43ea0 con 0x7f5e34085040 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94680435141312] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94680435144000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b43e10 con 0x7f5e34085040 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 20 add_watcher() i=0 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 20 add_watcher() i=1 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 20 add_watcher() i=2 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 20 add_watcher() i=3 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 20 add_watcher() i=4 2024-04-23T19:10:26.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 20 add_watcher() i=5 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.707+0000 7f5e44346800 20 add_watcher() i=6 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.708+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94680435144000] v30'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.708+0000 7f5e44346800 20 add_watcher() i=7 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.708+0000 7f5e44346800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.708+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b472d0 con 0x561c81b331f0 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.708+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.708+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94680435145520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b44780 con 0x561c81b331f0 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.709+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94680435145520] v30'56 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5cf37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 5 note: GC not initialized 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5cf27fc640 20 reqs_thread_entry: start 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 20 Filter name: none 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5c63fff640 20 reqs_thread_entry: start 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.710+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b7a7c0 con 0x561c81b331f0 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e44346800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e44346800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e44346800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e44346800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b7d0f0 con 0x7f5e3c0123a0 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.711+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b80790 con 0x561c81b331f0 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v30'2 uv2 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5e44346800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5e44346800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5e44346800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94680435393024 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561c81b7c0e0 con 0x561c81b331f0 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94680435137680 notify 128849018881 ret 0) ==== 609+0+0 (crc 0 0 0) 0x561c818ae7f0 con 0x561c81b331f0 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5d257fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018881 cookie 94680435137680 notifier 4310 bl.length()=567 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5d257fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:26.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5d257fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(65 notify.4 [notify cookie 94680435393024 out=8b] v0'0 uv33 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.712+0000 7f5d257fa640 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f5d04003ea0 con 0x561c81b331f0 2024-04-23T19:10:26.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.713+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.713+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94680435393024 notify 128849018881 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f5e3803c460 con 0x561c81b331f0 2024-04-23T19:10:26.730 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.714+0000 7f5cf37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.714+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94680435145520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b46e10 con 0x561c81b331f0 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.715+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94680435145520] v30'57 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.715+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94680435120880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b43b00 con 0x7f5e3c0123a0 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.715+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94680435120880] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.716+0000 7f5e44346800 20 remove_watcher() i=0 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.716+0000 7f5e44346800 2 removed watcher, disabling cache 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.716+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94680435126608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3daf0 con 0x561c81b331f0 2024-04-23T19:10:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.717+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94680435126608] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.717+0000 7f5e44346800 20 remove_watcher() i=1 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.717+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94680435130160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3f150 con 0x561c81b39440 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.718+0000 7f5e4264d640 1 -- 172.21.15.63:0/3778156895 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94680435130160] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e3c067040 con 0x561c81b39440 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.718+0000 7f5e44346800 20 remove_watcher() i=2 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.718+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94680435127664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3f150 con 0x7f5e3c0123a0 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.719+0000 7f5e41e4c640 1 -- 172.21.15.63:0/3778156895 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94680435127664] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e34075030 con 0x7f5e3c0123a0 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.719+0000 7f5e44346800 20 remove_watcher() i=3 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.719+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94680435137680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3f600 con 0x561c81b331f0 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.720+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94680435137680] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x561c81b331f0 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.720+0000 7f5e44346800 20 remove_watcher() i=4 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.720+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94680435134112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b3f600 con 0x7f5e34085040 2024-04-23T19:10:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.721+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94680435134112] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.721+0000 7f5e44346800 20 remove_watcher() i=5 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.721+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94680435141312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b40ea0 con 0x7f5e34085040 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.722+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94680435141312] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.722+0000 7f5e44346800 20 remove_watcher() i=6 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.722+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94680435144000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561c81b42ac0 con 0x7f5e34085040 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94680435144000] v30'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5e38009050 con 0x7f5e34085040 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 20 remove_watcher() i=7 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5e34085040 msgr2=0x7f5e340773b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5e34085040 0x7f5e340773b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c81b331f0 msgr2=0x561c81b356b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c81b331f0 0x561c81b356b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5e3c0123a0 msgr2=0x7f5e3c03ea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5e3c0123a0 0x7f5e3c03ea40 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561c81b39440 msgr2=0x561c81b3b890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561c81b39440 0x561c81b3b890 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d0c036eb0 msgr2=0x7f5d0c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d0c036eb0 0x7f5d0c039370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e3405cfd0 tx=0x7f5e34084000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 msgr2=0x561c81b27740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e4164b640 1 -- 172.21.15.63:0/3778156895 reap_dead start 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c81b27740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c001df0 tx=0x7f5e3c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 shutdown_connections 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 --2- 172.21.15.63:0/3778156895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fcaa0 0x561c81b27740 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 >> 172.21.15.63:0/3778156895 conn(0x561c81b1a310 msgr2=0x561c81b17970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 shutdown_connections 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.723+0000 7f5e44346800 1 -- 172.21.15.63:0/3778156895 wait complete. 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d1c036eb0 msgr2=0x7f5d1c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d1c036eb0 0x7f5d1c039370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e3408cd20 tx=0x7f5e3405ee80 comp rx=0 tx=0).stop 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c81b10cd0 msgr2=0x561c81b110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c81b10cd0 0x561c81b110f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x561c81af6240 tx=0x7f5e3c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 shutdown_connections 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d1c036eb0 0x7f5d1c039370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/2794155230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c81b10cd0 0x561c81b110f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 >> 172.21.15.63:0/2794155230 conn(0x561c81b04d70 msgr2=0x561c81b05170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 shutdown_connections 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/2794155230 wait complete. 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561c81afff40 msgr2=0x561c81b02330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561c81afff40 0x561c81b02330 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c819fe0b0 msgr2=0x561c81afc770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c819fe0b0 0x561c81afc770 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d28036e60 msgr2=0x7f5d28039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d28036e60 0x7f5d28039320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000f60 tx=0x7f5e380012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 msgr2=0x561c81af5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81af5530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400a470 tx=0x7f5e34003020 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 shutdown_connections 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561c81afff40 0x561c81b02330 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561c819fe0b0 0x561c81afc770 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d28036e60 0x7f5d28039320 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 --2- 172.21.15.63:0/779654180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c819fd750 0x561c81af5530 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 >> 172.21.15.63:0/779654180 conn(0x561c819f5db0 msgr2=0x561c81ae7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 shutdown_connections 2024-04-23T19:10:26.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.724+0000 7f5e44346800 1 -- 172.21.15.63:0/779654180 wait complete. 2024-04-23T19:10:26.743 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:26.743 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:26.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:26.743 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:26.789 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:26.789 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:26.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.789+0000 7f9f3572a800 1 Processor -- start 2024-04-23T19:10:26.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.789+0000 7f9f3572a800 1 -- start start 2024-04-23T19:10:26.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.789+0000 7f9f3572a800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e3da9d56a0 0x55e3da9d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.789+0000 7f9f3572a800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e3da8837f0 con 0x55e3da9d1b90 2024-04-23T19:10:26.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.789+0000 7f9f3572a800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55e3da872e20 con 0x55e3da9d56a0 2024-04-23T19:10:26.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.790+0000 7f9f30bf1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e3da9d56a0 0x55e3da9d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.790+0000 7f9f313f2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55e3da9d1b90 0x55e3da9d3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52696/0 (socket says 172.21.15.63:52696) 2024-04-23T19:10:26.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.790+0000 7f9f313f2640 1 -- 172.21.15.63:0/3545484936 learned_addr learned my addr 172.21.15.63:0/3545484936 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:26.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.790+0000 7f9f30bf1640 1 -- 172.21.15.63:0/3545484936 >> v1:172.21.15.16:6789/0 conn(0x55e3da9d1b90 legacy=0x55e3da9d3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.790+0000 7f9f30bf1640 1 -- 172.21.15.63:0/3545484936 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3da891b50 con 0x55e3da9d56a0 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f30bf1640 1 --2- 172.21.15.63:0/3545484936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d56a0 0x55e3da9d7a90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800a470 tx=0x7f9f2801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca86c65582aa842 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f23fff640 1 -- 172.21.15.63:0/3545484936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f28057070 con 0x55e3da9d56a0 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f23fff640 1 -- 172.21.15.63:0/3545484936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f28052570 con 0x55e3da9d56a0 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f23fff640 1 -- 172.21.15.63:0/3545484936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f28052870 con 0x55e3da9d56a0 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f3572a800 1 -- 172.21.15.63:0/3545484936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d56a0 msgr2=0x55e3da9d7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f3572a800 1 --2- 172.21.15.63:0/3545484936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d56a0 0x55e3da9d7a90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800a470 tx=0x7f9f2801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f3572a800 1 -- 172.21.15.63:0/3545484936 shutdown_connections 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f3572a800 1 --2- 172.21.15.63:0/3545484936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d56a0 0x55e3da9d7a90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f3572a800 1 -- 172.21.15.63:0/3545484936 >> 172.21.15.63:0/3545484936 conn(0x55e3da9cb600 msgr2=0x55e3da9cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f3572a800 1 -- 172.21.15.63:0/3545484936 shutdown_connections 2024-04-23T19:10:26.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.792+0000 7f9f3572a800 1 -- 172.21.15.63:0/3545484936 wait complete. 2024-04-23T19:10:26.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.805+0000 7f9f3572a800 1 Processor -- start 2024-04-23T19:10:26.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.805+0000 7f9f3572a800 1 -- start start 2024-04-23T19:10:26.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.805+0000 7f9f3572a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3da9d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.805+0000 7f9f3572a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3da91c6f0 con 0x55e3da9d1b90 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9f313f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3da9d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9f313f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3da9d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48448/0 (socket says 172.21.15.63:48448) 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9f313f2640 1 -- 172.21.15.63:0/2393756021 learned_addr learned my addr 172.21.15.63:0/2393756021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9f313f2640 1 -- 172.21.15.63:0/2393756021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3da9cfa90 con 0x55e3da9d1b90 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9f313f2640 1 --2- 172.21.15.63:0/2393756021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3da9d14d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9f24001de0 tx=0x7f9f2400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6be2da624622e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9e1f606640 1 -- 172.21.15.63:0/2393756021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f24012020 con 0x55e3da9d1b90 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9e1f606640 1 -- 172.21.15.63:0/2393756021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2400e5a0 con 0x55e3da9d1b90 2024-04-23T19:10:26.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.806+0000 7f9e1f606640 1 -- 172.21.15.63:0/2393756021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2400e8c0 con 0x55e3da9d1b90 2024-04-23T19:10:26.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 -- 172.21.15.63:0/2393756021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 msgr2=0x55e3da9d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 --2- 172.21.15.63:0/2393756021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3da9d14d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9f24001de0 tx=0x7f9f2400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 -- 172.21.15.63:0/2393756021 shutdown_connections 2024-04-23T19:10:26.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 --2- 172.21.15.63:0/2393756021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3da9d14d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 -- 172.21.15.63:0/2393756021 >> 172.21.15.63:0/2393756021 conn(0x55e3da9cb600 msgr2=0x55e3daabdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 -- 172.21.15.63:0/2393756021 shutdown_connections 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 -- 172.21.15.63:0/2393756021 wait complete. 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 Processor -- start 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.807+0000 7f9f3572a800 1 -- start start 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f3572a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3daacc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f3572a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3da9d82d0 con 0x55e3da9d1b90 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f313f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3daacc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f313f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3daacc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48458/0 (socket says 172.21.15.63:48458) 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f313f2640 1 -- 172.21.15.63:0/1044701234 learned_addr learned my addr 172.21.15.63:0/1044701234 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f313f2640 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3daacc610 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3daacc0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c010 tx=0x7f9f24000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9e1de03640 1 -- 172.21.15.63:0/1044701234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f24012070 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9e1de03640 1 -- 172.21.15.63:0/1044701234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2401a7a0 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e3daacc8a0 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9e1de03640 1 -- 172.21.15.63:0/1044701234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2401aa80 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.808+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3daacf550 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.809+0000 7f9e1de03640 1 -- 172.21.15.63:0/1044701234 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9f24015020 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.809+0000 7f9e1de03640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e18036e60 0x7f9e18039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.809+0000 7f9e1de03640 1 -- 172.21.15.63:0/1044701234 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f9f24020070 con 0x55e3da9d1b90 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.809+0000 7f9f30bf1640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e18036e60 0x7f9e18039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.810+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3da9d56a0 0x55e3daad3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.810+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3daad3850 con 0x55e3da9d56a0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.810+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3da9d56a0 0x55e3daad3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.810+0000 7f9f30bf1640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e18036e60 0x7f9e18039320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9f28054000 tx=0x7f9f2804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.810+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3da9d56a0 0x55e3daad3310 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.810+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1044701234 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f2c00a4a0 con 0x55e3da9d56a0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.810+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3daad08e0 con 0x55e3da9d56a0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.811+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1044701234 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9f2c00a4a0 con 0x55e3da9d56a0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.811+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3daad56e0 con 0x55e3da9d56a0 2024-04-23T19:10:26.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.811+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1044701234 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9f2c00a4a0 con 0x55e3da9d56a0 2024-04-23T19:10:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.811+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3daad4a20 con 0x55e3da9d56a0 2024-04-23T19:10:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.811+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1044701234 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9f2c00a4a0 con 0x55e3da9d56a0 2024-04-23T19:10:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.811+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e3daad6a90 0x55e3daad8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.811+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3daad9440 con 0x55e3daad6a90 2024-04-23T19:10:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.812+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e3daad6a90 0x55e3daad8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.812+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e3daad6a90 0x55e3daad8e80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.812+0000 7f9f313f2640 1 -- 172.21.15.63:0/1044701234 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9f2402b070 con 0x55e3daad6a90 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f3572a800 1 Processor -- start 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f3572a800 1 -- start start 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f3572a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daae7840 0x55e3daae7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f3572a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3da8c2e30 con 0x55e3daae7840 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f31bf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daae7840 0x55e3daae7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f31bf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daae7840 0x55e3daae7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48462/0 (socket says 172.21.15.63:48462) 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f31bf3640 1 -- 172.21.15.63:0/2496794900 learned_addr learned my addr 172.21.15.63:0/2496794900 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f31bf3640 1 -- 172.21.15.63:0/2496794900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3daae81a0 con 0x55e3daae7840 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.813+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daae7840 0x55e3daae7c60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55e3daad3c50 tx=0x7f9f2c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.814+0000 7f9e16ffd640 1 -- 172.21.15.63:0/2496794900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c013440 con 0x55e3daae7840 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.814+0000 7f9e16ffd640 1 -- 172.21.15.63:0/2496794900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0135e0 con 0x55e3daae7840 2024-04-23T19:10:26.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.814+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e3daae8c10 con 0x55e3daae7840 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.814+0000 7f9e16ffd640 1 -- 172.21.15.63:0/2496794900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c0138c0 con 0x55e3daae7840 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.814+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3daae9110 con 0x55e3daae7840 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.814+0000 7f9e16ffd640 1 -- 172.21.15.63:0/2496794900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9f2c011040 con 0x55e3daae7840 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9e16ffd640 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e08036eb0 0x7f9e08039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9e16ffd640 1 -- 172.21.15.63:0/2496794900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f9f2c01d070 con 0x55e3daae7840 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f313f2640 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e08036eb0 0x7f9e08039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f3572a800 1 Processor -- start 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f3572a800 1 -- start start 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f313f2640 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e08036eb0 0x7f9e08039370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f2404b350 tx=0x7f9f24041040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f3572a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daaf4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f3572a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3da9cfe10 con 0x55e3daaf4b00 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f31bf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daaf4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f31bf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daaf4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48468/0 (socket says 172.21.15.63:48468) 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.815+0000 7f9f31bf3640 1 -- 172.21.15.63:0/412872714 learned_addr learned my addr 172.21.15.63:0/412872714 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f31bf3640 1 -- 172.21.15.63:0/412872714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3daaf54f0 con 0x55e3daaf4b00 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/412872714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daaf4f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c04cbf0 tx=0x7f9f2c012770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dac73b6334db19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9e14ff9640 1 -- 172.21.15.63:0/412872714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c027040 con 0x55e3daaf4b00 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9e14ff9640 1 -- 172.21.15.63:0/412872714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c012c90 con 0x55e3daaf4b00 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9e14ff9640 1 -- 172.21.15.63:0/412872714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c0123a0 con 0x55e3daaf4b00 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f3572a800 1 -- 172.21.15.63:0/412872714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 msgr2=0x55e3daaf4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f3572a800 1 --2- 172.21.15.63:0/412872714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daaf4f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c04cbf0 tx=0x7f9f2c012770 comp rx=0 tx=0).stop 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f3572a800 1 -- 172.21.15.63:0/412872714 shutdown_connections 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f3572a800 1 --2- 172.21.15.63:0/412872714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daaf4f20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f3572a800 1 -- 172.21.15.63:0/412872714 >> 172.21.15.63:0/412872714 conn(0x55e3daaf0e10 msgr2=0x55e3daaeee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f3572a800 1 -- 172.21.15.63:0/412872714 shutdown_connections 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.816+0000 7f9f3572a800 1 -- 172.21.15.63:0/412872714 wait complete. 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f3572a800 1 Processor -- start 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f3572a800 1 -- start start 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f3572a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daafc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f3572a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3daaf5c40 con 0x55e3daaf4b00 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f31bf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daafc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f31bf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daafc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48472/0 (socket says 172.21.15.63:48472) 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 learned_addr learned my addr 172.21.15.63:0/1679124596 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3daafc5d0 con 0x55e3daaf4b00 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.817+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daafc090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c015ac0 tx=0x7f9f2c04cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.818+0000 7f9e06ffd640 1 -- 172.21.15.63:0/1679124596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c027070 con 0x55e3daaf4b00 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.818+0000 7f9e06ffd640 1 -- 172.21.15.63:0/1679124596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c013d90 con 0x55e3daaf4b00 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.818+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e3daafc860 con 0x55e3daaf4b00 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.818+0000 7f9e06ffd640 1 -- 172.21.15.63:0/1679124596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c02abb0 con 0x55e3daaf4b00 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.818+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3daaff510 con 0x55e3daaf4b00 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.818+0000 7f9e06ffd640 1 -- 172.21.15.63:0/1679124596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9f2c02a350 con 0x55e3daaf4b00 2024-04-23T19:10:26.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9e06ffd640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9dfc036eb0 0x7f9dfc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9e06ffd640 1 -- 172.21.15.63:0/1679124596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f9f2c00e040 con 0x55e3daaf4b00 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9dfc036eb0 0x7f9dfc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3dab07cf0 0x55e3dab0a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0a900 con 0x55e3dab07cf0 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f30bf1640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3dab07cf0 0x55e3dab0a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9dfc036eb0 0x7f9dfc039370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f24044650 tx=0x7f9f24031000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f30bf1640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3dab07cf0 0x55e3dab0a1b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.819+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 realm 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0acd0 con 0x55e3dab07cf0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0b5a0 con 0x55e3dab07cf0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0b940 con 0x55e3dab07cf0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.820+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0bd60 con 0x55e3dab07cf0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f3572a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f3572a800 20 searching for the correct realm 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e3dab0dd60 0x55e3dab101b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e3dab0dd60 0x55e3dab101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e3dab0dd60 0x55e3dab101b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f31bf3640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9f2c02bc00 0x7f9f2c02c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.821+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f2c02bc00 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.822+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9f2c02bc00 0x7f9f2c02c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.822+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9f2c02bc00 0x7f9f2c02c460 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.822+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.822+0000 7f9f313f2640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9f24017050 0x7f9f2400d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.822+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.822+0000 7f9f30bf1640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9f24017050 0x7f9f2400d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.823+0000 7f9f30bf1640 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9f24017050 0x7f9f2400d8e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.823+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.823+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.823+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.823+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f2c02bc00 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.824+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.825+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f2c02bc00 2024-04-23T19:10:26.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.826+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.827+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f2c02bc00 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.828+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.829+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.829+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.829+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.829+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.829+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.829+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f2c02bc00 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab0dd60 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f24017050 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x55e3dab07cf0 2024-04-23T19:10:26.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.830+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab10880 con 0x7f9f2c02bc00 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0c7d0 con 0x55e3dab07cf0 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0d070 con 0x55e3dab07cf0 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0c410 con 0x7f9f24017050 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 zone default found 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 4 Realm: () 2024-04-23T19:10:26.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.831+0000 7f9f3572a800 20 zonegroup default 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab0d070 con 0x7f9f24017050 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab12730 con 0x55e3dab0dd60 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab12b00 con 0x55e3dab0dd60 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 started sync module instance, tier type = 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.832+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab11bd0 con 0x7f9f2c02bc00 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.833+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.833+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94437114978240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab13410 con 0x7f9f2c02bc00 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.833+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab13c60 con 0x55e3dab07cf0 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.834+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94437114978240] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.835+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.835+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94437114982240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab13d00 con 0x55e3dab07cf0 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.835+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab14820 con 0x55e3dab0dd60 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.836+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94437114982240] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.836+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.836+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94437114987248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab14c10 con 0x55e3dab0dd60 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.836+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab15630 con 0x7f9f2c02bc00 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.837+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94437114987248] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.837+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.837+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94437114990880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab159d0 con 0x7f9f2c02bc00 2024-04-23T19:10:26.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.837+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab16490 con 0x55e3dab07cf0 2024-04-23T19:10:26.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.838+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.838+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94437114990880] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.838+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94437114994528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab16450 con 0x55e3dab07cf0 2024-04-23T19:10:26.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.838+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab137f0 con 0x7f9f24017050 2024-04-23T19:10:26.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.839+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.839+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94437114995792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab17430 con 0x7f9f24017050 2024-04-23T19:10:26.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.839+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab17e70 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.839+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94437114994528] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.840+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94437114995792] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.840+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.840+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94437114996864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab17430 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.840+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab182e0 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94437114996864] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94437115002416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab18250 con 0x7f9f24017050 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 20 add_watcher() i=0 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 20 add_watcher() i=1 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 20 add_watcher() i=2 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 20 add_watcher() i=3 2024-04-23T19:10:26.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 20 add_watcher() i=4 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 20 add_watcher() i=5 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.841+0000 7f9f3572a800 20 add_watcher() i=6 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.842+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94437115002416] v30'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.842+0000 7f9f3572a800 20 add_watcher() i=7 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.842+0000 7f9f3572a800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.842+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab1b710 con 0x55e3dab07cf0 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.843+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.843+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94437114998656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab18b90 con 0x55e3dab07cf0 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94437114998656] v30'58 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9de77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f3572a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f3572a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f3572a800 5 note: GC not initialized 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9de67e4640 20 reqs_thread_entry: start 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f3572a800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f3572a800 20 Filter name: none 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9d57fff640 20 reqs_thread_entry: start 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f3572a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.844+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab4e5f0 con 0x55e3dab07cf0 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.845+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.845+0000 7f9f3572a800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:26.864 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.846+0000 7f9de77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.846+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94437114998656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab1b000 con 0x55e3dab07cf0 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.847+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94437114998656] v30'59 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.847+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94437114978240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab16f80 con 0x7f9f2c02bc00 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.847+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94437114978240] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.847+0000 7f9f3572a800 20 remove_watcher() i=0 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.847+0000 7f9f3572a800 2 removed watcher, disabling cache 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.848+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94437114982240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab16f80 con 0x55e3dab07cf0 2024-04-23T19:10:26.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.849+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94437114982240] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.849+0000 7f9f3572a800 20 remove_watcher() i=1 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.849+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94437114987248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab12f60 con 0x55e3dab0dd60 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.850+0000 7f9f31bf3640 1 -- 172.21.15.63:0/1679124596 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94437114987248] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2c040040 con 0x55e3dab0dd60 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.850+0000 7f9f3572a800 20 remove_watcher() i=2 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.850+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94437114990880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab16f80 con 0x7f9f2c02bc00 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.851+0000 7f9f313f2640 1 -- 172.21.15.63:0/1679124596 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94437114990880] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2402b070 con 0x7f9f2c02bc00 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.851+0000 7f9f3572a800 20 remove_watcher() i=3 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.851+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94437114994528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab142f0 con 0x55e3dab07cf0 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.852+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94437114994528] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x55e3dab07cf0 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.852+0000 7f9f3572a800 20 remove_watcher() i=4 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.852+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94437114995792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab15120 con 0x7f9f24017050 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.852+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94437114995792] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.852+0000 7f9f3572a800 20 remove_watcher() i=5 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.853+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94437114996864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab16f80 con 0x7f9f24017050 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.853+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94437114996864] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.854+0000 7f9f3572a800 20 remove_watcher() i=6 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.854+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94437115002416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e3dab16900 con 0x7f9f24017050 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.854+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94437115002416] v30'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f2805a040 con 0x7f9f24017050 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.854+0000 7f9f3572a800 20 remove_watcher() i=7 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9f24017050 msgr2=0x7f9f2400d8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9f24017050 0x7f9f2400d8e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3dab07cf0 msgr2=0x55e3dab0a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3dab07cf0 0x55e3dab0a1b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9f2c02bc00 msgr2=0x7f9f2c02c460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9f2c02bc00 0x7f9f2c02c460 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e3dab0dd60 msgr2=0x55e3dab101b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e3dab0dd60 0x55e3dab101b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9dfc036eb0 msgr2=0x7f9dfc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9dfc036eb0 0x7f9dfc039370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f24044650 tx=0x7f9f24031000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 msgr2=0x55e3daafc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1679124596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daaf4b00 0x55e3daafc090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c015ac0 tx=0x7f9f2c04cb80 comp rx=0 tx=0).stop 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 reap_dead start 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f30bf1640 1 -- 172.21.15.63:0/1679124596 reap_dead start 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 shutdown_connections 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 >> 172.21.15.63:0/1679124596 conn(0x55e3daaf0e10 msgr2=0x55e3daaee470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 shutdown_connections 2024-04-23T19:10:26.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/1679124596 wait complete. 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e08036eb0 msgr2=0x7f9e08039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e08036eb0 0x7f9e08039370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f2404b350 tx=0x7f9f24041040 comp rx=0 tx=0).stop 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daae7840 msgr2=0x55e3daae7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daae7840 0x55e3daae7c60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55e3daad3c50 tx=0x7f9f2c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 shutdown_connections 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e08036eb0 0x7f9e08039370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 --2- 172.21.15.63:0/2496794900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3daae7840 0x55e3daae7c60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 >> 172.21.15.63:0/2496794900 conn(0x55e3daadb8e0 msgr2=0x55e3daadbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 shutdown_connections 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.855+0000 7f9f3572a800 1 -- 172.21.15.63:0/2496794900 wait complete. 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e3daad6a90 msgr2=0x55e3daad8e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e3daad6a90 0x55e3daad8e80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3da9d56a0 msgr2=0x55e3daad3310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3da9d56a0 0x55e3daad3310 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e18036e60 msgr2=0x7f9e18039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e18036e60 0x7f9e18039320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9f28054000 tx=0x7f9f2804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 msgr2=0x55e3daacc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3daacc0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400c010 tx=0x7f9f24000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 shutdown_connections 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e3daad6a90 0x55e3daad8e80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e3da9d56a0 0x55e3daad3310 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9e18036e60 0x7f9e18039320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 --2- 172.21.15.63:0/1044701234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3da9d1b90 0x55e3daacc0d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 >> 172.21.15.63:0/1044701234 conn(0x55e3da9cb600 msgr2=0x55e3da9d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 shutdown_connections 2024-04-23T19:10:26.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.856+0000 7f9f3572a800 1 -- 172.21.15.63:0/1044701234 wait complete. 2024-04-23T19:10:26.876 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:26.876 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-23T19:10:26.876 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-23T19:10:26.876 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-23T19:10:26.920 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:26.920 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f607ad1f800 1 Processor -- start 2024-04-23T19:10:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f607ad1f800 1 -- start start 2024-04-23T19:10:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f607ad1f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55de24549570 0x55de24552070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f607ad1f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55de243fb7f0 con 0x55de24547a30 2024-04-23T19:10:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f607ad1f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55de243eae20 con 0x55de24549570 2024-04-23T19:10:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f60761c2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55de24549570 0x55de24552070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f60769c3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55de24547a30 0x55de24547e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52708/0 (socket says 172.21.15.63:52708) 2024-04-23T19:10:26.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.921+0000 7f60769c3640 1 -- 172.21.15.63:0/538708563 learned_addr learned my addr 172.21.15.63:0/538708563 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:26.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.922+0000 7f60761c2640 1 -- 172.21.15.63:0/538708563 >> v1:172.21.15.16:6789/0 conn(0x55de24547a30 legacy=0x55de24547e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.922+0000 7f60759c1640 1 -- 172.21.15.63:0/538708563 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095545361 0 0) 0x55de243fb7f0 con 0x55de24547a30 2024-04-23T19:10:26.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.922+0000 7f60761c2640 1 -- 172.21.15.63:0/538708563 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de24409b50 con 0x55de24549570 2024-04-23T19:10:26.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.924+0000 7f60761c2640 1 --2- 172.21.15.63:0/538708563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24549570 0x55de24552070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f606c00a470 tx=0x7f606c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143f6a52cdbd8b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.924+0000 7f60759c1640 1 -- 172.21.15.63:0/538708563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f606c057070 con 0x55de24549570 2024-04-23T19:10:26.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.924+0000 7f60759c1640 1 -- 172.21.15.63:0/538708563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606c04adc0 con 0x55de24549570 2024-04-23T19:10:26.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.924+0000 7f60759c1640 1 -- 172.21.15.63:0/538708563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f606c052590 con 0x55de24549570 2024-04-23T19:10:26.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.925+0000 7f607ad1f800 1 -- 172.21.15.63:0/538708563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24549570 msgr2=0x55de24552070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.925+0000 7f607ad1f800 1 --2- 172.21.15.63:0/538708563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24549570 0x55de24552070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f606c00a470 tx=0x7f606c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.925+0000 7f607ad1f800 1 -- 172.21.15.63:0/538708563 shutdown_connections 2024-04-23T19:10:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.925+0000 7f607ad1f800 1 --2- 172.21.15.63:0/538708563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24549570 0x55de24552070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.925+0000 7f607ad1f800 1 -- 172.21.15.63:0/538708563 >> 172.21.15.63:0/538708563 conn(0x55de24543630 msgr2=0x55de24545a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.925+0000 7f607ad1f800 1 -- 172.21.15.63:0/538708563 shutdown_connections 2024-04-23T19:10:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.925+0000 7f607ad1f800 1 -- 172.21.15.63:0/538708563 wait complete. 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.937+0000 7f607ad1f800 1 Processor -- start 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.937+0000 7f607ad1f800 1 -- start start 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f607ad1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de2463c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f607ad1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de244946f0 con 0x55de24547a30 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f60769c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de2463c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f60769c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de2463c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48488/0 (socket says 172.21.15.63:48488) 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f60769c3640 1 -- 172.21.15.63:0/1748599408 learned_addr learned my addr 172.21.15.63:0/1748599408 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f60769c3640 1 -- 172.21.15.63:0/1748599408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2463ccb0 con 0x55de24547a30 2024-04-23T19:10:26.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f60769c3640 1 --2- 172.21.15.63:0/1748599408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de2463c6e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f606800b210 tx=0x7f606800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8085ea2fadc94ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f5f64e09640 1 -- 172.21.15.63:0/1748599408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6068008070 con 0x55de24547a30 2024-04-23T19:10:26.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f5f64e09640 1 -- 172.21.15.63:0/1748599408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606800d500 con 0x55de24547a30 2024-04-23T19:10:26.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.938+0000 7f5f64e09640 1 -- 172.21.15.63:0/1748599408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f606800d840 con 0x55de24547a30 2024-04-23T19:10:26.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 -- 172.21.15.63:0/1748599408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 msgr2=0x55de2463c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1748599408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de2463c6e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f606800b210 tx=0x7f606800b6d0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 -- 172.21.15.63:0/1748599408 shutdown_connections 2024-04-23T19:10:26.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1748599408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de2463c6e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 -- 172.21.15.63:0/1748599408 >> 172.21.15.63:0/1748599408 conn(0x55de24543630 msgr2=0x55de24638000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 -- 172.21.15.63:0/1748599408 shutdown_connections 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 -- 172.21.15.63:0/1748599408 wait complete. 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.939+0000 7f607ad1f800 1 Processor -- start 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.940+0000 7f607ad1f800 1 -- start start 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.940+0000 7f607ad1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de24644260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.940+0000 7f607ad1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de24554890 con 0x55de24547a30 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.940+0000 7f60769c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de24644260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.940+0000 7f60769c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de24644260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48492/0 (socket says 172.21.15.63:48492) 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.940+0000 7f60769c3640 1 -- 172.21.15.63:0/1584423163 learned_addr learned my addr 172.21.15.63:0/1584423163 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.940+0000 7f60769c3640 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de246447a0 con 0x55de24547a30 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.941+0000 7f60769c3640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de24644260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f606800b210 tx=0x7f6068000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.941+0000 7f5f63606640 1 -- 172.21.15.63:0/1584423163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f606800f950 con 0x55de24547a30 2024-04-23T19:10:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.941+0000 7f5f63606640 1 -- 172.21.15.63:0/1584423163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606800faf0 con 0x55de24547a30 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.941+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de24644a30 con 0x55de24547a30 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.941+0000 7f5f63606640 1 -- 172.21.15.63:0/1584423163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f606800fdd0 con 0x55de24547a30 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.941+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de246476e0 con 0x55de24547a30 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.941+0000 7f5f63606640 1 -- 172.21.15.63:0/1584423163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6068017040 con 0x55de24547a30 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f5f63606640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f5c036e60 0x7f5f5c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f5f63606640 1 -- 172.21.15.63:0/1584423163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f6068015020 con 0x55de24547a30 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f60761c2640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f5c036e60 0x7f5f5c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de24549570 0x55de2464b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de2464b9e0 con 0x55de24549570 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f60771c4640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de24549570 0x55de2464b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f60761c2640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f5c036e60 0x7f5f5c039320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f606c054000 tx=0x7f606c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.942+0000 7f60771c4640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de24549570 0x55de2464b4a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.943+0000 7f60771c4640 1 -- 172.21.15.63:0/1584423163 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f607000a4a0 con 0x55de24549570 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.943+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24648a70 con 0x55de24549570 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.943+0000 7f60771c4640 1 -- 172.21.15.63:0/1584423163 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f607000a4a0 con 0x55de24549570 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.943+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de2464d7e0 con 0x55de24549570 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.943+0000 7f60771c4640 1 -- 172.21.15.63:0/1584423163 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f607000a4a0 con 0x55de24549570 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.943+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de2464cb20 con 0x55de24549570 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.944+0000 7f60771c4640 1 -- 172.21.15.63:0/1584423163 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f607000a4a0 con 0x55de24549570 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.944+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de2464eb90 0x55de24650f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.944+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24651500 con 0x55de2464eb90 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.944+0000 7f60769c3640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de2464eb90 0x55de24650f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.944+0000 7f60769c3640 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de2464eb90 0x55de24650f80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.944+0000 7f60769c3640 1 -- 172.21.15.63:0/1584423163 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f606802d070 con 0x55de2464eb90 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f607ad1f800 1 Processor -- start 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f607ad1f800 1 -- start start 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f607ad1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2465f920 0x55de2465fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f607ad1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de2443ae30 con 0x55de2465f920 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f60771c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2465f920 0x55de2465fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f60771c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2465f920 0x55de2465fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48500/0 (socket says 172.21.15.63:48500) 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f60771c4640 1 -- 172.21.15.63:0/655138602 learned_addr learned my addr 172.21.15.63:0/655138602 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.945+0000 7f60771c4640 1 -- 172.21.15.63:0/655138602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de24660280 con 0x55de2465f920 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.946+0000 7f60771c4640 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2465f920 0x55de2465fd40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55de24645660 tx=0x7f60700178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.946+0000 7f5f60e01640 1 -- 172.21.15.63:0/655138602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6070013440 con 0x55de2465f920 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.946+0000 7f5f60e01640 1 -- 172.21.15.63:0/655138602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60700135e0 con 0x55de2465f920 2024-04-23T19:10:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.946+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de24660cf0 con 0x55de2465f920 2024-04-23T19:10:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.946+0000 7f5f60e01640 1 -- 172.21.15.63:0/655138602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60700138c0 con 0x55de2465f920 2024-04-23T19:10:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.946+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de246611f0 con 0x55de2465f920 2024-04-23T19:10:26.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.946+0000 7f5f60e01640 1 -- 172.21.15.63:0/655138602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6070011040 con 0x55de2465f920 2024-04-23T19:10:26.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f5f60e01640 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f50036eb0 0x7f5f50039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f5f60e01640 1 -- 172.21.15.63:0/655138602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f607001d070 con 0x55de2465f920 2024-04-23T19:10:26.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f60769c3640 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f50036eb0 0x7f5f50039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f607ad1f800 1 Processor -- start 2024-04-23T19:10:26.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f60769c3640 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f50036eb0 0x7f5f50039370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f606800b6f0 tx=0x7f606804ba00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f607ad1f800 1 -- start start 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f607ad1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de2466d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f607ad1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de2463d030 con 0x55de2466cc00 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f60771c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de2466d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f60771c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de2466d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48512/0 (socket says 172.21.15.63:48512) 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.947+0000 7f60771c4640 1 -- 172.21.15.63:0/1960613291 learned_addr learned my addr 172.21.15.63:0/1960613291 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f60771c4640 1 -- 172.21.15.63:0/1960613291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2466d5f0 con 0x55de2466cc00 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f60771c4640 1 --2- 172.21.15.63:0/1960613291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de2466d020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f607004cca0 tx=0x7f60700127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ff0d9521aed9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f5f5a7fc640 1 -- 172.21.15.63:0/1960613291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6070042030 con 0x55de2466cc00 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f5f5a7fc640 1 -- 172.21.15.63:0/1960613291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6070012cc0 con 0x55de2466cc00 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f5f5a7fc640 1 -- 172.21.15.63:0/1960613291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6070027040 con 0x55de2466cc00 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f607ad1f800 1 -- 172.21.15.63:0/1960613291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 msgr2=0x55de2466d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:26.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1960613291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de2466d020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f607004cca0 tx=0x7f60700127a0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f607ad1f800 1 -- 172.21.15.63:0/1960613291 shutdown_connections 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1960613291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de2466d020 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.948+0000 7f607ad1f800 1 -- 172.21.15.63:0/1960613291 >> 172.21.15.63:0/1960613291 conn(0x55de24668f60 msgr2=0x55de24666fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f607ad1f800 1 -- 172.21.15.63:0/1960613291 shutdown_connections 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f607ad1f800 1 -- 172.21.15.63:0/1960613291 wait complete. 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f607ad1f800 1 Processor -- start 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f607ad1f800 1 -- start start 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f607ad1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de24674210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f607ad1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de2466dd60 con 0x55de2466cc00 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f60771c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de24674210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f60771c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de24674210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48522/0 (socket says 172.21.15.63:48522) 2024-04-23T19:10:26.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.949+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 learned_addr learned my addr 172.21.15.63:0/1959122507 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.950+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de24674750 con 0x55de2466cc00 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.950+0000 7f60771c4640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de24674210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f607004bb90 tx=0x7f607004cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.950+0000 7f5f58ff9640 1 -- 172.21.15.63:0/1959122507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6070027030 con 0x55de2466cc00 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.950+0000 7f5f58ff9640 1 -- 172.21.15.63:0/1959122507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6070021c60 con 0x55de2466cc00 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.950+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de246749e0 con 0x55de2466cc00 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.950+0000 7f5f58ff9640 1 -- 172.21.15.63:0/1959122507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f607002a350 con 0x55de2466cc00 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.950+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de24677690 con 0x55de2466cc00 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.951+0000 7f5f58ff9640 1 -- 172.21.15.63:0/1959122507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6070045070 con 0x55de2466cc00 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.951+0000 7f5f58ff9640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f40036eb0 0x7f5f40039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.951+0000 7f5f58ff9640 1 -- 172.21.15.63:0/1959122507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f6070043070 con 0x55de2466cc00 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.951+0000 7f60769c3640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f40036eb0 0x7f5f40039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.951+0000 7f60769c3640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f40036eb0 0x7f5f40039370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55de246518e0 tx=0x7f6068042000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de2467feb0 0x55de24682370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24682ac0 con 0x55de2467feb0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f60761c2640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de2467feb0 0x55de24682370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f60761c2640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de2467feb0 0x55de24682370 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 20 realm 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24682ef0 con 0x55de2467feb0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.952+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de246837c0 con 0x55de2467feb0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24683b60 con 0x55de2467feb0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 20 searching for the correct realm 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de24686120 0x55de24688510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de24688b40 con 0x55de24686120 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.953+0000 7f60771c4640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de24686120 0x55de24688510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.954+0000 7f60771c4640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de24686120 0x55de24688510 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.954+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.954+0000 7f60771c4640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f60700123a0 0x7f6070046400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.954+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de24688b40 con 0x7f60700123a0 2024-04-23T19:10:26.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.954+0000 7f60769c3640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f60700123a0 0x7f6070046400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.955+0000 7f60769c3640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f60700123a0 0x7f6070046400 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.955+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.955+0000 7f60769c3640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6068019470 0x7f606804a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.955+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de24688b40 con 0x7f6068019470 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.955+0000 7f60761c2640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6068019470 0x7f606804a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.955+0000 7f60761c2640 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6068019470 0x7f606804a0e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.956+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f60700123a0 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.957+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f60700123a0 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.958+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.959+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.959+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.959+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.959+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.959+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.959+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.960+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.960+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.960+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.960+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.960+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.960+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f60700123a0 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.961+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.962+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.962+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.962+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.962+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.962+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.962+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f60700123a0 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.962+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.963+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de24686120 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.963+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.963+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f6068019470 2024-04-23T19:10:26.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.963+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.963+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x55de2467feb0 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55de246843b0 con 0x7f60700123a0 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24684c00 con 0x55de2467feb0 2024-04-23T19:10:26.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24684ff0 con 0x55de2467feb0 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.964+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24684ff0 con 0x7f6068019470 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 20 zone default found 2024-04-23T19:10:26.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 4 Realm: () 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 20 zonegroup default 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24688fc0 con 0x7f6068019470 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de2468a9a0 con 0x55de24686120 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.965+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f607ad1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de2468ad40 con 0x55de24686120 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f607ad1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f607ad1f800 20 started sync module instance, tier type = 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f607ad1f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.966+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24689e40 con 0x7f60700123a0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.967+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.967+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94412581937760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24689aa0 con 0x7f60700123a0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.967+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468ba40 con 0x55de2467feb0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.968+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94412581937760] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.968+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.968+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94412581941952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468be30 con 0x55de2467feb0 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.968+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468c870 con 0x55de24686120 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.969+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.969+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94412581946176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468d110 con 0x55de24686120 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.969+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468dbf0 con 0x7f60700123a0 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.969+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94412581941952] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.970+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.970+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94412581951232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468dfe0 con 0x7f60700123a0 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.970+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468ea40 con 0x55de2467feb0 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.970+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94412581946176] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:26.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.971+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.971+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94412581948512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468ee30 con 0x55de2467feb0 2024-04-23T19:10:26.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.971+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468f830 con 0x7f6068019470 2024-04-23T19:10:26.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.971+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94412581951232] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.972+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.972+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94412581943104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468f7f0 con 0x7f6068019470 2024-04-23T19:10:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.972+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690260 con 0x7f6068019470 2024-04-23T19:10:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.972+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94412581948512] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.973+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94412581943104] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.974+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.974+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94412581961104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468f340 con 0x7f6068019470 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.974+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690b00 con 0x7f6068019470 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.974+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94412581961104] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.974+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.974+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94412581954848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468f340 con 0x7f6068019470 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.975+0000 7f607ad1f800 20 add_watcher() i=0 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.975+0000 7f607ad1f800 20 add_watcher() i=1 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.975+0000 7f607ad1f800 20 add_watcher() i=2 2024-04-23T19:10:26.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.975+0000 7f607ad1f800 20 add_watcher() i=3 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.975+0000 7f607ad1f800 20 add_watcher() i=4 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.975+0000 7f607ad1f800 20 add_watcher() i=5 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.975+0000 7f607ad1f800 20 add_watcher() i=6 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.976+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94412581954848] v30'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.976+0000 7f607ad1f800 20 add_watcher() i=7 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.976+0000 7f607ad1f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.976+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de24693b30 con 0x55de2467feb0 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.976+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.976+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94412581963344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690f90 con 0x55de2467feb0 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.977+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94412581963344] v30'60 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.977+0000 7f5f2d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.977+0000 7f607ad1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.977+0000 7f607ad1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.977+0000 7f607ad1f800 5 note: GC not initialized 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.977+0000 7f5ea7fff640 20 reqs_thread_entry: start 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 20 Filter name: none 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f5ea5ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:26.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de246c6ef0 con 0x55de2467feb0 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.978+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de246c9820 con 0x7f60700123a0 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.979+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.979+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de246ccec0 con 0x55de2467feb0 2024-04-23T19:10:26.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v30'3 uv3 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f607ad1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f607ad1f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f607ad1f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94412582209840 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55de246c8810 con 0x55de2467feb0 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94412581948512 notify 128849018882 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55de24409b50 con 0x55de2467feb0 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f5f5a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018882 cookie 94412581948512 notifier 4324 bl.length()=567 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f5f5a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f5f5a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(65 notify.4 [notify cookie 94412582209840 out=8b] v0'0 uv39 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f5f5a7fc640 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f5f3c003ea0 con 0x55de2467feb0 2024-04-23T19:10:26.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:27.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.980+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94412582209840 notify 128849018882 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f606c06a910 con 0x55de2467feb0 2024-04-23T19:10:27.000 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.981+0000 7f5f2d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.981+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94412581963344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de246cbd10 con 0x55de2467feb0 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.982+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94412581963344] v30'61 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.982+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94412581937760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690650 con 0x7f60700123a0 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.983+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94412581937760] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.983+0000 7f607ad1f800 20 remove_watcher() i=0 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.983+0000 7f607ad1f800 2 removed watcher, disabling cache 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.983+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94412581941952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690650 con 0x55de2467feb0 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.984+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94412581941952] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.984+0000 7f607ad1f800 20 remove_watcher() i=1 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.984+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94412581946176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690650 con 0x55de24686120 2024-04-23T19:10:27.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.986+0000 7f60771c4640 1 -- 172.21.15.63:0/1959122507 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94412581946176] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6070029070 con 0x55de24686120 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.986+0000 7f607ad1f800 20 remove_watcher() i=2 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.986+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94412581951232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690650 con 0x7f60700123a0 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.987+0000 7f60769c3640 1 -- 172.21.15.63:0/1959122507 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94412581951232] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606802d070 con 0x7f60700123a0 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.987+0000 7f607ad1f800 20 remove_watcher() i=3 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.987+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94412581948512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690650 con 0x55de2467feb0 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.988+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94412581948512] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x55de2467feb0 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.988+0000 7f607ad1f800 20 remove_watcher() i=4 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.988+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94412581943104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468cc60 con 0x7f6068019470 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.989+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94412581943104] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.989+0000 7f607ad1f800 20 remove_watcher() i=5 2024-04-23T19:10:27.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.989+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94412581961104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de24690650 con 0x7f6068019470 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.990+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94412581961104] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.990+0000 7f607ad1f800 20 remove_watcher() i=6 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.990+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94412581954848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55de2468fd90 con 0x7f6068019470 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.990+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94412581954848] v30'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f606c05a040 con 0x7f6068019470 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.990+0000 7f607ad1f800 20 remove_watcher() i=7 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6068019470 msgr2=0x7f606804a0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6068019470 0x7f606804a0e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de2467feb0 msgr2=0x55de24682370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de2467feb0 0x55de24682370 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f60700123a0 msgr2=0x7f6070046400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f60700123a0 0x7f6070046400 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de24686120 msgr2=0x55de24688510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de24686120 0x55de24688510 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f40036eb0 msgr2=0x7f5f40039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f40036eb0 0x7f5f40039370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55de246518e0 tx=0x7f6068042000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 msgr2=0x55de24674210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1959122507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2466cc00 0x55de24674210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f607004bb90 tx=0x7f607004cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 reap_dead start 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f60761c2640 1 -- 172.21.15.63:0/1959122507 reap_dead start 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 shutdown_connections 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 >> 172.21.15.63:0/1959122507 conn(0x55de24668f60 msgr2=0x55de246665c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 shutdown_connections 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/1959122507 wait complete. 2024-04-23T19:10:27.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f50036eb0 msgr2=0x7f5f50039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f50036eb0 0x7f5f50039370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f606800b6f0 tx=0x7f606804ba00 comp rx=0 tx=0).stop 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2465f920 msgr2=0x55de2465fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2465f920 0x55de2465fd40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55de24645660 tx=0x7f60700178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 shutdown_connections 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f50036eb0 0x7f5f50039370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 --2- 172.21.15.63:0/655138602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de2465f920 0x55de2465fd40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 >> 172.21.15.63:0/655138602 conn(0x55de246539c0 msgr2=0x55de24653dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 shutdown_connections 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.991+0000 7f607ad1f800 1 -- 172.21.15.63:0/655138602 wait complete. 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de2464eb90 msgr2=0x55de24650f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de2464eb90 0x55de24650f80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de24549570 msgr2=0x55de2464b4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de24549570 0x55de2464b4a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f5c036e60 msgr2=0x7f5f5c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f5c036e60 0x7f5f5c039320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f606c054000 tx=0x7f606c04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 msgr2=0x55de24644260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de24644260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f606800b210 tx=0x7f6068000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 shutdown_connections 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de2464eb90 0x55de24650f80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de24549570 0x55de2464b4a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5f5c036e60 0x7f5f5c039320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 --2- 172.21.15.63:0/1584423163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de24547a30 0x55de24644260 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 >> 172.21.15.63:0/1584423163 conn(0x55de24543630 msgr2=0x55de24636860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 shutdown_connections 2024-04-23T19:10:27.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:26.992+0000 7f607ad1f800 1 -- 172.21.15.63:0/1584423163 wait complete. 2024-04-23T19:10:27.013 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.013 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.013 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.013 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:27.057 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:27.058 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:27.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0dad33800 1 Processor -- start 2024-04-23T19:10:27.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0dad33800 1 -- start start 2024-04-23T19:10:27.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0dad33800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x556cdbfd1620 0x556cdbfd3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0dad33800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556cdbe7f7f0 con 0x556cdbfcb400 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0dad33800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x556cdbe6ee20 con 0x556cdbfd1620 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0d61d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x556cdbfd1620 0x556cdbfd3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0d69d1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x556cdbfcb400 0x556cdbfcb820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52720/0 (socket says 172.21.15.63:52720) 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.058+0000 7fd0d69d1640 1 -- 172.21.15.63:0/516815305 learned_addr learned my addr 172.21.15.63:0/516815305 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.059+0000 7fd0d59cf640 1 -- 172.21.15.63:0/516815305 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024116063 0 0) 0x556cdbe7f7f0 con 0x556cdbfcb400 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.059+0000 7fd0d59cf640 1 -- 172.21.15.63:0/516815305 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0c0001e10 con 0x556cdbfcb400 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.059+0000 7fd0d61d0640 1 -- 172.21.15.63:0/516815305 >> v1:172.21.15.16:6789/0 conn(0x556cdbfcb400 legacy=0x556cdbfcb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.059+0000 7fd0d61d0640 1 -- 172.21.15.63:0/516815305 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdbe8db50 con 0x556cdbfd1620 2024-04-23T19:10:27.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0d61d0640 1 --2- 172.21.15.63:0/516815305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfd1620 0x556cdbfd3a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00a470 tx=0x7fd0cc01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6177b848efea0aad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0d59cf640 1 -- 172.21.15.63:0/516815305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0cc057070 con 0x556cdbfd1620 2024-04-23T19:10:27.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0d59cf640 1 -- 172.21.15.63:0/516815305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0cc052570 con 0x556cdbfd1620 2024-04-23T19:10:27.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0d59cf640 1 -- 172.21.15.63:0/516815305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0cc052870 con 0x556cdbfd1620 2024-04-23T19:10:27.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0dad33800 1 -- 172.21.15.63:0/516815305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfd1620 msgr2=0x556cdbfd3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0dad33800 1 --2- 172.21.15.63:0/516815305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfd1620 0x556cdbfd3a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00a470 tx=0x7fd0cc01cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0dad33800 1 -- 172.21.15.63:0/516815305 shutdown_connections 2024-04-23T19:10:27.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0dad33800 1 --2- 172.21.15.63:0/516815305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfd1620 0x556cdbfd3a10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0dad33800 1 -- 172.21.15.63:0/516815305 >> 172.21.15.63:0/516815305 conn(0x556cdbfc6db0 msgr2=0x556cdbfc91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0dad33800 1 -- 172.21.15.63:0/516815305 shutdown_connections 2024-04-23T19:10:27.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.061+0000 7fd0dad33800 1 -- 172.21.15.63:0/516815305 wait complete. 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.074+0000 7fd0dad33800 1 Processor -- start 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.074+0000 7fd0dad33800 1 -- start start 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.074+0000 7fd0dad33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdbfcdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.074+0000 7fd0dad33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cdbf186f0 con 0x556cdbfcb400 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fd0d69d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdbfcdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fd0d69d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdbfcdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48538/0 (socket says 172.21.15.63:48538) 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fd0d69d1640 1 -- 172.21.15.63:0/2683069233 learned_addr learned my addr 172.21.15.63:0/2683069233 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fd0d69d1640 1 -- 172.21.15.63:0/2683069233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdbfcc2f0 con 0x556cdbfcb400 2024-04-23T19:10:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/2683069233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdbfcdd30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8001de0 tx=0x7fd0c800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd3097550d890d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fcfc4e09640 1 -- 172.21.15.63:0/2683069233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c8012020 con 0x556cdbfcb400 2024-04-23T19:10:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fcfc4e09640 1 -- 172.21.15.63:0/2683069233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c800e5a0 con 0x556cdbfcb400 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.075+0000 7fcfc4e09640 1 -- 172.21.15.63:0/2683069233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c800e900 con 0x556cdbfcb400 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 -- 172.21.15.63:0/2683069233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 msgr2=0x556cdbfcdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 --2- 172.21.15.63:0/2683069233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdbfcdd30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8001de0 tx=0x7fd0c800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 -- 172.21.15.63:0/2683069233 shutdown_connections 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 --2- 172.21.15.63:0/2683069233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdbfcdd30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 -- 172.21.15.63:0/2683069233 >> 172.21.15.63:0/2683069233 conn(0x556cdbfc6db0 msgr2=0x556cdc0b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 -- 172.21.15.63:0/2683069233 shutdown_connections 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 -- 172.21.15.63:0/2683069233 wait complete. 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.076+0000 7fd0dad33800 1 Processor -- start 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0dad33800 1 -- start start 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0dad33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdc0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0dad33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cdbfd42b0 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0d69d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdc0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0d69d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdc0c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48548/0 (socket says 172.21.15.63:48548) 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0d69d1640 1 -- 172.21.15.63:0/1271861052 learned_addr learned my addr 172.21.15.63:0/1271861052 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0d69d1640 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdc0c84c0 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdc0c7f80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c010 tx=0x7fd0c8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.077+0000 7fcfc3606640 1 -- 172.21.15.63:0/1271861052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c8012030 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.078+0000 7fcfc3606640 1 -- 172.21.15.63:0/1271861052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c801a7a0 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.078+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556cdc0c8750 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.078+0000 7fcfc3606640 1 -- 172.21.15.63:0/1271861052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0c801aa80 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.078+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdc0cb400 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.078+0000 7fcfc3606640 1 -- 172.21.15.63:0/1271861052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd0c8015020 con 0x556cdbfcb400 2024-04-23T19:10:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fcfc3606640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfbc036eb0 0x7fcfbc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fcfc3606640 1 -- 172.21.15.63:0/1271861052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd0c8020030 con 0x556cdbfcb400 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fd0d61d0640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfbc036eb0 0x7fcfbc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdbfd1620 0x556cdc0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc0cf700 con 0x556cdbfd1620 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdbfd1620 0x556cdc0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fd0d61d0640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfbc036eb0 0x7fcfbc039370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc054000 tx=0x7fd0cc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.079+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdbfd1620 0x556cdc0cf1c0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.080+0000 7fd0d71d2640 1 -- 172.21.15.63:0/1271861052 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd0d000a4a0 con 0x556cdbfd1620 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.080+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc0cc790 con 0x556cdbfd1620 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.080+0000 7fd0d71d2640 1 -- 172.21.15.63:0/1271861052 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd0d000a4a0 con 0x556cdbfd1620 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.080+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc0d1550 con 0x556cdbfd1620 2024-04-23T19:10:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.080+0000 7fd0d71d2640 1 -- 172.21.15.63:0/1271861052 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd0d000a4a0 con 0x556cdbfd1620 2024-04-23T19:10:27.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.080+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc0d0890 con 0x556cdbfd1620 2024-04-23T19:10:27.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.081+0000 7fd0d71d2640 1 -- 172.21.15.63:0/1271861052 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd0d000a4a0 con 0x556cdbfd1620 2024-04-23T19:10:27.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.081+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556cdc0d2900 0x556cdc0d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.081+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc0d5270 con 0x556cdc0d2900 2024-04-23T19:10:27.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.081+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556cdc0d2900 0x556cdc0d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.081+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556cdc0d2900 0x556cdc0d4cf0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0d69d1640 1 -- 172.21.15.63:0/1271861052 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd0c800dcd0 con 0x556cdc0d2900 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0dad33800 1 Processor -- start 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0dad33800 1 -- start start 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0dad33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0e3620 0x556cdc0e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0dad33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cdbebee30 con 0x556cdc0e3620 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0d71d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0e3620 0x556cdc0e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0d71d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0e3620 0x556cdc0e3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48552/0 (socket says 172.21.15.63:48552) 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0d71d2640 1 -- 172.21.15.63:0/3638989756 learned_addr learned my addr 172.21.15.63:0/3638989756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.082+0000 7fd0d71d2640 1 -- 172.21.15.63:0/3638989756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdc0e3f80 con 0x556cdc0e3620 2024-04-23T19:10:27.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.083+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0e3620 0x556cdc0e3a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x556cdc0c9380 tx=0x7fd0d00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.083+0000 7fcfc0e01640 1 -- 172.21.15.63:0/3638989756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0d0013440 con 0x556cdc0e3620 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.083+0000 7fcfc0e01640 1 -- 172.21.15.63:0/3638989756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d00135e0 con 0x556cdc0e3620 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.083+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556cdc0e49f0 con 0x556cdc0e3620 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.083+0000 7fcfc0e01640 1 -- 172.21.15.63:0/3638989756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0d00138c0 con 0x556cdc0e3620 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.083+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdc0e4ef0 con 0x556cdc0e3620 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fcfc0e01640 1 -- 172.21.15.63:0/3638989756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd0d0011040 con 0x556cdc0e3620 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fcfc0e01640 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfb0036e60 0x7fcfb0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fcfc0e01640 1 -- 172.21.15.63:0/3638989756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd0d001d070 con 0x556cdc0e3620 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfb0036e60 0x7fcfb0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fd0dad33800 1 Processor -- start 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fd0dad33800 1 -- start start 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfb0036e60 0x7fcfb0039320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0c801a6f0 tx=0x7fd0c804b2e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fd0dad33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.084+0000 7fd0dad33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cdbfcc670 con 0x556cdc0f0940 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fd0d71d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fd0d71d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48560/0 (socket says 172.21.15.63:48560) 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fd0d71d2640 1 -- 172.21.15.63:0/1150180877 learned_addr learned my addr 172.21.15.63:0/1150180877 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fd0d71d2640 1 -- 172.21.15.63:0/1150180877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdc0f1330 con 0x556cdc0f0940 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/1150180877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f0d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd0d004ccb0 tx=0x7fd0d0012ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88f48ce2baf81b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fcfba7fc640 1 -- 172.21.15.63:0/1150180877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0d0012df0 con 0x556cdc0f0940 2024-04-23T19:10:27.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fcfba7fc640 1 -- 172.21.15.63:0/1150180877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d0001310 con 0x556cdc0f0940 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.085+0000 7fcfba7fc640 1 -- 172.21.15.63:0/1150180877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0d0001630 con 0x556cdc0f0940 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 -- 172.21.15.63:0/1150180877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 msgr2=0x556cdc0f0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1150180877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f0d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd0d004ccb0 tx=0x7fd0d0012ac0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 -- 172.21.15.63:0/1150180877 shutdown_connections 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1150180877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f0d60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 -- 172.21.15.63:0/1150180877 >> 172.21.15.63:0/1150180877 conn(0x556cdc0ecc50 msgr2=0x556cdc0ead00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 -- 172.21.15.63:0/1150180877 shutdown_connections 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 -- 172.21.15.63:0/1150180877 wait complete. 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 Processor -- start 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 -- start start 2024-04-23T19:10:27.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0dad33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cdc0f1b00 con 0x556cdc0f0940 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0d71d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0d71d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48570/0 (socket says 172.21.15.63:48570) 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.086+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 learned_addr learned my addr 172.21.15.63:0/77145414 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.087+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdc0f83d0 con 0x556cdc0f0940 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.087+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f7e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0d004bb30 tx=0x7fd0d004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.087+0000 7fcfb8ff9640 1 -- 172.21.15.63:0/77145414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0d0021c60 con 0x556cdc0f0940 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.087+0000 7fcfb8ff9640 1 -- 172.21.15.63:0/77145414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d0021e00 con 0x556cdc0f0940 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.087+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556cdc0f8660 con 0x556cdc0f0940 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.087+0000 7fcfb8ff9640 1 -- 172.21.15.63:0/77145414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0d002a350 con 0x556cdc0f0940 2024-04-23T19:10:27.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.087+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdc0fb310 con 0x556cdc0f0940 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.088+0000 7fcfb8ff9640 1 -- 172.21.15.63:0/77145414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd0d0040040 con 0x556cdc0f0940 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.088+0000 7fcfb8ff9640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfa0036eb0 0x7fcfa0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.088+0000 7fcfb8ff9640 1 -- 172.21.15.63:0/77145414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd0d0032030 con 0x556cdc0f0940 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.088+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfa0036eb0 0x7fcfa0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.088+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.088+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdc103b10 0x556cdc105fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.088+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc106720 con 0x556cdc103b10 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0d61d0640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdc103b10 0x556cdc105fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfa0036eb0 0x7fcfa0039370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x556cdc0d5650 tx=0x7fd0c8043000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0d61d0640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdc103b10 0x556cdc105fd0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 20 realm 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc106af0 con 0x556cdc103b10 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.089+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1073c0 con 0x556cdc103b10 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc107760 con 0x556cdc103b10 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc107b80 con 0x556cdc103b10 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 20 searching for the correct realm 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556cdc109b80 0x556cdc10bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.090+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556cdc109b80 0x556cdc10bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.091+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556cdc109b80 0x556cdc10bfd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.091+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.091+0000 7fd0d71d2640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd0d0046070 0x7fd0d0037430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.091+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0d0046070 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.091+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd0d0046070 0x7fd0d0037430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.091+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd0d0046070 0x7fd0d0037430 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.092+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.092+0000 7fd0d69d1640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd0c8017050 0x7fd0c8044140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.092+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.092+0000 7fd0d61d0640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd0c8017050 0x7fd0c8044140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.092+0000 7fd0d61d0640 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd0c8017050 0x7fd0c8044140 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.093+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.094+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.094+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0d0046070 2024-04-23T19:10:27.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.094+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.094+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.094+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.094+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.095+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.095+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.095+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.095+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.095+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.095+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0d0046070 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.096+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.096+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.096+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.096+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.096+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.096+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.097+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.098+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.098+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.098+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.098+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.098+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.098+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0d0046070 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.099+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0d0046070 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc109b80 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0c8017050 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.100+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x556cdc103b10 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10c6a0 con 0x7fd0d0046070 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc108620 con 0x556cdc103b10 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.101+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc108ec0 con 0x556cdc103b10 2024-04-23T19:10:27.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc108260 con 0x7fd0c8017050 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 zone default found 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 4 Realm: () 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 zonegroup default 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10cb70 con 0x7fd0c8017050 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.102+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10ea00 con 0x556cdc109b80 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0dad33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10eda0 con 0x556cdc109b80 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0dad33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0dad33800 20 started sync module instance, tier type = 2024-04-23T19:10:27.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0dad33800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.103+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10de40 con 0x7fd0d0046070 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.104+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.104+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93926036922928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10e650 con 0x7fd0d0046070 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.104+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10fc10 con 0x556cdc103b10 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.105+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93926036922928] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.105+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.105+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93926036928352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc110030 con 0x556cdc103b10 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.105+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1109f0 con 0x556cdc109b80 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.106+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93926036928352] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.107+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.107+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93926036929888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1109f0 con 0x556cdc109b80 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.107+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1113f0 con 0x7fd0d0046070 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.108+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93926036929888] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.108+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.108+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93926036931904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc110960 con 0x7fd0d0046070 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.108+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1122b0 con 0x556cdc103b10 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.109+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.109+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93926036931904] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.109+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93926036938720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc112b50 con 0x556cdc103b10 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.109+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc113090 con 0x7fd0c8017050 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.110+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.110+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93926036936672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc113540 con 0x7fd0c8017050 2024-04-23T19:10:27.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.110+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc113f80 con 0x7fd0c8017050 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.110+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93926036938720] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.111+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93926036936672] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.111+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.111+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93926036945488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc113090 con 0x7fd0c8017050 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.111+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc114820 con 0x7fd0c8017050 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93926036947824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc113090 con 0x7fd0c8017050 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 20 add_watcher() i=0 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 20 add_watcher() i=1 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 20 add_watcher() i=2 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 20 add_watcher() i=3 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 20 add_watcher() i=4 2024-04-23T19:10:27.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 20 add_watcher() i=5 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93926036945488] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.112+0000 7fd0dad33800 20 add_watcher() i=6 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.113+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93926036947824] v30'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.113+0000 7fd0dad33800 20 add_watcher() i=7 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.113+0000 7fd0dad33800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.113+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1126a0 con 0x556cdc103b10 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.113+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.113+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93926036940448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc114db0 con 0x556cdc103b10 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.114+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93926036940448] v30'62 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.114+0000 7fcf8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.114+0000 7fd0dad33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.114+0000 7fd0dad33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.114+0000 7fd0dad33800 5 note: GC not initialized 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fcf07fff640 20 reqs_thread_entry: start 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fd0dad33800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fd0dad33800 20 Filter name: none 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fcf05ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fd0dad33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x556cdc14a790 con 0x556cdc103b10 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.115+0000 7fd0dad33800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:27.131 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.116+0000 7fcf8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.116+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93926036940448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1011c0 con 0x556cdc103b10 2024-04-23T19:10:27.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.117+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93926036940448] v30'63 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.117+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93926036922928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc1126a0 con 0x7fd0d0046070 2024-04-23T19:10:27.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.118+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93926036922928] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.118+0000 7fd0dad33800 20 remove_watcher() i=0 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.118+0000 7fd0dad33800 2 removed watcher, disabling cache 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.118+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93926036928352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10e230 con 0x556cdc103b10 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.120+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93926036928352] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.120+0000 7fd0dad33800 20 remove_watcher() i=1 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.120+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93926036929888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10e230 con 0x556cdc109b80 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.121+0000 7fd0d71d2640 1 -- 172.21.15.63:0/77145414 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93926036929888] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0d0029070 con 0x556cdc109b80 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.121+0000 7fd0dad33800 20 remove_watcher() i=2 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.121+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93926036931904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10fd60 con 0x7fd0d0046070 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.122+0000 7fd0d69d1640 1 -- 172.21.15.63:0/77145414 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93926036931904] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0c800dcd0 con 0x7fd0d0046070 2024-04-23T19:10:27.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.122+0000 7fd0dad33800 20 remove_watcher() i=3 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.122+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93926036938720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10e230 con 0x556cdc103b10 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.123+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93926036938720] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x556cdc103b10 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.123+0000 7fd0dad33800 20 remove_watcher() i=4 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.123+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93926036936672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc110540 con 0x7fd0c8017050 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.124+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93926036936672] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.124+0000 7fd0dad33800 20 remove_watcher() i=5 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.124+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93926036945488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc111fe0 con 0x7fd0c8017050 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.125+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93926036945488] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.125+0000 7fd0dad33800 20 remove_watcher() i=6 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.125+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93926036947824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x556cdc10e230 con 0x7fd0c8017050 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.125+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93926036947824] v30'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd0cc05a040 con 0x7fd0c8017050 2024-04-23T19:10:27.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 20 remove_watcher() i=7 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd0c8017050 msgr2=0x7fd0c8044140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd0c8017050 0x7fd0c8044140 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdc103b10 msgr2=0x556cdc105fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdc103b10 0x556cdc105fd0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd0d0046070 msgr2=0x7fd0d0037430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd0d0046070 0x7fd0d0037430 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556cdc109b80 msgr2=0x556cdc10bfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556cdc109b80 0x556cdc10bfd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfa0036eb0 msgr2=0x7fcfa0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfa0036eb0 0x7fcfa0039370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x556cdc0d5650 tx=0x7fd0c8043000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 msgr2=0x556cdc0f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0d61d0640 1 -- 172.21.15.63:0/77145414 reap_dead start 2024-04-23T19:10:27.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f7e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0d004bb30 tx=0x7fd0d004cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 shutdown_connections 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/77145414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0f0940 0x556cdc0f7e90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 >> 172.21.15.63:0/77145414 conn(0x556cdc0ecc50 msgr2=0x556cdc0ea300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 shutdown_connections 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/77145414 wait complete. 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfb0036e60 msgr2=0x7fcfb0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfb0036e60 0x7fcfb0039320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0c801a6f0 tx=0x7fd0c804b2e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0e3620 msgr2=0x556cdc0e3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0e3620 0x556cdc0e3a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x556cdc0c9380 tx=0x7fd0d00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 shutdown_connections 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfb0036e60 0x7fcfb0039320 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 --2- 172.21.15.63:0/3638989756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdc0e3620 0x556cdc0e3a40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 >> 172.21.15.63:0/3638989756 conn(0x556cdc0d76c0 msgr2=0x556cdc0d7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 shutdown_connections 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.126+0000 7fd0dad33800 1 -- 172.21.15.63:0/3638989756 wait complete. 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556cdc0d2900 msgr2=0x556cdc0d4cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556cdc0d2900 0x556cdc0d4cf0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdbfd1620 msgr2=0x556cdc0cf1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdbfd1620 0x556cdc0cf1c0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfbc036eb0 msgr2=0x7fcfbc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfbc036eb0 0x7fcfbc039370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc054000 tx=0x7fd0cc04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 msgr2=0x556cdc0c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdc0c7f80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c010 tx=0x7fd0c8000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 shutdown_connections 2024-04-23T19:10:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556cdc0d2900 0x556cdc0d4cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556cdbfd1620 0x556cdc0cf1c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcfbc036eb0 0x7fcfbc039370 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 --2- 172.21.15.63:0/1271861052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cdbfcb400 0x556cdc0c7f80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 >> 172.21.15.63:0/1271861052 conn(0x556cdbfc6db0 msgr2=0x556cdbfccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 shutdown_connections 2024-04-23T19:10:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.127+0000 7fd0dad33800 1 -- 172.21.15.63:0/1271861052 wait complete. 2024-04-23T19:10:27.143 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.143 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-23T19:10:27.144 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-23T19:10:27.144 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-04-23T19:10:27.192 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:27.192 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.192+0000 7ff2847a2800 1 Processor -- start 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.192+0000 7ff2847a2800 1 -- start start 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff2847a2800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x564a1a6c85c0 0x564a1a6d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff2847a2800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564a1a57a7f0 con 0x564a1a6c6a80 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff2847a2800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x564a1a569e20 con 0x564a1a6c85c0 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff281aa0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x564a1a6c85c0 0x564a1a6d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff2822a1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x564a1a6c6a80 0x564a1a6c6ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52730/0 (socket says 172.21.15.63:52730) 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff2822a1640 1 -- 172.21.15.63:0/3644346050 learned_addr learned my addr 172.21.15.63:0/3644346050 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff28129f640 1 -- 172.21.15.63:0/3644346050 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234724391 0 0) 0x564a1a57a7f0 con 0x564a1a6c6a80 2024-04-23T19:10:27.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff28129f640 1 -- 172.21.15.63:0/3644346050 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff26c001e10 con 0x564a1a6c6a80 2024-04-23T19:10:27.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff281aa0640 1 -- 172.21.15.63:0/3644346050 >> v1:172.21.15.16:6789/0 conn(0x564a1a6c6a80 legacy=0x564a1a6c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.193+0000 7ff281aa0640 1 -- 172.21.15.63:0/3644346050 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1a588b50 con 0x564a1a6c85c0 2024-04-23T19:10:27.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.195+0000 7ff281aa0640 1 --2- 172.21.15.63:0/3644346050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c85c0 0x564a1a6d10c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff27800a470 tx=0x7ff27801cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7682d318b3d89209 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.195+0000 7ff28129f640 1 -- 172.21.15.63:0/3644346050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff278057070 con 0x564a1a6c85c0 2024-04-23T19:10:27.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff28129f640 1 -- 172.21.15.63:0/3644346050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff278052570 con 0x564a1a6c85c0 2024-04-23T19:10:27.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff28129f640 1 -- 172.21.15.63:0/3644346050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff278052850 con 0x564a1a6c85c0 2024-04-23T19:10:27.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff2847a2800 1 -- 172.21.15.63:0/3644346050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c85c0 msgr2=0x564a1a6d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3644346050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c85c0 0x564a1a6d10c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff27800a470 tx=0x7ff27801cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff2847a2800 1 -- 172.21.15.63:0/3644346050 shutdown_connections 2024-04-23T19:10:27.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3644346050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c85c0 0x564a1a6d10c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff2847a2800 1 -- 172.21.15.63:0/3644346050 >> 172.21.15.63:0/3644346050 conn(0x564a1a6c2620 msgr2=0x564a1a6c4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff2847a2800 1 -- 172.21.15.63:0/3644346050 shutdown_connections 2024-04-23T19:10:27.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.196+0000 7ff2847a2800 1 -- 172.21.15.63:0/3644346050 wait complete. 2024-04-23T19:10:27.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.209+0000 7ff2847a2800 1 Processor -- start 2024-04-23T19:10:27.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.209+0000 7ff2847a2800 1 -- start start 2024-04-23T19:10:27.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.209+0000 7ff2847a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.209+0000 7ff2847a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a1a613710 con 0x564a1a6c6a80 2024-04-23T19:10:27.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff2822a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff2822a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7bb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48588/0 (socket says 172.21.15.63:48588) 2024-04-23T19:10:27.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff2822a1640 1 -- 172.21.15.63:0/548905621 learned_addr learned my addr 172.21.15.63:0/548905621 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff2822a1640 1 -- 172.21.15.63:0/548905621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1a7bbd00 con 0x564a1a6c6a80 2024-04-23T19:10:27.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff2822a1640 1 --2- 172.21.15.63:0/548905621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7bb730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff274006f60 tx=0x7ff274006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f72c360d13b3df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff170608640 1 -- 172.21.15.63:0/548905621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff274004520 con 0x564a1a6c6a80 2024-04-23T19:10:27.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff170608640 1 -- 172.21.15.63:0/548905621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2740046e0 con 0x564a1a6c6a80 2024-04-23T19:10:27.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.210+0000 7ff170608640 1 -- 172.21.15.63:0/548905621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff274004a80 con 0x564a1a6c6a80 2024-04-23T19:10:27.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 -- 172.21.15.63:0/548905621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 msgr2=0x564a1a7bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 --2- 172.21.15.63:0/548905621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7bb730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff274006f60 tx=0x7ff274006f90 comp rx=0 tx=0).stop 2024-04-23T19:10:27.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 -- 172.21.15.63:0/548905621 shutdown_connections 2024-04-23T19:10:27.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 --2- 172.21.15.63:0/548905621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7bb730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 -- 172.21.15.63:0/548905621 >> 172.21.15.63:0/548905621 conn(0x564a1a6c2620 msgr2=0x564a1a7b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 -- 172.21.15.63:0/548905621 shutdown_connections 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 -- 172.21.15.63:0/548905621 wait complete. 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 Processor -- start 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 -- start start 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.211+0000 7ff2847a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a1a6d38e0 con 0x564a1a6c6a80 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff2822a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff2822a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7c3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48602/0 (socket says 172.21.15.63:48602) 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff2822a1640 1 -- 172.21.15.63:0/116643967 learned_addr learned my addr 172.21.15.63:0/116643967 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff2822a1640 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1a7c38a0 con 0x564a1a6c6a80 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff2822a1640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7c3360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff2740044f0 tx=0x7ff274000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff16ee05640 1 -- 172.21.15.63:0/116643967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff274022030 con 0x564a1a6c6a80 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff16ee05640 1 -- 172.21.15.63:0/116643967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff274019040 con 0x564a1a6c6a80 2024-04-23T19:10:27.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564a1a7c3b30 con 0x564a1a6c6a80 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff16ee05640 1 -- 172.21.15.63:0/116643967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff274014b80 con 0x564a1a6c6a80 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.212+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1a7c67e0 con 0x564a1a6c6a80 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.213+0000 7ff16ee05640 1 -- 172.21.15.63:0/116643967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff274025070 con 0x564a1a6c6a80 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.213+0000 7ff16ee05640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff168036eb0 0x7ff168039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.213+0000 7ff16ee05640 1 -- 172.21.15.63:0/116643967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff274023070 con 0x564a1a6c6a80 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.213+0000 7ff281aa0640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff168036eb0 0x7ff168039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.213+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a6c85c0 0x564a1a7ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.213+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a7cab20 con 0x564a1a6c85c0 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.214+0000 7ff282aa2640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a6c85c0 0x564a1a7ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.214+0000 7ff281aa0640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff168036eb0 0x7ff168039370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff278054000 tx=0x7ff27804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.214+0000 7ff282aa2640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a6c85c0 0x564a1a7ca5a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.214+0000 7ff282aa2640 1 -- 172.21.15.63:0/116643967 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff27c0083e0 con 0x564a1a6c85c0 2024-04-23T19:10:27.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.214+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a7c7b50 con 0x564a1a6c85c0 2024-04-23T19:10:27.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.214+0000 7ff282aa2640 1 -- 172.21.15.63:0/116643967 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff27c0083e0 con 0x564a1a6c85c0 2024-04-23T19:10:27.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.214+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a7cc920 con 0x564a1a6c85c0 2024-04-23T19:10:27.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.215+0000 7ff282aa2640 1 -- 172.21.15.63:0/116643967 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff27c0083e0 con 0x564a1a6c85c0 2024-04-23T19:10:27.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.215+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a7cbc60 con 0x564a1a6c85c0 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.215+0000 7ff282aa2640 1 -- 172.21.15.63:0/116643967 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff27c0083e0 con 0x564a1a6c85c0 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.215+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564a1a7cdcd0 0x564a1a7d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.215+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a7d0640 con 0x564a1a7cdcd0 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.215+0000 7ff2822a1640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564a1a7cdcd0 0x564a1a7d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.216+0000 7ff2822a1640 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564a1a7cdcd0 0x564a1a7d00c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.216+0000 7ff2822a1640 1 -- 172.21.15.63:0/116643967 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff27400f030 con 0x564a1a7cdcd0 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.216+0000 7ff2847a2800 1 Processor -- start 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.216+0000 7ff2847a2800 1 -- start start 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.216+0000 7ff2847a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7dea10 0x564a1a7dee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.216+0000 7ff2847a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a1a5b9e30 con 0x564a1a7dea10 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff282aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7dea10 0x564a1a7dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff282aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7dea10 0x564a1a7dee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48608/0 (socket says 172.21.15.63:48608) 2024-04-23T19:10:27.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff282aa2640 1 -- 172.21.15.63:0/2456204623 learned_addr learned my addr 172.21.15.63:0/2456204623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff282aa2640 1 -- 172.21.15.63:0/2456204623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1a7df370 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff282aa2640 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7dea10 0x564a1a7dee30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x564a1a7c4760 tx=0x7ff27c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff167fff640 1 -- 172.21.15.63:0/2456204623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c01c070 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff167fff640 1 -- 172.21.15.63:0/2456204623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27c00bdc0 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564a1a7dfde0 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff167fff640 1 -- 172.21.15.63:0/2456204623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c018540 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.217+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1a7e02e0 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.218+0000 7ff167fff640 1 -- 172.21.15.63:0/2456204623 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff27c01f070 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.218+0000 7ff167fff640 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff15c036e60 0x7ff15c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.218+0000 7ff167fff640 1 -- 172.21.15.63:0/2456204623 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff27c04e9d0 con 0x564a1a7dea10 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.218+0000 7ff2822a1640 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff15c036e60 0x7ff15c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff2847a2800 1 Processor -- start 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff2822a1640 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff15c036e60 0x7ff15c039320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x564a1a7d0a40 tx=0x7ff274014d20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff2847a2800 1 -- start start 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff2847a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7ec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff2847a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a1a7bc080 con 0x564a1a7ebe10 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff282aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff282aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7ec230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48616/0 (socket says 172.21.15.63:48616) 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff282aa2640 1 -- 172.21.15.63:0/964594090 learned_addr learned my addr 172.21.15.63:0/964594090 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff282aa2640 1 -- 172.21.15.63:0/964594090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1a7ec800 con 0x564a1a7ebe10 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff282aa2640 1 --2- 172.21.15.63:0/964594090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7ec230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff27c04cfa0 tx=0x7ff27c017460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac1c239c266e466 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.219+0000 7ff165ffb640 1 -- 172.21.15.63:0/964594090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c026070 con 0x564a1a7ebe10 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff165ffb640 1 -- 172.21.15.63:0/964594090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27c04a030 con 0x564a1a7ebe10 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff165ffb640 1 -- 172.21.15.63:0/964594090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c029c70 con 0x564a1a7ebe10 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 -- 172.21.15.63:0/964594090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 msgr2=0x564a1a7ec230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 --2- 172.21.15.63:0/964594090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7ec230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff27c04cfa0 tx=0x7ff27c017460 comp rx=0 tx=0).stop 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 -- 172.21.15.63:0/964594090 shutdown_connections 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 --2- 172.21.15.63:0/964594090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7ec230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 -- 172.21.15.63:0/964594090 >> 172.21.15.63:0/964594090 conn(0x564a1a7e8090 msgr2=0x564a1a7e60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 -- 172.21.15.63:0/964594090 shutdown_connections 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 -- 172.21.15.63:0/964594090 wait complete. 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 Processor -- start 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.220+0000 7ff2847a2800 1 -- start start 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff2847a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff2847a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a1a7ecf30 con 0x564a1a7ebe10 2024-04-23T19:10:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff282aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff282aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48622/0 (socket says 172.21.15.63:48622) 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 learned_addr learned my addr 172.21.15.63:0/3448475759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1a7f3870 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff282aa2640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7f3330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff27c04cfa0 tx=0x7ff27c04ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff153fff640 1 -- 172.21.15.63:0/3448475759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c039070 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff153fff640 1 -- 172.21.15.63:0/3448475759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27c016a40 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564a1a7f3b00 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff153fff640 1 -- 172.21.15.63:0/3448475759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27c016d20 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.221+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1a7f67b0 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.222+0000 7ff153fff640 1 -- 172.21.15.63:0/3448475759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff27c043020 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.222+0000 7ff153fff640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff14c036e60 0x7ff14c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.222+0000 7ff153fff640 1 -- 172.21.15.63:0/3448475759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff27c035c30 con 0x564a1a7ebe10 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.222+0000 7ff2822a1640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff14c036e60 0x7ff14c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.223+0000 7ff2822a1640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff14c036e60 0x7ff14c039320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff27404b6c0 tx=0x7ff274030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.223+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.223+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a7fee60 0x564a1a801320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.223+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a801a70 con 0x564a1a7fee60 2024-04-23T19:10:27.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.223+0000 7ff281aa0640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a7fee60 0x564a1a801320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.223+0000 7ff281aa0640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a7fee60 0x564a1a801320 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 realm 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a801e40 con 0x564a1a7fee60 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a802710 con 0x564a1a7fee60 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a802ab0 con 0x564a1a7fee60 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.224+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a803300 con 0x564a1a7fee60 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff2847a2800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff2847a2800 20 searching for the correct realm 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564a1a805070 0x564a1a807460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff282aa2640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564a1a805070 0x564a1a807460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff282aa2640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564a1a805070 0x564a1a807460 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff282aa2640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff27c021070 0x7ff27c0467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.225+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff27c021070 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.226+0000 7ff2822a1640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff27c021070 0x7ff27c0467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.226+0000 7ff2822a1640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff27c021070 0x7ff27c0467d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.226+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.226+0000 7ff2822a1640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff274056040 0x7ff27403f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.226+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.226+0000 7ff281aa0640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff274056040 0x7ff27403f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.227+0000 7ff281aa0640 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff274056040 0x7ff27403f0e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.227+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.227+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.227+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.227+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.227+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.227+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.228+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.228+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.228+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.228+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff27c021070 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.228+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.228+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.229+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff27c021070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.230+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.231+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff27c021070 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.232+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.233+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.233+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.233+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.233+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff27c021070 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a805070 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff274056040 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.234+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x564a1a7fee60 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8079f0 con 0x7ff27c021070 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a803c50 con 0x564a1a7fee60 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8044f0 con 0x564a1a7fee60 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.235+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8044f0 con 0x7ff274056040 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 zone default found 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 4 Realm: () 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 zonegroup default 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a807e70 con 0x7ff274056040 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8041f0 con 0x564a1a805070 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.236+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a809950 con 0x564a1a805070 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.237+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.237+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.237+0000 7ff2847a2800 20 started sync module instance, tier type = 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.237+0000 7ff2847a2800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.237+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a808cf0 con 0x7ff27c021070 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.238+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.238+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94876272201952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a809500 con 0x7ff27c021070 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.238+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80adb0 con 0x564a1a7fee60 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.239+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94876272201952] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.239+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.239+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94876272208128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80b1d0 con 0x564a1a7fee60 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.239+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80bb90 con 0x564a1a805070 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.240+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.240+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94876272211680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80bb40 con 0x564a1a805070 2024-04-23T19:10:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.240+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80cb40 con 0x7ff27c021070 2024-04-23T19:10:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.240+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94876272208128] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.241+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94876272211680] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.241+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.241+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94876272213888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80c650 con 0x7ff27c021070 2024-04-23T19:10:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.241+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80d550 con 0x564a1a7fee60 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.242+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94876272213888] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.242+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.242+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94876272218272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80d970 con 0x564a1a7fee60 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.242+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80e240 con 0x7ff274056040 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.243+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.243+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94876272220528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80dd90 con 0x7ff274056040 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.243+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80f1f0 con 0x7ff274056040 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.243+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94876272218272] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.244+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94876272220528] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.244+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.244+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94876272219328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80f1f0 con 0x7ff274056040 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.244+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80e6d0 con 0x7ff274056040 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94876272219328] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94876272223952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80f5e0 con 0x7ff274056040 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 20 add_watcher() i=0 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 20 add_watcher() i=1 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 20 add_watcher() i=2 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 20 add_watcher() i=3 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 20 add_watcher() i=4 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 20 add_watcher() i=5 2024-04-23T19:10:27.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.245+0000 7ff2847a2800 20 add_watcher() i=6 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.246+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94876272223952] v30'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.246+0000 7ff2847a2800 20 add_watcher() i=7 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.246+0000 7ff2847a2800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.246+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a812a70 con 0x564a1a7fee60 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.247+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.247+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94876272225488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80ff30 con 0x564a1a7fee60 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94876272225488] v30'64 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff138fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 5 note: GC not initialized 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff0b37fe640 20 reqs_thread_entry: start 2024-04-23T19:10:27.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 20 Filter name: none 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff0b17fa640 20 reqs_thread_entry: start 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.248+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a845f40 con 0x564a1a7fee60 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a846850 con 0x7ff274056040 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.249+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~502 in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a84c0a0 con 0x564a1a7fee60 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.250+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~502] v30'4 uv4 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.250+0000 7ff2847a2800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.250+0000 7ff2847a2800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.250+0000 7ff2847a2800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.250+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94876272474896 in=690b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a84abf0 con 0x564a1a7fee60 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94876272218272 notify 128849018883 ret 0) ==== 720+0+0 (crc 0 0 0) 0x564a1a588b50 con 0x564a1a7fee60 2024-04-23T19:10:27.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff165ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018883 cookie 94876272218272 notifier 4338 bl.length()=678 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff165ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff165ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(65 notify.4 [notify cookie 94876272474896 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff165ffb640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7ff148003ea0 con 0x564a1a7fee60 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94876272474896 notify 128849018883 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff27808d490 con 0x564a1a7fee60 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.251+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a84bb10 con 0x7ff274056040 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.252+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v30'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.252+0000 7ff2847a2800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.252+0000 7ff2847a2800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.252+0000 7ff2847a2800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:10:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.252+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94876272472096 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564a1a84bb10 con 0x564a1a805070 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== watch-notify(notify (1) cookie 94876272211680 notify 128849018880 ret 0) ==== 219+0+0 (crc 0 0 0) 0x564a1a7f3b00 con 0x564a1a805070 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff1657fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018880 cookie 94876272211680 notifier 4338 bl.length()=177 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 notify.2 [notify cookie 94876272472096 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff1657fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff1657fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff1657fa640 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7ff0b8003400 con 0x564a1a805070 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== watch-notify(notify_complete (2) cookie 94876272472096 notify 128849018880 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564a1a7f3870 con 0x564a1a805070 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.253+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.265 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-23T19:10:27.250094Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.254+0000 7ff138fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.254+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94876272225488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a84a740 con 0x564a1a7fee60 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.255+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94876272225488] v30'65 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.255+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94876272201952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a84abc0 con 0x7ff27c021070 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.256+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94876272201952] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.256+0000 7ff2847a2800 20 remove_watcher() i=0 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.256+0000 7ff2847a2800 2 removed watcher, disabling cache 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.256+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94876272208128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8090e0 con 0x564a1a7fee60 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.257+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94876272208128] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.257+0000 7ff2847a2800 20 remove_watcher() i=1 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.257+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94876272211680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80a900 con 0x564a1a805070 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.258+0000 7ff282aa2640 1 -- 172.21.15.63:0/3448475759 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94876272211680] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27c03b030 con 0x564a1a805070 2024-04-23T19:10:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.258+0000 7ff2847a2800 20 remove_watcher() i=2 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.258+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94876272213888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8090e0 con 0x7ff27c021070 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.259+0000 7ff2822a1640 1 -- 172.21.15.63:0/3448475759 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94876272213888] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff27400f030 con 0x7ff27c021070 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.259+0000 7ff2847a2800 20 remove_watcher() i=3 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.259+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94876272218272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80b6e0 con 0x564a1a7fee60 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.260+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94876272218272] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x564a1a7fee60 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.260+0000 7ff2847a2800 20 remove_watcher() i=4 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.260+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94876272220528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80a900 con 0x7ff274056040 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.261+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94876272220528] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.261+0000 7ff2847a2800 20 remove_watcher() i=5 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.261+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94876272219328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a8090e0 con 0x7ff274056040 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.262+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94876272219328] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.262+0000 7ff2847a2800 20 remove_watcher() i=6 2024-04-23T19:10:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.262+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94876272223952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564a1a80d970 con 0x7ff274056040 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94876272223952] v30'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff278052ca0 con 0x7ff274056040 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 20 remove_watcher() i=7 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff274056040 msgr2=0x7ff27403f0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff274056040 0x7ff27403f0e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a7fee60 msgr2=0x564a1a801320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a7fee60 0x564a1a801320 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff27c021070 msgr2=0x7ff27c0467d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff27c021070 0x7ff27c0467d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564a1a805070 msgr2=0x564a1a807460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564a1a805070 0x564a1a807460 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff14c036e60 msgr2=0x7ff14c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff14c036e60 0x7ff14c039320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff27404b6c0 tx=0x7ff274030000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 msgr2=0x564a1a7f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7f3330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff27c04cfa0 tx=0x7ff27c04ccb0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff281aa0640 1 -- 172.21.15.63:0/3448475759 reap_dead start 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 shutdown_connections 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/3448475759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7ebe10 0x564a1a7f3330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 >> 172.21.15.63:0/3448475759 conn(0x564a1a7e8090 msgr2=0x564a1a7e56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 shutdown_connections 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/3448475759 wait complete. 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff15c036e60 msgr2=0x7ff15c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff15c036e60 0x7ff15c039320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x564a1a7d0a40 tx=0x7ff274014d20 comp rx=0 tx=0).stop 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7dea10 msgr2=0x564a1a7dee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.263+0000 7ff2847a2800 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7dea10 0x564a1a7dee30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x564a1a7c4760 tx=0x7ff27c00ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 shutdown_connections 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff15c036e60 0x7ff15c039320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/2456204623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a7dea10 0x564a1a7dee30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 >> 172.21.15.63:0/2456204623 conn(0x564a1a7d2ab0 msgr2=0x564a1a7d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 shutdown_connections 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/2456204623 wait complete. 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564a1a7cdcd0 msgr2=0x564a1a7d00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564a1a7cdcd0 0x564a1a7d00c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a6c85c0 msgr2=0x564a1a7ca5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a6c85c0 0x564a1a7ca5a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff168036eb0 msgr2=0x7ff168039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff168036eb0 0x7ff168039370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff278054000 tx=0x7ff27804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 msgr2=0x564a1a7c3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7c3360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff2740044f0 tx=0x7ff274000f40 comp rx=0 tx=0).stop 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 shutdown_connections 2024-04-23T19:10:27.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564a1a7cdcd0 0x564a1a7d00c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564a1a6c85c0 0x564a1a7ca5a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff168036eb0 0x7ff168039370 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 --2- 172.21.15.63:0/116643967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1a6c6a80 0x564a1a7c3360 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 >> 172.21.15.63:0/116643967 conn(0x564a1a6c2620 msgr2=0x564a1a7b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 shutdown_connections 2024-04-23T19:10:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.264+0000 7ff2847a2800 1 -- 172.21.15.63:0/116643967 wait complete. 2024-04-23T19:10:27.277 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-23T19:10:27.250094Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.277 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.277 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:27.322 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:27.322 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:27.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f476951f800 1 Processor -- start 2024-04-23T19:10:27.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f476951f800 1 -- start start 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f476951f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561af09546b0 0x561af0956aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f476951f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561af08027f0 con 0x561af0950ba0 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f476951f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x561af07f1e20 con 0x561af09546b0 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f47649c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561af09546b0 0x561af0956aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f47651ca640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x561af0950ba0 0x561af0952f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52742/0 (socket says 172.21.15.63:52742) 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f47651ca640 1 -- 172.21.15.63:0/1929081955 learned_addr learned my addr 172.21.15.63:0/1929081955 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f47649c9640 1 -- 172.21.15.63:0/1929081955 >> v1:172.21.15.16:6789/0 conn(0x561af0950ba0 legacy=0x561af0952f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.323+0000 7f47649c9640 1 -- 172.21.15.63:0/1929081955 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af0810b50 con 0x561af09546b0 2024-04-23T19:10:27.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.325+0000 7f47649c9640 1 --2- 172.21.15.63:0/1929081955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af09546b0 0x561af0956aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f475c00a470 tx=0x7f475c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40fa3706d2a43d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.325+0000 7f4757fff640 1 -- 172.21.15.63:0/1929081955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f475c057070 con 0x561af09546b0 2024-04-23T19:10:27.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.325+0000 7f4757fff640 1 -- 172.21.15.63:0/1929081955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475c052570 con 0x561af09546b0 2024-04-23T19:10:27.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.325+0000 7f4757fff640 1 -- 172.21.15.63:0/1929081955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f475c052870 con 0x561af09546b0 2024-04-23T19:10:27.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.326+0000 7f476951f800 1 -- 172.21.15.63:0/1929081955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af09546b0 msgr2=0x561af0956aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.326+0000 7f476951f800 1 --2- 172.21.15.63:0/1929081955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af09546b0 0x561af0956aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f475c00a470 tx=0x7f475c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:27.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.326+0000 7f476951f800 1 -- 172.21.15.63:0/1929081955 shutdown_connections 2024-04-23T19:10:27.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.326+0000 7f476951f800 1 --2- 172.21.15.63:0/1929081955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af09546b0 0x561af0956aa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.326+0000 7f476951f800 1 -- 172.21.15.63:0/1929081955 >> 172.21.15.63:0/1929081955 conn(0x561af094a630 msgr2=0x561af094ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.326+0000 7f476951f800 1 -- 172.21.15.63:0/1929081955 shutdown_connections 2024-04-23T19:10:27.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.326+0000 7f476951f800 1 -- 172.21.15.63:0/1929081955 wait complete. 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f476951f800 1 Processor -- start 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f476951f800 1 -- start start 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f476951f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f476951f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561af089b6f0 con 0x561af0950ba0 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f47651ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f47651ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48646/0 (socket says 172.21.15.63:48646) 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f47651ca640 1 -- 172.21.15.63:0/662960052 learned_addr learned my addr 172.21.15.63:0/662960052 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.339+0000 7f47651ca640 1 -- 172.21.15.63:0/662960052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af0a41980 con 0x561af0950ba0 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.340+0000 7f47651ca640 1 --2- 172.21.15.63:0/662960052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a413b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4758001de0 tx=0x7f475800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75a3d880d951a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.340+0000 7f4653606640 1 -- 172.21.15.63:0/662960052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4758012020 con 0x561af0950ba0 2024-04-23T19:10:27.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.340+0000 7f4653606640 1 -- 172.21.15.63:0/662960052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475800e5a0 con 0x561af0950ba0 2024-04-23T19:10:27.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.340+0000 7f4653606640 1 -- 172.21.15.63:0/662960052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f475800e8c0 con 0x561af0950ba0 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.340+0000 7f476951f800 1 -- 172.21.15.63:0/662960052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 msgr2=0x561af0a413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.340+0000 7f476951f800 1 --2- 172.21.15.63:0/662960052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a413b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4758001de0 tx=0x7f475800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 -- 172.21.15.63:0/662960052 shutdown_connections 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 --2- 172.21.15.63:0/662960052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a413b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 -- 172.21.15.63:0/662960052 >> 172.21.15.63:0/662960052 conn(0x561af094a630 msgr2=0x561af0a3ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 -- 172.21.15.63:0/662960052 shutdown_connections 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 -- 172.21.15.63:0/662960052 wait complete. 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 Processor -- start 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 -- start start 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a48ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f476951f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561af09572e0 con 0x561af0950ba0 2024-04-23T19:10:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f47651ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a48ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f47651ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a48ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48656/0 (socket says 172.21.15.63:48656) 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.341+0000 7f47651ca640 1 -- 172.21.15.63:0/2491648074 learned_addr learned my addr 172.21.15.63:0/2491648074 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.342+0000 7f47651ca640 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af0a49430 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.342+0000 7f47651ca640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a48ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4758001de0 tx=0x7f4758000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.342+0000 7f4651e03640 1 -- 172.21.15.63:0/2491648074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4758012030 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.342+0000 7f4651e03640 1 -- 172.21.15.63:0/2491648074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475801a7a0 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.342+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561af0a496c0 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.342+0000 7f4651e03640 1 -- 172.21.15.63:0/2491648074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f475801aac0 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.342+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af0a4c370 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.343+0000 7f4651e03640 1 -- 172.21.15.63:0/2491648074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4758015020 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.343+0000 7f4651e03640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f464c036e60 0x7f464c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.343+0000 7f4651e03640 1 -- 172.21.15.63:0/2491648074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4758020070 con 0x561af0950ba0 2024-04-23T19:10:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.343+0000 7f47649c9640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f464c036e60 0x7f464c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.343+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af09546b0 0x561af0a50130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.343+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a50670 con 0x561af09546b0 2024-04-23T19:10:27.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.343+0000 7f47659cb640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af09546b0 0x561af0a50130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.344+0000 7f47649c9640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f464c036e60 0x7f464c039320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f475c054000 tx=0x7f475c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.344+0000 7f47659cb640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af09546b0 0x561af0a50130 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.344+0000 7f47659cb640 1 -- 172.21.15.63:0/2491648074 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f476000a4a0 con 0x561af09546b0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.344+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a4d700 con 0x561af09546b0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.344+0000 7f47659cb640 1 -- 172.21.15.63:0/2491648074 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f476000a4a0 con 0x561af09546b0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.344+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a524b0 con 0x561af09546b0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.345+0000 7f47659cb640 1 -- 172.21.15.63:0/2491648074 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f476000a4a0 con 0x561af09546b0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.345+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a517f0 con 0x561af09546b0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.345+0000 7f47659cb640 1 -- 172.21.15.63:0/2491648074 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f476000a4a0 con 0x561af09546b0 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.345+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561af0a53860 0x561af0a55c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.345+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a56210 con 0x561af0a53860 2024-04-23T19:10:27.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.345+0000 7f47651ca640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561af0a53860 0x561af0a55c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.345+0000 7f47651ca640 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561af0a53860 0x561af0a55c50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f47651ca640 1 -- 172.21.15.63:0/2491648074 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4758024980 con 0x561af0a53860 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f476951f800 1 Processor -- start 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f476951f800 1 -- start start 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f476951f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950740 0x561af094ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f476951f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561af0841e30 con 0x561af0950740 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f47659cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950740 0x561af094ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f47659cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950740 0x561af094ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48668/0 (socket says 172.21.15.63:48668) 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.346+0000 7f47659cb640 1 -- 172.21.15.63:0/515811294 learned_addr learned my addr 172.21.15.63:0/515811294 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.347+0000 7f47659cb640 1 -- 172.21.15.63:0/515811294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af094f0f0 con 0x561af0950740 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.347+0000 7f47659cb640 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950740 0x561af094ebb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x561af0a4a2f0 tx=0x7f47600178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.347+0000 7f464affd640 1 -- 172.21.15.63:0/515811294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760013440 con 0x561af0950740 2024-04-23T19:10:27.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.347+0000 7f464affd640 1 -- 172.21.15.63:0/515811294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47600135e0 con 0x561af0950740 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.347+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561af094f380 con 0x561af0950740 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.347+0000 7f464affd640 1 -- 172.21.15.63:0/515811294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47600138c0 con 0x561af0950740 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.347+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af094f880 con 0x561af0950740 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f464affd640 1 -- 172.21.15.63:0/515811294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4760011040 con 0x561af0950740 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f464affd640 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f463c036e60 0x7f463c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f464affd640 1 -- 172.21.15.63:0/515811294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f476001d070 con 0x561af0950740 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f47651ca640 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f463c036e60 0x7f463c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f476951f800 1 Processor -- start 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f476951f800 1 -- start start 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f47651ca640 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f463c036e60 0x7f463c039320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f475800efd0 tx=0x7f475804b4c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f476951f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a73f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.348+0000 7f476951f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561af0a41d00 con 0x561af0a73b60 2024-04-23T19:10:27.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f47659cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a73f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f47659cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a73f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48684/0 (socket says 172.21.15.63:48684) 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f47659cb640 1 -- 172.21.15.63:0/990151196 learned_addr learned my addr 172.21.15.63:0/990151196 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f47659cb640 1 -- 172.21.15.63:0/990151196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af0a74550 con 0x561af0a73b60 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f47659cb640 1 --2- 172.21.15.63:0/990151196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a73f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f476004cca0 tx=0x7f4760012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a3f478fb826621 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f4648ff9640 1 -- 172.21.15.63:0/990151196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760042030 con 0x561af0a73b60 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f4648ff9640 1 -- 172.21.15.63:0/990151196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4760001310 con 0x561af0a73b60 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.349+0000 7f4648ff9640 1 -- 172.21.15.63:0/990151196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760027040 con 0x561af0a73b60 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 -- 172.21.15.63:0/990151196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 msgr2=0x561af0a73f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 --2- 172.21.15.63:0/990151196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a73f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f476004cca0 tx=0x7f4760012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 -- 172.21.15.63:0/990151196 shutdown_connections 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 --2- 172.21.15.63:0/990151196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a73f80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 -- 172.21.15.63:0/990151196 >> 172.21.15.63:0/990151196 conn(0x561af0a6fec0 msgr2=0x561af0a6df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 -- 172.21.15.63:0/990151196 shutdown_connections 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 -- 172.21.15.63:0/990151196 wait complete. 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 Processor -- start 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 -- start start 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a7b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f476951f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561af0a74c60 con 0x561af0a73b60 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f47659cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a7b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f47659cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a7b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48688/0 (socket says 172.21.15.63:48688) 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.350+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 learned_addr learned my addr 172.21.15.63:0/4121717493 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af0a7b630 con 0x561af0a73b60 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f47659cb640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a7b0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4760001bc0 tx=0x7f476004cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f463affd640 1 -- 172.21.15.63:0/4121717493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760027030 con 0x561af0a73b60 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f463affd640 1 -- 172.21.15.63:0/4121717493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4760021e70 con 0x561af0a73b60 2024-04-23T19:10:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561af0a7b8c0 con 0x561af0a73b60 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f463affd640 1 -- 172.21.15.63:0/4121717493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476002a350 con 0x561af0a73b60 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af0a7e570 con 0x561af0a73b60 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.351+0000 7f463affd640 1 -- 172.21.15.63:0/4121717493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4760045070 con 0x561af0a73b60 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f463affd640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4630036e60 0x7f4630039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f463affd640 1 -- 172.21.15.63:0/4121717493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4760043070 con 0x561af0a73b60 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f47651ca640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4630036e60 0x7f4630039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af0a86dc0 0x561af0a89280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a899d0 con 0x561af0a86dc0 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f47649c9640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af0a86dc0 0x561af0a89280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f47651ca640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4630036e60 0x7f4630039320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f475804d150 tx=0x7f475804d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.352+0000 7f47649c9640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af0a86dc0 0x561af0a89280 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 realm 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a89da0 con 0x561af0a86dc0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8a670 con 0x561af0a86dc0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8aa10 con 0x561af0a86dc0 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.353+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8ae30 con 0x561af0a86dc0 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f476951f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f476951f800 20 searching for the correct realm 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561af0a8ce30 0x561af0a8f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f47659cb640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561af0a8ce30 0x561af0a8f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.354+0000 7f47659cb640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561af0a8ce30 0x561af0a8f280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47659cb640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47600123a0 0x7f4760046400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f47600123a0 2024-04-23T19:10:27.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47651ca640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47600123a0 0x7f4760046400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47651ca640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47600123a0 0x7f4760046400 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47651ca640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4758049040 0x7f47580165b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.355+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.356+0000 7f47649c9640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4758049040 0x7f47580165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.356+0000 7f47649c9640 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4758049040 0x7f47580165b0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.356+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.356+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.357+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f47600123a0 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.358+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f47600123a0 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.359+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.360+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.361+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.361+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.361+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.361+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.361+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.361+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f47600123a0 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760029070 con 0x561af0a8ce30 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.362+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.363+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.363+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f47600123a0 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.363+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.363+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a8ce30 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.363+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760069040 con 0x561af0a8ce30 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.363+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f4758049040 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x561af0a86dc0 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8f950 con 0x7f47600123a0 2024-04-23T19:10:27.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.364+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8b8d0 con 0x561af0a86dc0 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8c170 con 0x561af0a86dc0 2024-04-23T19:10:27.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8b510 con 0x7f4758049040 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 zone default found 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 4 Realm: () 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 zonegroup default 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.365+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a8c170 con 0x7f4758049040 2024-04-23T19:10:27.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a91800 con 0x561af0a8ce30 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4760069040 con 0x561af0a8ce30 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a91ba0 con 0x561af0a8ce30 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4760069040 con 0x561af0a8ce30 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 20 started sync module instance, tier type = 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.366+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a90ca0 con 0x7f47600123a0 2024-04-23T19:10:27.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.367+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.367+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94673706750144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a92460 con 0x7f47600123a0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.367+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a92cb0 con 0x561af0a86dc0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.368+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94673706750144] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.369+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.369+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94673706756096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a930d0 con 0x561af0a86dc0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.369+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a93920 con 0x561af0a8ce30 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.370+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94673706756096] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.370+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4760069040 con 0x561af0a8ce30 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.370+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94673706757664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a93cc0 con 0x561af0a8ce30 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.370+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a94710 con 0x7f47600123a0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.371+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94673706757664] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4760069040 con 0x561af0a8ce30 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.371+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.371+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94673706762752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a94b00 con 0x7f47600123a0 2024-04-23T19:10:27.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.371+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a95530 con 0x561af0a86dc0 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.372+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.372+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94673706762752] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.372+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94673706758816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a95950 con 0x561af0a86dc0 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.372+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a96350 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.373+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.373+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94673706770016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a962c0 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.373+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a97300 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.373+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94673706758816] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.374+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94673706770016] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.374+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.374+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94673706766400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a96e10 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.374+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a977d0 con 0x7f4758049040 2024-04-23T19:10:27.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94673706775328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a97740 con 0x7f4758049040 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 20 add_watcher() i=0 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 20 add_watcher() i=1 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 20 add_watcher() i=2 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 20 add_watcher() i=3 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 20 add_watcher() i=4 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 20 add_watcher() i=5 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94673706766400] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.375+0000 7f476951f800 20 add_watcher() i=6 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.376+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94673706775328] v30'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.376+0000 7f476951f800 20 add_watcher() i=7 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.376+0000 7f476951f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:27.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.376+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0a966f0 con 0x561af0a86dc0 2024-04-23T19:10:27.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.377+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.377+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94673706777360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a966f0 con 0x561af0a86dc0 2024-04-23T19:10:27.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94673706777360] v30'66 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f461b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f476951f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:27.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f476951f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f476951f800 5 note: GC not initialized 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f461a7e4640 20 reqs_thread_entry: start 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f476951f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f476951f800 20 Filter name: none 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f458bfff640 20 reqs_thread_entry: start 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f476951f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.378+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561af0acdb40 con 0x561af0a86dc0 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.379+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.379+0000 7f476951f800 20 rados_obj.operate() r=0 bl.length=502 2024-04-23T19:10:27.390 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-23T19:10:27.250094Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.380+0000 7f461b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.380+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94673706777360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0acc710 con 0x561af0a86dc0 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.381+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94673706777360] v30'67 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.381+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94673706750144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a97b10 con 0x7f47600123a0 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.382+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94673706750144] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.382+0000 7f476951f800 20 remove_watcher() i=0 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.382+0000 7f476951f800 2 removed watcher, disabling cache 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.382+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94673706756096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a910c0 con 0x561af0a86dc0 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.383+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94673706756096] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.383+0000 7f476951f800 20 remove_watcher() i=1 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.383+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94673706757664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a910c0 con 0x561af0a8ce30 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.384+0000 7f47659cb640 1 -- 172.21.15.63:0/4121717493 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94673706757664] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4760069040 con 0x561af0a8ce30 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.384+0000 7f476951f800 20 remove_watcher() i=2 2024-04-23T19:10:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.384+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94673706762752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a92e20 con 0x7f47600123a0 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.385+0000 7f47651ca640 1 -- 172.21.15.63:0/4121717493 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94673706762752] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475803f050 con 0x7f47600123a0 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.385+0000 7f476951f800 20 remove_watcher() i=3 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.385+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94673706758816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a910c0 con 0x561af0a86dc0 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.386+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94673706758816] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x561af0a86dc0 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.386+0000 7f476951f800 20 remove_watcher() i=4 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.386+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94673706770016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a94200 con 0x7f4758049040 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.387+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94673706770016] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.387+0000 7f476951f800 20 remove_watcher() i=5 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.387+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94673706766400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a92e20 con 0x7f4758049040 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.388+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94673706766400] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.388+0000 7f476951f800 20 remove_watcher() i=6 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.388+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94673706775328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561af0a910c0 con 0x7f4758049040 2024-04-23T19:10:27.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94673706775328] v30'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f475c05a040 con 0x7f4758049040 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 20 remove_watcher() i=7 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4758049040 msgr2=0x7f47580165b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4758049040 0x7f47580165b0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af0a86dc0 msgr2=0x561af0a89280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af0a86dc0 0x561af0a89280 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47600123a0 msgr2=0x7f4760046400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47600123a0 0x7f4760046400 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561af0a8ce30 msgr2=0x561af0a8f280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561af0a8ce30 0x561af0a8f280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4630036e60 msgr2=0x7f4630039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4630036e60 0x7f4630039320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f475804d150 tx=0x7f475804d000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 msgr2=0x561af0a7b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 --2- 172.21.15.63:0/4121717493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0a73b60 0x561af0a7b0f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4760001bc0 tx=0x7f476004cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 reap_dead start 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f47649c9640 1 -- 172.21.15.63:0/4121717493 reap_dead start 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 shutdown_connections 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 >> 172.21.15.63:0/4121717493 conn(0x561af0a6fec0 msgr2=0x561af0a6d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 shutdown_connections 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.389+0000 7f476951f800 1 -- 172.21.15.63:0/4121717493 wait complete. 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f463c036e60 msgr2=0x7f463c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f463c036e60 0x7f463c039320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f475800efd0 tx=0x7f475804b4c0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950740 msgr2=0x561af094ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950740 0x561af094ebb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x561af0a4a2f0 tx=0x7f47600178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 shutdown_connections 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f463c036e60 0x7f463c039320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/515811294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950740 0x561af094ebb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 >> 172.21.15.63:0/515811294 conn(0x561af0a58720 msgr2=0x561af0a58b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 shutdown_connections 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/515811294 wait complete. 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561af0a53860 msgr2=0x561af0a55c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561af0a53860 0x561af0a55c50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af09546b0 msgr2=0x561af0a50130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af09546b0 0x561af0a50130 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f464c036e60 msgr2=0x7f464c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f464c036e60 0x7f464c039320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f475c054000 tx=0x7f475c04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 msgr2=0x561af0a48ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a48ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4758001de0 tx=0x7f4758000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 shutdown_connections 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561af0a53860 0x561af0a55c50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561af09546b0 0x561af0a50130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f464c036e60 0x7f464c039320 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 --2- 172.21.15.63:0/2491648074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561af0950ba0 0x561af0a48ef0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 >> 172.21.15.63:0/2491648074 conn(0x561af094a630 msgr2=0x561af0a3b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 shutdown_connections 2024-04-23T19:10:27.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.390+0000 7f476951f800 1 -- 172.21.15.63:0/2491648074 wait complete. 2024-04-23T19:10:27.402 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-23T19:10:27.250094Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.402 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-23T19:10:27.402 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-23T19:10:27.402 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-04-23T19:10:27.448 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:27.448 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:27.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.448+0000 7fe9d9d46800 1 Processor -- start 2024-04-23T19:10:27.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d9d46800 1 -- start start 2024-04-23T19:10:27.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d9d46800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561f365e56a0 0x561f365e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d9d46800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561f364937f0 con 0x561f365e1b90 2024-04-23T19:10:27.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d9d46800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x561f36482e20 con 0x561f365e56a0 2024-04-23T19:10:27.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d7044640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561f365e56a0 0x561f365e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d7845640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x561f365e1b90 0x561f365e3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52756/0 (socket says 172.21.15.63:52756) 2024-04-23T19:10:27.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d7845640 1 -- 172.21.15.63:0/2593950091 learned_addr learned my addr 172.21.15.63:0/2593950091 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:27.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d7044640 1 -- 172.21.15.63:0/2593950091 >> v1:172.21.15.16:6789/0 conn(0x561f365e1b90 legacy=0x561f365e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.449+0000 7fe9d7044640 1 -- 172.21.15.63:0/2593950091 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f364a1b50 con 0x561f365e56a0 2024-04-23T19:10:27.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.451+0000 7fe9d7044640 1 --2- 172.21.15.63:0/2593950091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e56a0 0x561f365e7a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00a470 tx=0x7fe9cc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7003e6c860659c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.451+0000 7fe9d6843640 1 -- 172.21.15.63:0/2593950091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc057070 con 0x561f365e56a0 2024-04-23T19:10:27.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.451+0000 7fe9d6843640 1 -- 172.21.15.63:0/2593950091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc052570 con 0x561f365e56a0 2024-04-23T19:10:27.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.451+0000 7fe9d6843640 1 -- 172.21.15.63:0/2593950091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc052870 con 0x561f365e56a0 2024-04-23T19:10:27.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.452+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2593950091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e56a0 msgr2=0x561f365e7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.452+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2593950091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e56a0 0x561f365e7a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00a470 tx=0x7fe9cc01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:27.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.452+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2593950091 shutdown_connections 2024-04-23T19:10:27.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.452+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2593950091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e56a0 0x561f365e7a90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.452+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2593950091 >> 172.21.15.63:0/2593950091 conn(0x561f365db600 msgr2=0x561f365dda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.452+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2593950091 shutdown_connections 2024-04-23T19:10:27.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.452+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2593950091 wait complete. 2024-04-23T19:10:27.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d9d46800 1 Processor -- start 2024-04-23T19:10:27.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d9d46800 1 -- start start 2024-04-23T19:10:27.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d9d46800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f365e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d9d46800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f3652c6f0 con 0x561f365e1b90 2024-04-23T19:10:27.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d7845640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f365e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d7845640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f365e14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48696/0 (socket says 172.21.15.63:48696) 2024-04-23T19:10:27.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d7845640 1 -- 172.21.15.63:0/1528659478 learned_addr learned my addr 172.21.15.63:0/1528659478 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.465+0000 7fe9d7845640 1 -- 172.21.15.63:0/1528659478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f365dfa90 con 0x561f365e1b90 2024-04-23T19:10:27.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe9d7845640 1 --2- 172.21.15.63:0/1528659478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f365e14d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001de0 tx=0x7fe9c800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d4bc72f3ee0073 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe8c5e0b640 1 -- 172.21.15.63:0/1528659478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c8012020 con 0x561f365e1b90 2024-04-23T19:10:27.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe8c5e0b640 1 -- 172.21.15.63:0/1528659478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c800e5a0 con 0x561f365e1b90 2024-04-23T19:10:27.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe8c5e0b640 1 -- 172.21.15.63:0/1528659478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c800e8c0 con 0x561f365e1b90 2024-04-23T19:10:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1528659478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 msgr2=0x561f365e14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/1528659478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f365e14d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001de0 tx=0x7fe9c800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1528659478 shutdown_connections 2024-04-23T19:10:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/1528659478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f365e14d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.466+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1528659478 >> 172.21.15.63:0/1528659478 conn(0x561f365db600 msgr2=0x561f366cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1528659478 shutdown_connections 2024-04-23T19:10:27.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1528659478 wait complete. 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d9d46800 1 Processor -- start 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d9d46800 1 -- start start 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d9d46800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f366dc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d9d46800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f365e82d0 con 0x561f365e1b90 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d7845640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f366dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d7845640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f366dc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48712/0 (socket says 172.21.15.63:48712) 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d7845640 1 -- 172.21.15.63:0/662752135 learned_addr learned my addr 172.21.15.63:0/662752135 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.467+0000 7fe9d7845640 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f366dc660 con 0x561f365e1b90 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.468+0000 7fe9d7845640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f366dc120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001de0 tx=0x7fe9c8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.468+0000 7fe8bffff640 1 -- 172.21.15.63:0/662752135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c8012030 con 0x561f365e1b90 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.468+0000 7fe8bffff640 1 -- 172.21.15.63:0/662752135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c801a7a0 con 0x561f365e1b90 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.468+0000 7fe8bffff640 1 -- 172.21.15.63:0/662752135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c801aac0 con 0x561f365e1b90 2024-04-23T19:10:27.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.468+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f366dc8f0 con 0x561f365e1b90 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.468+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f366df5a0 con 0x561f365e1b90 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe8bffff640 1 -- 172.21.15.63:0/662752135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe9c8015020 con 0x561f365e1b90 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe8bffff640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8c0036e60 0x7fe8c0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe8bffff640 1 -- 172.21.15.63:0/662752135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe9c8020070 con 0x561f365e1b90 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe9d7044640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8c0036e60 0x7fe8c0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f365e56a0 0x561f366e3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f366e38a0 con 0x561f365e56a0 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe9d8046640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f365e56a0 0x561f366e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe9d7044640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8c0036e60 0x7fe8c0039320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc054000 tx=0x7fe9cc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.469+0000 7fe9d8046640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f365e56a0 0x561f366e3360 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.470+0000 7fe9d8046640 1 -- 172.21.15.63:0/662752135 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe9d000a4a0 con 0x561f365e56a0 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.470+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f366e0930 con 0x561f365e56a0 2024-04-23T19:10:27.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.470+0000 7fe9d8046640 1 -- 172.21.15.63:0/662752135 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe9d000a4a0 con 0x561f365e56a0 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.470+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f366e56c0 con 0x561f365e56a0 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.470+0000 7fe9d8046640 1 -- 172.21.15.63:0/662752135 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe9d000a4a0 con 0x561f365e56a0 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.470+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f366e4a00 con 0x561f365e56a0 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.471+0000 7fe9d8046640 1 -- 172.21.15.63:0/662752135 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe9d000a4a0 con 0x561f365e56a0 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.471+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561f366e6a70 0x561f366e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.471+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f366e93e0 con 0x561f366e6a70 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.471+0000 7fe9d7845640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561f366e6a70 0x561f366e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.471+0000 7fe9d7845640 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561f366e6a70 0x561f366e8e60 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.471+0000 7fe9d7845640 1 -- 172.21.15.63:0/662752135 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe9c8024980 con 0x561f366e6a70 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.472+0000 7fe9d9d46800 1 Processor -- start 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.472+0000 7fe9d9d46800 1 -- start start 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.472+0000 7fe9d9d46800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f366f7800 0x561f366f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.472+0000 7fe9d9d46800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f364d2e30 con 0x561f366f7800 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.472+0000 7fe9d8046640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f366f7800 0x561f366f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.472+0000 7fe9d8046640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f366f7800 0x561f366f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48724/0 (socket says 172.21.15.63:48724) 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.472+0000 7fe9d8046640 1 -- 172.21.15.63:0/4227298430 learned_addr learned my addr 172.21.15.63:0/4227298430 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.473+0000 7fe9d8046640 1 -- 172.21.15.63:0/4227298430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f366f8160 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.473+0000 7fe9d8046640 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f366f7800 0x561f366f7c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561f366dce30 tx=0x7fe9d00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.473+0000 7fe8bd7fa640 1 -- 172.21.15.63:0/4227298430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0013440 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.473+0000 7fe8bd7fa640 1 -- 172.21.15.63:0/4227298430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d00135e0 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.473+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f366f8bd0 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.473+0000 7fe8bd7fa640 1 -- 172.21.15.63:0/4227298430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d00138c0 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.473+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f366f90d0 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe8bd7fa640 1 -- 172.21.15.63:0/4227298430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe9d0011040 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe8bd7fa640 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8b0036e60 0x7fe8b0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe8bd7fa640 1 -- 172.21.15.63:0/4227298430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe9d001d070 con 0x561f366f7800 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe9d7845640 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8b0036e60 0x7fe8b0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe9d9d46800 1 Processor -- start 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe9d9d46800 1 -- start start 2024-04-23T19:10:27.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe9d7845640 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8b0036e60 0x7fe8b0039320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8024c90 tx=0x7fe9c8017f30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe9d9d46800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f36704f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.474+0000 7fe9d9d46800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f365dfe10 con 0x561f36704b60 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe9d8046640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f36704f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe9d8046640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f36704f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48740/0 (socket says 172.21.15.63:48740) 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe9d8046640 1 -- 172.21.15.63:0/1939284197 learned_addr learned my addr 172.21.15.63:0/1939284197 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe9d8046640 1 -- 172.21.15.63:0/1939284197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f36705550 con 0x561f36704b60 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe9d8046640 1 --2- 172.21.15.63:0/1939284197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f36704f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9d004ccb0 tx=0x7fe9d0012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f89b7f9d70e025 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe8b6ffd640 1 -- 172.21.15.63:0/1939284197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0012df0 con 0x561f36704b60 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe8b6ffd640 1 -- 172.21.15.63:0/1939284197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d0001310 con 0x561f36704b60 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.475+0000 7fe8b6ffd640 1 -- 172.21.15.63:0/1939284197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0001630 con 0x561f36704b60 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1939284197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 msgr2=0x561f36704f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/1939284197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f36704f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9d004ccb0 tx=0x7fe9d0012910 comp rx=0 tx=0).stop 2024-04-23T19:10:27.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1939284197 shutdown_connections 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/1939284197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f36704f80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1939284197 >> 172.21.15.63:0/1939284197 conn(0x561f36700e80 msgr2=0x561f366feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1939284197 shutdown_connections 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 -- 172.21.15.63:0/1939284197 wait complete. 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 Processor -- start 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 -- start start 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f3670c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d9d46800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f36705cd0 con 0x561f36704b60 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d8046640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f3670c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d8046640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f3670c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48744/0 (socket says 172.21.15.63:48744) 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.476+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 learned_addr learned my addr 172.21.15.63:0/2556069792 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.477+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f3670c690 con 0x561f36704b60 2024-04-23T19:10:27.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.477+0000 7fe9d8046640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f3670c150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9d004bb80 tx=0x7fe9d004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.477+0000 7fe8b57fa640 1 -- 172.21.15.63:0/2556069792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0001c30 con 0x561f36704b60 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.477+0000 7fe8b57fa640 1 -- 172.21.15.63:0/2556069792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d002ae30 con 0x561f36704b60 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.477+0000 7fe8b57fa640 1 -- 172.21.15.63:0/2556069792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0021db0 con 0x561f36704b60 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.477+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f3670c920 con 0x561f36704b60 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.477+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f3670f5d0 con 0x561f36704b60 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe8b57fa640 1 -- 172.21.15.63:0/2556069792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe9d0040040 con 0x561f36704b60 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe8b57fa640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8a0036e60 0x7fe8a0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe8b57fa640 1 -- 172.21.15.63:0/2556069792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe9d0032070 con 0x561f36704b60 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe9d7845640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8a0036e60 0x7fe8a0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe9d7845640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8a0036e60 0x7fe8a0039320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001400 tx=0x7fe9c8037000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f36717d70 0x561f3671a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.478+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671a980 con 0x561f36717d70 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d7044640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f36717d70 0x561f3671a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d7044640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f36717d70 0x561f3671a230 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 20 realm 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671ad50 con 0x561f36717d70 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.479+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671b620 con 0x561f36717d70 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671b9c0 con 0x561f36717d70 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671c210 con 0x561f36717d70 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 20 searching for the correct realm 2024-04-23T19:10:27.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561f3671df80 0x561f36720370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.480+0000 7fe9d8046640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561f3671df80 0x561f36720370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.481+0000 7fe9d8046640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561f3671df80 0x561f36720370 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.481+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.481+0000 7fe9d8046640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe9d00123a0 0x7fe9d001ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.481+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9d00123a0 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.481+0000 7fe9d7845640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe9d00123a0 0x7fe9d001ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.482+0000 7fe9d7845640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe9d00123a0 0x7fe9d001ba60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.482+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.482+0000 7fe9d7845640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe9c8054040 0x7fe9c804b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.482+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.482+0000 7fe9d7044640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe9c8054040 0x7fe9c804b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.482+0000 7fe9d7044640 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe9c8054040 0x7fe9c804b550 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.483+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9d00123a0 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.484+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.485+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.485+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.485+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.485+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9d00123a0 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.486+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.487+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.488+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.488+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.488+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.488+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9d00123a0 2024-04-23T19:10:27.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.488+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.488+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.488+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.489+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.489+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.489+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.489+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.489+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.489+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.489+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9d00123a0 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.490+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.490+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f3671df80 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.490+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.490+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9c8054040 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.490+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.490+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x561f36717d70 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561f36720900 con 0x7fe9d00123a0 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671cb90 con 0x561f36717d70 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671d430 con 0x561f36717d70 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.491+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3671d430 con 0x7fe9c8054040 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 20 zone default found 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 4 Realm: () 2024-04-23T19:10:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 20 zonegroup default 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f36720d80 con 0x7fe9c8054040 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.492+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f36722c10 con 0x561f3671df80 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d9d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f36722fb0 con 0x561f3671df80 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d9d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d9d46800 20 started sync module instance, tier type = 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d9d46800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.493+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36722050 con 0x7fe9d00123a0 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.494+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.494+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94692057425008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36723800 con 0x7fe9d00123a0 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.494+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36724050 con 0x561f36717d70 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.495+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.495+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94692057430944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36724470 con 0x561f36717d70 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.495+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36724f70 con 0x561f3671df80 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.495+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94692057425008] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.496+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94692057430944] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.496+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.496+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94692057434720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36723fc0 con 0x561f3671df80 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.496+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36725810 con 0x7fe9d00123a0 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.497+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.497+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94692057434720] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.497+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94692057428816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36723fc0 con 0x7fe9d00123a0 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.498+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f367262d0 con 0x561f36717d70 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.499+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.499+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94692057439648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36726b70 con 0x561f36717d70 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.499+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f367275f0 con 0x7fe9c8054040 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.499+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94692057428816] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.500+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.500+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94692057444576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f367275b0 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.500+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f367275b0 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.500+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94692057439648] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.500+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94692057444576] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.501+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.501+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94692057437024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36727b80 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.501+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36728540 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94692057437024] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94692057441984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36728540 con 0x7fe9c8054040 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 20 add_watcher() i=0 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 20 add_watcher() i=1 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 20 add_watcher() i=2 2024-04-23T19:10:27.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 20 add_watcher() i=3 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 20 add_watcher() i=4 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 20 add_watcher() i=5 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.502+0000 7fe9d9d46800 20 add_watcher() i=6 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.503+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94692057441984] v30'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.503+0000 7fe9d9d46800 20 add_watcher() i=7 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.503+0000 7fe9d9d46800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.503+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3672b800 con 0x561f36717d70 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.503+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.503+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94692057448960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36728ad0 con 0x561f36717d70 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.504+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94692057448960] v30'68 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.504+0000 7fe88dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.504+0000 7fe9d9d46800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:27.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.504+0000 7fe9d9d46800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.504+0000 7fe9d9d46800 5 note: GC not initialized 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe88cfe1640 20 reqs_thread_entry: start 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 20 Filter name: none 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe8067fc640 20 reqs_thread_entry: start 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f3675ed00 con 0x561f36717d70 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 20 rados_obj.operate() r=0 bl.length=502 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:27.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.505+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f367650d0 con 0x561f36717d70 2024-04-23T19:10:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.506+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v30'5 uv5 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.506+0000 7fe9d9d46800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.506+0000 7fe9d9d46800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.506+0000 7fe9d9d46800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.506+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94692057697088 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f36760920 con 0x561f36717d70 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94692057439648 notify 128849018884 ret 0) ==== 609+0+0 (crc 0 0 0) 0x561f364a1b50 con 0x561f36717d70 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe8b6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018884 cookie 94692057439648 notifier 4352 bl.length()=567 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe8b6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(64 notify.4 [notify cookie 94692057697088 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe8b6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe8b6ffd640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fe89c003ea0 con 0x561f36717d70 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== watch-notify(notify_complete (2) cookie 94692057697088 notify 128849018884 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fe9cc05dc90 con 0x561f36717d70 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.507+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:66 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36760920 con 0x7fe9c8054040 2024-04-23T19:10:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.508+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(66 p5YnriCv1nAtykxBrupQ [delete] v30'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.508+0000 7fe9d9d46800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.508+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94692057697088 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561f36760920 con 0x561f3671df80 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.508+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== watch-notify(notify (1) cookie 94692057434720 notify 128849018881 ret 0) ==== 212+0+0 (crc 0 0 0) 0x561f3670f5d0 con 0x561f3671df80 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.508+0000 7fe8b6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018881 cookie 94692057434720 notifier 4352 bl.length()=170 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.508+0000 7fe8b6ffd640 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fe89c003ea0 con 0x561f3671df80 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.508+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 notify.2 [notify cookie 94692057697088 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.509+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== watch-notify(notify_complete (2) cookie 94692057697088 notify 128849018881 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561f3670c920 con 0x561f3671df80 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.509+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.522 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.510+0000 7fe88dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.510+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94692057448960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f3672ae50 con 0x561f36717d70 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.511+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94692057448960] v30'69 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.511+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94692057425008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36728200 con 0x7fe9d00123a0 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.512+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94692057425008] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.512+0000 7fe9d9d46800 20 remove_watcher() i=0 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.512+0000 7fe9d9d46800 2 removed watcher, disabling cache 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.512+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94692057430944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36722470 con 0x561f36717d70 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.513+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94692057430944] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.513+0000 7fe9d9d46800 20 remove_watcher() i=1 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.513+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94692057434720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36722470 con 0x561f3671df80 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.514+0000 7fe9d8046640 1 -- 172.21.15.63:0/2556069792 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94692057434720] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9d0029070 con 0x561f3671df80 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.514+0000 7fe9d9d46800 20 remove_watcher() i=2 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.514+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94692057428816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36722470 con 0x7fe9d00123a0 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.515+0000 7fe9d7845640 1 -- 172.21.15.63:0/2556069792 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94692057428816] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9c8042040 con 0x7fe9d00123a0 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.515+0000 7fe9d9d46800 20 remove_watcher() i=3 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.515+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94692057439648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36723350 con 0x561f36717d70 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.516+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94692057439648] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x561f36717d70 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.516+0000 7fe9d9d46800 20 remove_watcher() i=4 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.516+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94692057444576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36722470 con 0x7fe9c8054040 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.517+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94692057444576] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.517+0000 7fe9d9d46800 20 remove_watcher() i=5 2024-04-23T19:10:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.517+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94692057437024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f367270e0 con 0x7fe9c8054040 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.518+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94692057437024] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.518+0000 7fe9d9d46800 20 remove_watcher() i=6 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.518+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94692057441984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561f36722470 con 0x7fe9c8054040 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.518+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94692057441984] v30'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe9cc05a040 con 0x7fe9c8054040 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.518+0000 7fe9d9d46800 20 remove_watcher() i=7 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe9c8054040 msgr2=0x7fe9c804b550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe9c8054040 0x7fe9c804b550 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f36717d70 msgr2=0x561f3671a230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f36717d70 0x561f3671a230 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe9d00123a0 msgr2=0x7fe9d001ba60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe9d00123a0 0x7fe9d001ba60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561f3671df80 msgr2=0x561f36720370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561f3671df80 0x561f36720370 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8a0036e60 msgr2=0x7fe8a0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8a0036e60 0x7fe8a0039320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001400 tx=0x7fe9c8037000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 msgr2=0x561f3670c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d7044640 1 -- 172.21.15.63:0/2556069792 reap_dead start 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f3670c150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9d004bb80 tx=0x7fe9d004cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 shutdown_connections 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/2556069792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f36704b60 0x561f3670c150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 >> 172.21.15.63:0/2556069792 conn(0x561f36700e80 msgr2=0x561f366fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 shutdown_connections 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/2556069792 wait complete. 2024-04-23T19:10:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8b0036e60 msgr2=0x7fe8b0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8b0036e60 0x7fe8b0039320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8024c90 tx=0x7fe9c8017f30 comp rx=0 tx=0).stop 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f366f7800 msgr2=0x561f366f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f366f7800 0x561f366f7c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561f366dce30 tx=0x7fe9d00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 shutdown_connections 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8b0036e60 0x7fe8b0039320 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/4227298430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f366f7800 0x561f366f7c20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 >> 172.21.15.63:0/4227298430 conn(0x561f366eb8a0 msgr2=0x561f366ebca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 shutdown_connections 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.519+0000 7fe9d9d46800 1 -- 172.21.15.63:0/4227298430 wait complete. 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561f366e6a70 msgr2=0x561f366e8e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561f366e6a70 0x561f366e8e60 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f365e56a0 msgr2=0x561f366e3360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f365e56a0 0x561f366e3360 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8c0036e60 msgr2=0x7fe8c0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8c0036e60 0x7fe8c0039320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc054000 tx=0x7fe9cc04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 msgr2=0x561f366dc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f366dc120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8001de0 tx=0x7fe9c8000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 shutdown_connections 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561f366e6a70 0x561f366e8e60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561f365e56a0 0x561f366e3360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe8c0036e60 0x7fe8c0039320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 --2- 172.21.15.63:0/662752135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f365e1b90 0x561f366dc120 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 >> 172.21.15.63:0/662752135 conn(0x561f365db600 msgr2=0x561f365e0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 shutdown_connections 2024-04-23T19:10:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.520+0000 7fe9d9d46800 1 -- 172.21.15.63:0/662752135 wait complete. 2024-04-23T19:10:27.534 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.534 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-23T19:10:27.535 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-23T19:10:27.535 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-04-23T19:10:27.581 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:27.581 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.581+0000 7ff6c8040800 1 Processor -- start 2024-04-23T19:10:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.581+0000 7ff6c8040800 1 -- start start 2024-04-23T19:10:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c8040800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56554918ea40 0x56554918ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c8040800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5655490427f0 con 0x56554918f3a0 2024-04-23T19:10:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c8040800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x565549031e20 con 0x56554918ea40 2024-04-23T19:10:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c2cdc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56554918ea40 0x56554918ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c24db640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x56554918f3a0 0x56554918f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:52770/0 (socket says 172.21.15.63:52770) 2024-04-23T19:10:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c24db640 1 -- 172.21.15.63:0/627149023 learned_addr learned my addr 172.21.15.63:0/627149023 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/627149023 >> v1:172.21.15.16:6789/0 conn(0x56554918f3a0 legacy=0x56554918f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.582+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/627149023 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x565549050b50 con 0x56554918ea40 2024-04-23T19:10:27.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.584+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/627149023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554918ee60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400a470 tx=0x7ff6b401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b55191e527b182 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.584+0000 7ff6c1cda640 1 -- 172.21.15.63:0/627149023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b4057070 con 0x56554918ea40 2024-04-23T19:10:27.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.584+0000 7ff6c1cda640 1 -- 172.21.15.63:0/627149023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b404adc0 con 0x56554918ea40 2024-04-23T19:10:27.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.584+0000 7ff6c1cda640 1 -- 172.21.15.63:0/627149023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b4052590 con 0x56554918ea40 2024-04-23T19:10:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.585+0000 7ff6c8040800 1 -- 172.21.15.63:0/627149023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 msgr2=0x56554918ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.585+0000 7ff6c8040800 1 --2- 172.21.15.63:0/627149023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554918ee60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400a470 tx=0x7ff6b401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.585+0000 7ff6c8040800 1 -- 172.21.15.63:0/627149023 shutdown_connections 2024-04-23T19:10:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.585+0000 7ff6c8040800 1 --2- 172.21.15.63:0/627149023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554918ee60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.585+0000 7ff6c8040800 1 -- 172.21.15.63:0/627149023 >> 172.21.15.63:0/627149023 conn(0x56554918a600 msgr2=0x56554918ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.585+0000 7ff6c8040800 1 -- 172.21.15.63:0/627149023 shutdown_connections 2024-04-23T19:10:27.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.585+0000 7ff6c8040800 1 -- 172.21.15.63:0/627149023 wait complete. 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c8040800 1 Processor -- start 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c8040800 1 -- start start 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c8040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x565549283660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c8040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5655490db6f0 con 0x56554918ea40 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c2cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x565549283660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c2cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x565549283660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48750/0 (socket says 172.21.15.63:48750) 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/642050475 learned_addr learned my addr 172.21.15.63:0/642050475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.598+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/642050475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565549283c30 con 0x56554918ea40 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/642050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x565549283660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4055040 tx=0x7ff6b404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c971c857748ead8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff5b0e09640 1 -- 172.21.15.63:0/642050475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b4057040 con 0x56554918ea40 2024-04-23T19:10:27.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff5b0e09640 1 -- 172.21.15.63:0/642050475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b4063070 con 0x56554918ea40 2024-04-23T19:10:27.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff5b0e09640 1 -- 172.21.15.63:0/642050475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b405d790 con 0x56554918ea40 2024-04-23T19:10:27.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff6c8040800 1 -- 172.21.15.63:0/642050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 msgr2=0x565549283660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff6c8040800 1 --2- 172.21.15.63:0/642050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x565549283660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4055040 tx=0x7ff6b404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff6c8040800 1 -- 172.21.15.63:0/642050475 shutdown_connections 2024-04-23T19:10:27.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff6c8040800 1 --2- 172.21.15.63:0/642050475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x565549283660 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.599+0000 7ff6c8040800 1 -- 172.21.15.63:0/642050475 >> 172.21.15.63:0/642050475 conn(0x56554918a600 msgr2=0x56554927ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c8040800 1 -- 172.21.15.63:0/642050475 shutdown_connections 2024-04-23T19:10:27.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c8040800 1 -- 172.21.15.63:0/642050475 wait complete. 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c8040800 1 Processor -- start 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c8040800 1 -- start start 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c8040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554928b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c8040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565549190200 con 0x56554918ea40 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c2cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554928b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c2cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554928b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48754/0 (socket says 172.21.15.63:48754) 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.600+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/388876716 learned_addr learned my addr 172.21.15.63:0/388876716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.601+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554928b640 con 0x56554918ea40 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.601+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554928b100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4060040 tx=0x7ff6b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.601+0000 7ff5af606640 1 -- 172.21.15.63:0/388876716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b4057050 con 0x56554918ea40 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.601+0000 7ff5af606640 1 -- 172.21.15.63:0/388876716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b405dcd0 con 0x56554918ea40 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.601+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56554928b8d0 con 0x56554918ea40 2024-04-23T19:10:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.601+0000 7ff5af606640 1 -- 172.21.15.63:0/388876716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6b4052440 con 0x56554918ea40 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.601+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554928e580 con 0x56554918ea40 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff5af606640 1 -- 172.21.15.63:0/388876716 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff6b4052650 con 0x56554918ea40 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff5af606640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff5a8036e60 0x7ff5a8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff5af606640 1 -- 172.21.15.63:0/388876716 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff6b408e2e0 con 0x56554918ea40 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff6c24db640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff5a8036e60 0x7ff5a8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56554918f3a0 0x565549292340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492928c0 con 0x56554918f3a0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56554918f3a0 0x565549292340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff6c24db640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff5a8036e60 0x7ff5a8039320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8000f60 tx=0x7ff6b80012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.602+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56554918f3a0 0x565549292340 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.603+0000 7ff6c34dd640 1 -- 172.21.15.63:0/388876716 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x56554918f3a0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.603+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56554928f910 con 0x56554918f3a0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.603+0000 7ff6c34dd640 1 -- 172.21.15.63:0/388876716 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x56554918f3a0 2024-04-23T19:10:27.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.603+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x565549294710 con 0x56554918f3a0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.603+0000 7ff6c34dd640 1 -- 172.21.15.63:0/388876716 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x56554918f3a0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.603+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x565549293a50 con 0x56554918f3a0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.604+0000 7ff6c34dd640 1 -- 172.21.15.63:0/388876716 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x56554918f3a0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.604+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x565549295ac0 0x565549297eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.604+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x565549298430 con 0x565549295ac0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.604+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x565549295ac0 0x565549297eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.604+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x565549295ac0 0x565549297eb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/388876716 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff6b4078070 con 0x565549295ac0 2024-04-23T19:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c8040800 1 Processor -- start 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c8040800 1 -- start start 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c8040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492a6810 0x5655492a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c8040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565549081e30 con 0x5655492a6810 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c34dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492a6810 0x5655492a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c34dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492a6810 0x5655492a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48768/0 (socket says 172.21.15.63:48768) 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.605+0000 7ff6c34dd640 1 -- 172.21.15.63:0/1141862396 learned_addr learned my addr 172.21.15.63:0/1141862396 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.606+0000 7ff6c34dd640 1 -- 172.21.15.63:0/1141862396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655492a7170 con 0x5655492a6810 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.606+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492a6810 0x5655492a6c30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x565549292c80 tx=0x7ff6bc0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.606+0000 7ff5ace01640 1 -- 172.21.15.63:0/1141862396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6bc013440 con 0x5655492a6810 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.606+0000 7ff5ace01640 1 -- 172.21.15.63:0/1141862396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc0135e0 con 0x5655492a6810 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.606+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5655492a7be0 con 0x5655492a6810 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.606+0000 7ff5ace01640 1 -- 172.21.15.63:0/1141862396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6bc0138c0 con 0x5655492a6810 2024-04-23T19:10:27.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.606+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655492a80e0 con 0x5655492a6810 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.607+0000 7ff5ace01640 1 -- 172.21.15.63:0/1141862396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff6bc011040 con 0x5655492a6810 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.607+0000 7ff5ace01640 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff59c036e60 0x7ff59c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.607+0000 7ff5ace01640 1 -- 172.21.15.63:0/1141862396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff6bc01d070 con 0x5655492a6810 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.607+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff59c036e60 0x7ff59c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.607+0000 7ff6c8040800 1 Processor -- start 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.607+0000 7ff6c8040800 1 -- start start 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff59c036e60 0x7ff59c039320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff6b408e290 tx=0x7ff6b407d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c8040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c8040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565549283ef0 con 0x5655492b3bc0 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c34dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c34dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48784/0 (socket says 172.21.15.63:48784) 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c34dd640 1 -- 172.21.15.63:0/2358668671 learned_addr learned my addr 172.21.15.63:0/2358668671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c34dd640 1 -- 172.21.15.63:0/2358668671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655492b45b0 con 0x5655492b3bc0 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/2358668671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492b3fe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc04cd30 tx=0x7ff6bc04d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a776a0a69c825c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff5a67fc640 1 -- 172.21.15.63:0/2358668671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6bc015a80 con 0x5655492b3bc0 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff5a67fc640 1 -- 172.21.15.63:0/2358668671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc015430 con 0x5655492b3bc0 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.608+0000 7ff5a67fc640 1 -- 172.21.15.63:0/2358668671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6bc021bf0 con 0x5655492b3bc0 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 -- 172.21.15.63:0/2358668671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 msgr2=0x5655492b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 --2- 172.21.15.63:0/2358668671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492b3fe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc04cd30 tx=0x7ff6bc04d000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 -- 172.21.15.63:0/2358668671 shutdown_connections 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 --2- 172.21.15.63:0/2358668671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492b3fe0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 -- 172.21.15.63:0/2358668671 >> 172.21.15.63:0/2358668671 conn(0x5655492afe90 msgr2=0x5655492adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 -- 172.21.15.63:0/2358668671 shutdown_connections 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 -- 172.21.15.63:0/2358668671 wait complete. 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 Processor -- start 2024-04-23T19:10:27.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 -- start start 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492bb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c8040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5655492b4d40 con 0x5655492b3bc0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.609+0000 7ff6c34dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff6c34dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492bb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:48794/0 (socket says 172.21.15.63:48794) 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 learned_addr learned my addr 172.21.15.63:0/3893492164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655492bb5c0 con 0x5655492b3bc0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492bb080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc04cd30 tx=0x7ff6bc04ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff5a4ff9640 1 -- 172.21.15.63:0/3893492164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6bc02a560 con 0x5655492b3bc0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff5a4ff9640 1 -- 172.21.15.63:0/3893492164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc012ab0 con 0x5655492b3bc0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5655492bb8b0 con 0x5655492b3bc0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff5a4ff9640 1 -- 172.21.15.63:0/3893492164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6bc012d90 con 0x5655492b3bc0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.610+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655492be500 con 0x5655492b3bc0 2024-04-23T19:10:27.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.611+0000 7ff5a4ff9640 1 -- 172.21.15.63:0/3893492164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff6bc02b3a0 con 0x5655492b3bc0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.611+0000 7ff5a4ff9640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff58c036e60 0x7ff58c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.611+0000 7ff5a4ff9640 1 -- 172.21.15.63:0/3893492164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff6bc034030 con 0x5655492b3bc0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.611+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff58c036e60 0x7ff58c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.611+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff58c036e60 0x7ff58c039320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x565549298810 tx=0x7ff6b4083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5655492c6e10 0x5655492c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492c9a20 con 0x5655492c6e10 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c24db640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5655492c6e10 0x5655492c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c24db640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5655492c6e10 0x5655492c92d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff6b8009050 con 0x5655492c6e10 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c8040800 20 realm 2024-04-23T19:10:27.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.612+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492c6260 con 0x5655492c6e10 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff6b8009050 con 0x5655492c6e10 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492ca510 con 0x5655492c6e10 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff6b8009050 con 0x5655492c6e10 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492cade0 con 0x5655492c6e10 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff6b8009050 con 0x5655492c6e10 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492cb180 con 0x5655492c6e10 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff6b8009050 con 0x5655492c6e10 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.613+0000 7ff6c8040800 20 searching for the correct realm 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.614+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5655492cd060 0x5655492cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.614+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.614+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5655492cd060 0x5655492cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.614+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5655492cd060 0x5655492cf450 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.614+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.614+0000 7ff6c34dd640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff6bc041020 0x7ff6bc013a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.614+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6bc041020 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.615+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff6bc041020 0x7ff6bc013a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.615+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff6bc041020 0x7ff6bc013a60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.615+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.615+0000 7ff6c2cdc640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff6b409b040 0x7ff6b405e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.615+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.615+0000 7ff6c24db640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff6b409b040 0x7ff6b405e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.616+0000 7ff6c24db640 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff6b409b040 0x7ff6b405e900 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.616+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b8009050 con 0x7ff6b409b040 2024-04-23T19:10:27.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.616+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.616+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b8009050 con 0x5655492c6e10 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.616+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6bc041020 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.617+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.618+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6bc041020 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.619+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.620+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.621+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.621+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.621+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.621+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6bc041020 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.621+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.621+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.622+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.622+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.622+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.622+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.622+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.622+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.622+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.623+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6bc041020 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.623+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.623+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492cd060 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.623+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.623+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6b409b040 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.623+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.623+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x5655492c6e10 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5655492cfb00 con 0x7ff6bc041020 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492cbb00 con 0x5655492c6e10 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492cc3a0 con 0x5655492c6e10 2024-04-23T19:10:27.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.624+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492cff80 con 0x7ff6b409b040 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 zone default found 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 4 Realm: () 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 zonegroup default 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492d0430 con 0x7ff6b409b040 2024-04-23T19:10:27.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492d1e10 con 0x5655492cd060 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.625+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492d22c0 con 0x5655492cd060 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.626+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.626+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.626+0000 7ff6c8040800 20 started sync module instance, tier type = 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.626+0000 7ff6c8040800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.626+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d12a0 con 0x7ff6bc041020 2024-04-23T19:10:27.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.627+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.627+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94924299900560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d2bd0 con 0x7ff6bc041020 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.627+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d3420 con 0x5655492c6e10 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.628+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94924299900560] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.628+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.628+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94924299904800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d3460 con 0x5655492c6e10 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.628+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d3f70 con 0x5655492cd060 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.629+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94924299904800] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.629+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.629+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94924299909664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d4360 con 0x5655492cd060 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.629+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d4dc0 con 0x7ff6bc041020 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.630+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.630+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94924299913376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d5660 con 0x7ff6bc041020 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.630+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d6110 con 0x5655492c6e10 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.630+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94924299909664] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.631+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94924299913376] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.631+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.631+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94924299918368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d51b0 con 0x5655492c6e10 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.631+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d6730 con 0x7ff6b409b040 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.632+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.632+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94924299920000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d6b50 con 0x7ff6b409b040 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.633+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d7580 con 0x7ff6b409b040 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.633+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94924299918368] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.633+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94924299920000] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.634+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.634+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94924299921056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d2f70 con 0x7ff6b409b040 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.634+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d7a30 con 0x7ff6b409b040 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94924299924864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d79a0 con 0x7ff6b409b040 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 20 add_watcher() i=0 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 20 add_watcher() i=1 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 20 add_watcher() i=2 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 20 add_watcher() i=3 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 20 add_watcher() i=4 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 20 add_watcher() i=5 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94924299921056] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.635+0000 7ff6c8040800 20 add_watcher() i=6 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.636+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94924299924864] v30'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.636+0000 7ff6c8040800 20 add_watcher() i=7 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.636+0000 7ff6c8040800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.636+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5655492d7090 con 0x5655492c6e10 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.636+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.636+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94924299926992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d7090 con 0x5655492c6e10 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.637+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94924299926992] v30'70 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.637+0000 7ff5797e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.637+0000 7ff6c8040800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.637+0000 7ff6c8040800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.637+0000 7ff6c8040800 5 note: GC not initialized 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.637+0000 7ff4f3fff640 20 reqs_thread_entry: start 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.637+0000 7ff6c8040800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 20 Filter name: none 2024-04-23T19:10:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff4f1ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56554930e400 con 0x5655492c6e10 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56554930df50 con 0x5655492c6e10 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.638+0000 7ff6c8040800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.639+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x565549314880 con 0x5655492c6e10 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~556] v30'6 uv6 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c8040800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c8040800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c8040800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94924300174128 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x565549310250 con 0x5655492c6e10 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify (1) cookie 94924299918368 notify 128849018885 ret 0) ==== 774+0+0 (crc 0 0 0) 0x5655490427f0 con 0x5655492c6e10 2024-04-23T19:10:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff5a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018885 cookie 94924299918368 notifier 4359 bl.length()=732 2024-04-23T19:10:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff5a67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff5a67fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(65 notify.4 [notify cookie 94924300174128 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff5a67fc640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7ff588003ea0 con 0x5655492c6e10 2024-04-23T19:10:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== watch-notify(notify_complete (2) cookie 94924300174128 notify 128849018885 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff6b802e720 con 0x5655492c6e10 2024-04-23T19:10:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.640+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x565549314330 con 0x5655492c6e10 2024-04-23T19:10:27.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v30'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c8040800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-23T19:10:27.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c8040800 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c8040800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94924300174128 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x565549310200 con 0x7ff6bc041020 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== watch-notify(notify (1) cookie 94924299900560 notify 128849018882 ret 0) ==== 208+0+0 (crc 0 0 0) 0x56554928e580 con 0x7ff6bc041020 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff5a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018882 cookie 94924299900560 notifier 4359 bl.length()=166 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff5a67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff5a67fc640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(68 notify.0 [notify cookie 94924300174128 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff5a67fc640 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7ff588003ea0 con 0x7ff6bc041020 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.642+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== watch-notify(notify_complete (2) cookie 94924300174128 notify 128849018882 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56554928b8d0 con 0x7ff6bc041020 2024-04-23T19:10:27.658 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"JDppwXM1XHcAqowOWWn04zQ23C4g0TbQOpVKviX0","active":true,"create_date":"2024-04-23T19:10:27.639364Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.643+0000 7ff5797e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.643+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94924299926992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x565549313800 con 0x5655492c6e10 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.644+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94924299926992] v30'71 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.644+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94924299900560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d7dd0 con 0x7ff6bc041020 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.645+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94924299900560] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.645+0000 7ff6c8040800 20 remove_watcher() i=0 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.645+0000 7ff6c8040800 2 removed watcher, disabling cache 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.645+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94924299904800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d7dd0 con 0x5655492c6e10 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.646+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94924299904800] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.646+0000 7ff6c8040800 20 remove_watcher() i=1 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.646+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94924299909664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d2720 con 0x5655492cd060 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.647+0000 7ff6c34dd640 1 -- 172.21.15.63:0/3893492164 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94924299909664] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6bc00a4a0 con 0x5655492cd060 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.647+0000 7ff6c8040800 20 remove_watcher() i=2 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.647+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94924299913376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d3a20 con 0x7ff6bc041020 2024-04-23T19:10:27.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.648+0000 7ff6c2cdc640 1 -- 172.21.15.63:0/3893492164 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94924299913376] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b4078070 con 0x7ff6bc041020 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.648+0000 7ff6c8040800 20 remove_watcher() i=3 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.648+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94924299918368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d2720 con 0x5655492c6e10 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.649+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94924299918368] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x5655492c6e10 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.649+0000 7ff6c8040800 20 remove_watcher() i=4 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.649+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94924299920000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d48a0 con 0x7ff6b409b040 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.650+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94924299920000] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.650+0000 7ff6c8040800 20 remove_watcher() i=5 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.650+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94924299921056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d3a20 con 0x7ff6b409b040 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.651+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94924299921056] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.651+0000 7ff6c8040800 20 remove_watcher() i=6 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.651+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94924299924864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5655492d2720 con 0x7ff6b409b040 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94924299924864] v30'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff6b802a040 con 0x7ff6b409b040 2024-04-23T19:10:27.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 20 remove_watcher() i=7 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff6b409b040 msgr2=0x7ff6b405e900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff6b409b040 0x7ff6b405e900 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5655492c6e10 msgr2=0x5655492c92d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5655492c6e10 0x5655492c92d0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff6bc041020 msgr2=0x7ff6bc013a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff6bc041020 0x7ff6bc013a60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5655492cd060 msgr2=0x5655492cf450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5655492cd060 0x5655492cf450 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff58c036e60 msgr2=0x7ff58c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff58c036e60 0x7ff58c039320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x565549298810 tx=0x7ff6b4083000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 msgr2=0x5655492bb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492bb080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc04cd30 tx=0x7ff6bc04ccc0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c24db640 1 -- 172.21.15.63:0/3893492164 reap_dead start 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 shutdown_connections 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 --2- 172.21.15.63:0/3893492164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492b3bc0 0x5655492bb080 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.652+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 >> 172.21.15.63:0/3893492164 conn(0x5655492afe90 msgr2=0x5655492ad4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 shutdown_connections 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/3893492164 wait complete. 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff59c036e60 msgr2=0x7ff59c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff59c036e60 0x7ff59c039320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff6b408e290 tx=0x7ff6b407d040 comp rx=0 tx=0).stop 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492a6810 msgr2=0x5655492a6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492a6810 0x5655492a6c30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x565549292c80 tx=0x7ff6bc0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 shutdown_connections 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff59c036e60 0x7ff59c039320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/1141862396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655492a6810 0x5655492a6c30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 >> 172.21.15.63:0/1141862396 conn(0x56554929a8b0 msgr2=0x56554929acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 shutdown_connections 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/1141862396 wait complete. 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x565549295ac0 msgr2=0x565549297eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x565549295ac0 0x565549297eb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56554918f3a0 msgr2=0x565549292340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56554918f3a0 0x565549292340 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff5a8036e60 msgr2=0x7ff5a8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff5a8036e60 0x7ff5a8039320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8000f60 tx=0x7ff6b80012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 msgr2=0x56554928b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554928b100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4060040 tx=0x7ff6b4002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 shutdown_connections 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x565549295ac0 0x565549297eb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56554918f3a0 0x565549292340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff5a8036e60 0x7ff5a8039320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 --2- 172.21.15.63:0/388876716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56554918ea40 0x56554928b100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 >> 172.21.15.63:0/388876716 conn(0x56554918a600 msgr2=0x56554927d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.653+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 shutdown_connections 2024-04-23T19:10:27.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.654+0000 7ff6c8040800 1 -- 172.21.15.63:0/388876716 wait complete. 2024-04-23T19:10:27.669 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'JDppwXM1XHcAqowOWWn04zQ23C4g0TbQOpVKviX0', 'active': True, 'create_date': '2024-04-23T19:10:27.639364Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.669 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-23T19:10:27.669 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-23T19:10:27.670 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-04-23T19:10:27.718 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:27.718 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:27.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b7f75800 1 Processor -- start 2024-04-23T19:10:27.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b7f75800 1 -- start start 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b7f75800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561d7fe365a0 0x561d7fe3f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b7f75800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561d7fce87f0 con 0x561d7fe34a60 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b7f75800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x561d7fcd7e20 con 0x561d7fe365a0 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b5273640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x561d7fe365a0 0x561d7fe3f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b5a74640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x561d7fe34a60 0x561d7fe34e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38590/0 (socket says 172.21.15.63:38590) 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b5a74640 1 -- 172.21.15.63:0/3657897466 learned_addr learned my addr 172.21.15.63:0/3657897466 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b5273640 1 -- 172.21.15.63:0/3657897466 >> v1:172.21.15.16:6789/0 conn(0x561d7fe34a60 legacy=0x561d7fe34e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.719+0000 7f55b5273640 1 -- 172.21.15.63:0/3657897466 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7fcf6b50 con 0x561d7fe365a0 2024-04-23T19:10:27.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.721+0000 7f55b5273640 1 --2- 172.21.15.63:0/3657897466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe365a0 0x561d7fe3f0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a470 tx=0x7f55ac01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264f0492822ef471 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.721+0000 7f55b4a72640 1 -- 172.21.15.63:0/3657897466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55ac057070 con 0x561d7fe365a0 2024-04-23T19:10:27.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b4a72640 1 -- 172.21.15.63:0/3657897466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac052440 con 0x561d7fe365a0 2024-04-23T19:10:27.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b4a72640 1 -- 172.21.15.63:0/3657897466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55ac052740 con 0x561d7fe365a0 2024-04-23T19:10:27.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b7f75800 1 -- 172.21.15.63:0/3657897466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe365a0 msgr2=0x561d7fe3f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b7f75800 1 --2- 172.21.15.63:0/3657897466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe365a0 0x561d7fe3f0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a470 tx=0x7f55ac01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:27.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b7f75800 1 -- 172.21.15.63:0/3657897466 shutdown_connections 2024-04-23T19:10:27.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b7f75800 1 --2- 172.21.15.63:0/3657897466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe365a0 0x561d7fe3f0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b7f75800 1 -- 172.21.15.63:0/3657897466 >> 172.21.15.63:0/3657897466 conn(0x561d7fe30620 msgr2=0x561d7fe32a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b7f75800 1 -- 172.21.15.63:0/3657897466 shutdown_connections 2024-04-23T19:10:27.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.722+0000 7f55b7f75800 1 -- 172.21.15.63:0/3657897466 wait complete. 2024-04-23T19:10:27.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b7f75800 1 Processor -- start 2024-04-23T19:10:27.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b7f75800 1 -- start start 2024-04-23T19:10:27.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b7f75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b7f75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d7fd81710 con 0x561d7fe34a60 2024-04-23T19:10:27.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b5a74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b5a74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff29820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53486/0 (socket says 172.21.15.63:53486) 2024-04-23T19:10:27.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b5a74640 1 -- 172.21.15.63:0/1798205268 learned_addr learned my addr 172.21.15.63:0/1798205268 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.735+0000 7f55b5a74640 1 -- 172.21.15.63:0/1798205268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7ff29df0 con 0x561d7fe34a60 2024-04-23T19:10:27.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.736+0000 7f55b5a74640 1 --2- 172.21.15.63:0/1798205268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff29820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f55a8001de0 tx=0x7f55a800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a16696f8270f254 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.736+0000 7f54a3e07640 1 -- 172.21.15.63:0/1798205268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a8012020 con 0x561d7fe34a60 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.736+0000 7f54a3e07640 1 -- 172.21.15.63:0/1798205268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a800e5a0 con 0x561d7fe34a60 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.736+0000 7f54a3e07640 1 -- 172.21.15.63:0/1798205268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a800e8c0 con 0x561d7fe34a60 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.736+0000 7f55b7f75800 1 -- 172.21.15.63:0/1798205268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 msgr2=0x561d7ff29820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 --2- 172.21.15.63:0/1798205268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff29820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f55a8001de0 tx=0x7f55a800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 -- 172.21.15.63:0/1798205268 shutdown_connections 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 --2- 172.21.15.63:0/1798205268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff29820 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 -- 172.21.15.63:0/1798205268 >> 172.21.15.63:0/1798205268 conn(0x561d7fe30620 msgr2=0x561d7ff25140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 -- 172.21.15.63:0/1798205268 shutdown_connections 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 -- 172.21.15.63:0/1798205268 wait complete. 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 Processor -- start 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 -- start start 2024-04-23T19:10:27.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff31390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b7f75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d7fe418c0 con 0x561d7fe34a60 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b5a74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff31390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b5a74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff31390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53500/0 (socket says 172.21.15.63:53500) 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.737+0000 7f55b5a74640 1 -- 172.21.15.63:0/2409303699 learned_addr learned my addr 172.21.15.63:0/2409303699 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.738+0000 7f55b5a74640 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7ff318d0 con 0x561d7fe34a60 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.738+0000 7f55b5a74640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff31390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f55a8001de0 tx=0x7f55a8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.738+0000 7f54a2604640 1 -- 172.21.15.63:0/2409303699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a8012030 con 0x561d7fe34a60 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.738+0000 7f54a2604640 1 -- 172.21.15.63:0/2409303699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a801a7a0 con 0x561d7fe34a60 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.738+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d7ff31b60 con 0x561d7fe34a60 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.738+0000 7f54a2604640 1 -- 172.21.15.63:0/2409303699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a801aac0 con 0x561d7fe34a60 2024-04-23T19:10:27.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.738+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7ff34810 con 0x561d7fe34a60 2024-04-23T19:10:27.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.739+0000 7f54a2604640 1 -- 172.21.15.63:0/2409303699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f55a8015020 con 0x561d7fe34a60 2024-04-23T19:10:27.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.739+0000 7f54a2604640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f549c036e60 0x7f549c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.739+0000 7f54a2604640 1 -- 172.21.15.63:0/2409303699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f55a8020070 con 0x561d7fe34a60 2024-04-23T19:10:27.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.739+0000 7f55b5273640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f549c036e60 0x7f549c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.739+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7fe365a0 0x561d7ff385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.739+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff38b50 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.739+0000 7f55b6275640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7fe365a0 0x561d7ff385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.740+0000 7f55b5273640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f549c036e60 0x7f549c039320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55ac054000 tx=0x7f55ac04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.740+0000 7f55b6275640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7fe365a0 0x561d7ff385d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.740+0000 7f55b6275640 1 -- 172.21.15.63:0/2409303699 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f55b000a4a0 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.740+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff35ba0 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.740+0000 7f55b6275640 1 -- 172.21.15.63:0/2409303699 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f55b000a4a0 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.740+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff3a9a0 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.741+0000 7f55b6275640 1 -- 172.21.15.63:0/2409303699 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f55b000a4a0 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.741+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff39ce0 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.741+0000 7f55b6275640 1 -- 172.21.15.63:0/2409303699 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f55b000a4a0 con 0x561d7fe365a0 2024-04-23T19:10:27.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.741+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561d7ff3bd50 0x561d7ff3e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.741+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff3e6c0 con 0x561d7ff3bd50 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.741+0000 7f55b5a74640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561d7ff3bd50 0x561d7ff3e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.742+0000 7f55b5a74640 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561d7ff3bd50 0x561d7ff3e140 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.742+0000 7f55b5a74640 1 -- 172.21.15.63:0/2409303699 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f55a8024980 con 0x561d7ff3bd50 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b7f75800 1 Processor -- start 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b7f75800 1 -- start start 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b7f75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff4cae0 0x561d7ff4cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b7f75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d7fd27e30 con 0x561d7ff4cae0 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b6275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff4cae0 0x561d7ff4cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b6275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff4cae0 0x561d7ff4cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53510/0 (socket says 172.21.15.63:53510) 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b6275640 1 -- 172.21.15.63:0/509747575 learned_addr learned my addr 172.21.15.63:0/509747575 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.743+0000 7f55b6275640 1 -- 172.21.15.63:0/509747575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7ff4d440 con 0x561d7ff4cae0 2024-04-23T19:10:27.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.744+0000 7f55b6275640 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff4cae0 0x561d7ff4cf00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x561d7ff38f10 tx=0x7f55b00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.744+0000 7f549b7fe640 1 -- 172.21.15.63:0/509747575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b0013440 con 0x561d7ff4cae0 2024-04-23T19:10:27.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.744+0000 7f549b7fe640 1 -- 172.21.15.63:0/509747575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b00135e0 con 0x561d7ff4cae0 2024-04-23T19:10:27.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.744+0000 7f549b7fe640 1 -- 172.21.15.63:0/509747575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b00138c0 con 0x561d7ff4cae0 2024-04-23T19:10:27.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.744+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d7ff4deb0 con 0x561d7ff4cae0 2024-04-23T19:10:27.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.744+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7ff4e3b0 con 0x561d7ff4cae0 2024-04-23T19:10:27.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f549b7fe640 1 -- 172.21.15.63:0/509747575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f55b0011040 con 0x561d7ff4cae0 2024-04-23T19:10:27.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f549b7fe640 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5490036e60 0x7f5490039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f549b7fe640 1 -- 172.21.15.63:0/509747575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f55b001d070 con 0x561d7ff4cae0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f55b5a74640 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5490036e60 0x7f5490039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f55b7f75800 1 Processor -- start 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f55b7f75800 1 -- start start 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f55b5a74640 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5490036e60 0x7f5490039320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55a8024c90 tx=0x7f55a8017f30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f55b7f75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff5a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.745+0000 7f55b7f75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d7ff2a170 con 0x561d7ff59de0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f55b6275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff5a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f55b6275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff5a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53514/0 (socket says 172.21.15.63:53514) 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f55b6275640 1 -- 172.21.15.63:0/2388817726 learned_addr learned my addr 172.21.15.63:0/2388817726 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f55b6275640 1 -- 172.21.15.63:0/2388817726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7ff5a7d0 con 0x561d7ff59de0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f55b6275640 1 --2- 172.21.15.63:0/2388817726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff5a200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f55b0001470 tx=0x7f55b0012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae46bd129335decd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f54997fa640 1 -- 172.21.15.63:0/2388817726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b0015430 con 0x561d7ff59de0 2024-04-23T19:10:27.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f54997fa640 1 -- 172.21.15.63:0/2388817726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b002ac50 con 0x561d7ff59de0 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f54997fa640 1 -- 172.21.15.63:0/2388817726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b00155d0 con 0x561d7ff59de0 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.746+0000 7f55b7f75800 1 -- 172.21.15.63:0/2388817726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 msgr2=0x561d7ff5a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2388817726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff5a200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f55b0001470 tx=0x7f55b0012a30 comp rx=0 tx=0).stop 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 -- 172.21.15.63:0/2388817726 shutdown_connections 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2388817726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff5a200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 -- 172.21.15.63:0/2388817726 >> 172.21.15.63:0/2388817726 conn(0x561d7ff56140 msgr2=0x561d7ff541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 -- 172.21.15.63:0/2388817726 shutdown_connections 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 -- 172.21.15.63:0/2388817726 wait complete. 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 Processor -- start 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 -- start start 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b7f75800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d7ff5af40 con 0x561d7ff59de0 2024-04-23T19:10:27.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b6275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b6275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53520/0 (socket says 172.21.15.63:53520) 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.747+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 learned_addr learned my addr 172.21.15.63:0/4168184366 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.748+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7ff618f0 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.748+0000 7f55b6275640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff613b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f55b0001da0 tx=0x7f55b004cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.748+0000 7f54877fe640 1 -- 172.21.15.63:0/4168184366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b0013d00 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.748+0000 7f54877fe640 1 -- 172.21.15.63:0/4168184366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0030070 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.748+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d7ff61b80 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.748+0000 7f54877fe640 1 -- 172.21.15.63:0/4168184366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b00123a0 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.748+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7ff64830 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f54877fe640 1 -- 172.21.15.63:0/4168184366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f55b0012540 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f54877fe640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5480036e60 0x7f5480039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f54877fe640 1 -- 172.21.15.63:0/4168184366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f55b000e040 con 0x561d7ff59de0 2024-04-23T19:10:27.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f55b5a74640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5480036e60 0x7f5480039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f55b5a74640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5480036e60 0x7f5480039320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55a804b370 tx=0x7f55a8037000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7ff6d0a0 0x561d7ff6f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.749+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff6fcb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b5273640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7ff6d0a0 0x561d7ff6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b5273640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7ff6d0a0 0x561d7ff6f560 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 20 realm 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff70080 con 0x561d7ff6d0a0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.750+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff70950 con 0x561d7ff6d0a0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff70cf0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff71540 con 0x561d7ff6d0a0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:27.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 20 searching for the correct realm 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561d7ff732b0 0x561d7ff756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.751+0000 7f55b6275640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561d7ff732b0 0x561d7ff756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.752+0000 7f55b6275640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561d7ff732b0 0x561d7ff756a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.752+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.752+0000 7f55b6275640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f55b004f9c0 0x7f55b002b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.752+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55b004f9c0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.752+0000 7f55b5a74640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f55b004f9c0 0x7f55b002b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.752+0000 7f55b5a74640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f55b004f9c0 0x7f55b002b5c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.753+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.753+0000 7f55b5a74640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f55a8054040 0x7f55a804b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.753+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.753+0000 7f55b5273640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f55a8054040 0x7f55a804b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.753+0000 7f55b5273640 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f55a8054040 0x7f55a804b550 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.754+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55b004f9c0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.755+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55b004f9c0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.756+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.757+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.758+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55b004f9c0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.759+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55b004f9c0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff732b0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55a8054040 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.760+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff75cb0 con 0x7f55b004f9c0 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff71ec0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff72760 con 0x561d7ff6d0a0 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.761+0000 7f55b7f75800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff72760 con 0x7f55a8054040 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 20 zone default found 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 4 Realm: () 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 20 zonegroup default 2024-04-23T19:10:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff76130 con 0x7f55a8054040 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.762+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff77b70 con 0x561d7ff732b0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff78060 con 0x561d7ff732b0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 20 started sync module instance, tier type = 2024-04-23T19:10:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.763+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff76fb0 con 0x7f55b004f9c0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.764+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.764+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94684700963744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff76c10 con 0x7f55b004f9c0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.764+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff78f90 con 0x561d7ff6d0a0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.765+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94684700963744] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.765+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.765+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94684700969696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff793b0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.765+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff79d70 con 0x561d7ff732b0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.766+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94684700969696] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.767+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.767+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94684700973728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff793b0 con 0x561d7ff732b0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.767+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7a7c0 con 0x7f55b004f9c0 2024-04-23T19:10:27.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.768+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94684700973728] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.768+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.768+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94684700970752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff79320 con 0x7f55b004f9c0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.768+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7b6b0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.769+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.769+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94684700980192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7bdf0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.769+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7c910 con 0x7f55a8054040 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.769+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94684700970752] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.770+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.770+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94684700984288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7c8d0 con 0x7f55a8054040 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.770+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7d320 con 0x7f55a8054040 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.770+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94684700980192] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.771+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94684700984288] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.771+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.771+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94684700978144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7ce70 con 0x7f55a8054040 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.771+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7d830 con 0x7f55a8054040 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94684700978144] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94684700981312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7d830 con 0x7f55a8054040 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 20 add_watcher() i=0 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 20 add_watcher() i=1 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 20 add_watcher() i=2 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 20 add_watcher() i=3 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 20 add_watcher() i=4 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 20 add_watcher() i=5 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.772+0000 7f55b7f75800 20 add_watcher() i=6 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.773+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94684700981312] v30'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.773+0000 7f55b7f75800 20 add_watcher() i=7 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.773+0000 7f55b7f75800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.773+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff80b50 con 0x561d7ff6d0a0 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.773+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.773+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94684700988656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7ddc0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.774+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94684700988656] v30'72 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.774+0000 7f545f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 5 note: GC not initialized 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f545e7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 20 Filter name: none 2024-04-23T19:10:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f53d7fff640 20 reqs_thread_entry: start 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.775+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ffb4060 con 0x561d7ff6d0a0 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.776+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv6 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.776+0000 7f55b7f75800 20 rados_obj.operate() r=0 bl.length=556 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.776+0000 7f55b7f75800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.776+0000 7f55b7f75800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.776+0000 7f55b7f75800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:27.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.776+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ffba540 con 0x561d7ff6d0a0 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v30'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f55b7f75800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f55b7f75800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f55b7f75800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94684701237168 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x561d7ffb9090 con 0x561d7ff6d0a0 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94684700980192 notify 128849018886 ret 0) ==== 774+0+0 (crc 0 0 0) 0x561d7fcf6b50 con 0x561d7ff6d0a0 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f54997fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018886 cookie 94684700980192 notifier 4366 bl.length()=732 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f54997fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f54997fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(64 notify.4 [notify cookie 94684701237168 out=8b] v0'0 uv57 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.777+0000 7f54997fa640 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f5478003ea0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.778+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.778+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94684701237168 notify 128849018886 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f55ac05dac0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.789 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.779+0000 7f545f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.779+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94684700988656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ffb8be0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.779+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94684700988656] v30'73 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.780+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94684700963744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ffb9090 con 0x7f55b004f9c0 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.781+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94684700963744] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.781+0000 7f55b7f75800 20 remove_watcher() i=0 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.781+0000 7f55b7f75800 2 removed watcher, disabling cache 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.781+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94684700969696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff773a0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.782+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94684700969696] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.782+0000 7f55b7f75800 20 remove_watcher() i=1 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.782+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94684700973728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff78ae0 con 0x561d7ff732b0 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.783+0000 7f55b6275640 1 -- 172.21.15.63:0/4168184366 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94684700973728] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55b0029070 con 0x561d7ff732b0 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.783+0000 7f55b7f75800 20 remove_watcher() i=2 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.783+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94684700970752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff773a0 con 0x7f55b004f9c0 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.784+0000 7f55b5a74640 1 -- 172.21.15.63:0/4168184366 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94684700970752] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55a8042040 con 0x7f55b004f9c0 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.784+0000 7f55b7f75800 20 remove_watcher() i=3 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.784+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94684700980192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff773a0 con 0x561d7ff6d0a0 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.785+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94684700980192] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x561d7ff6d0a0 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.785+0000 7f55b7f75800 20 remove_watcher() i=4 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.785+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94684700984288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7b3e0 con 0x7f55a8054040 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.785+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94684700984288] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.785+0000 7f55b7f75800 20 remove_watcher() i=5 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.785+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94684700978144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff773a0 con 0x7f55a8054040 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.786+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94684700978144] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.786+0000 7f55b7f75800 20 remove_watcher() i=6 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.786+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94684700981312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x561d7ff7abe0 con 0x7f55a8054040 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94684700981312] v30'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f55ac05a040 con 0x7f55a8054040 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 20 remove_watcher() i=7 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f55a8054040 msgr2=0x7f55a804b550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f55a8054040 0x7f55a804b550 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7ff6d0a0 msgr2=0x561d7ff6f560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7ff6d0a0 0x561d7ff6f560 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f55b004f9c0 msgr2=0x7f55b002b5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f55b004f9c0 0x7f55b002b5c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561d7ff732b0 msgr2=0x561d7ff756a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x561d7ff732b0 0x561d7ff756a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5480036e60 msgr2=0x7f5480039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5480036e60 0x7f5480039320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55a804b370 tx=0x7f55a8037000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 msgr2=0x561d7ff613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b5273640 1 -- 172.21.15.63:0/4168184366 reap_dead start 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff613b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f55b0001da0 tx=0x7f55b004cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 shutdown_connections 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.787+0000 7f55b7f75800 1 --2- 172.21.15.63:0/4168184366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff59de0 0x561d7ff613b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 >> 172.21.15.63:0/4168184366 conn(0x561d7ff56140 msgr2=0x561d7ff537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 shutdown_connections 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/4168184366 wait complete. 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5490036e60 msgr2=0x7f5490039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5490036e60 0x7f5490039320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55a8024c90 tx=0x7f55a8017f30 comp rx=0 tx=0).stop 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff4cae0 msgr2=0x561d7ff4cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff4cae0 0x561d7ff4cf00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x561d7ff38f10 tx=0x7f55b00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 shutdown_connections 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5490036e60 0x7f5490039320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/509747575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7ff4cae0 0x561d7ff4cf00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 >> 172.21.15.63:0/509747575 conn(0x561d7ff40b80 msgr2=0x561d7ff40f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 shutdown_connections 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/509747575 wait complete. 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561d7ff3bd50 msgr2=0x561d7ff3e140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561d7ff3bd50 0x561d7ff3e140 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7fe365a0 msgr2=0x561d7ff385d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7fe365a0 0x561d7ff385d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f549c036e60 msgr2=0x7f549c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f549c036e60 0x7f549c039320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55ac054000 tx=0x7f55ac04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 msgr2=0x561d7ff31390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.788+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff31390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f55a8001de0 tx=0x7f55a8000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 shutdown_connections 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x561d7ff3bd50 0x561d7ff3e140 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x561d7fe365a0 0x561d7ff385d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f549c036e60 0x7f549c039320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 --2- 172.21.15.63:0/2409303699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d7fe34a60 0x561d7ff31390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 >> 172.21.15.63:0/2409303699 conn(0x561d7fe30620 msgr2=0x561d7ff23940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 shutdown_connections 2024-04-23T19:10:27.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.789+0000 7f55b7f75800 1 -- 172.21.15.63:0/2409303699 wait complete. 2024-04-23T19:10:27.800 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.801 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.801 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.801 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:27.844 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:27.845 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.845+0000 7f7f5de1f800 1 Processor -- start 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.845+0000 7f7f5de1f800 1 -- start start 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.845+0000 7f7f5de1f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55cd870a3a30 0x55cd870a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.845+0000 7f7f5de1f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55cd86f577f0 con 0x55cd870a4390 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.845+0000 7f7f5de1f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55cd86f46e20 con 0x55cd870a3a30 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.846+0000 7f7f5b91e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55cd870a3a30 0x55cd870a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.846+0000 7f7f5b11d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55cd870a4390 0x55cd870a4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38606/0 (socket says 172.21.15.63:38606) 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.846+0000 7f7f5b11d640 1 -- 172.21.15.63:0/3522353769 learned_addr learned my addr 172.21.15.63:0/3522353769 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:27.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.846+0000 7f7f5b91e640 1 -- 172.21.15.63:0/3522353769 >> v1:172.21.15.16:6789/0 conn(0x55cd870a4390 legacy=0x55cd870a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.846+0000 7f7f5b91e640 1 -- 172.21.15.63:0/3522353769 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd86f65b50 con 0x55cd870a3a30 2024-04-23T19:10:27.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/3522353769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd870a3e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400a470 tx=0x7f7f5401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201ed20c8e65c15c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5a91c640 1 -- 172.21.15.63:0/3522353769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f54057070 con 0x55cd870a3a30 2024-04-23T19:10:27.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5a91c640 1 -- 172.21.15.63:0/3522353769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5404adc0 con 0x55cd870a3a30 2024-04-23T19:10:27.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5a91c640 1 -- 172.21.15.63:0/3522353769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f54052590 con 0x55cd870a3a30 2024-04-23T19:10:27.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3522353769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 msgr2=0x55cd870a3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3522353769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd870a3e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400a470 tx=0x7f7f5401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:27.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3522353769 shutdown_connections 2024-04-23T19:10:27.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3522353769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd870a3e50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3522353769 >> 172.21.15.63:0/3522353769 conn(0x55cd8709f630 msgr2=0x55cd870a1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3522353769 shutdown_connections 2024-04-23T19:10:27.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.848+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3522353769 wait complete. 2024-04-23T19:10:27.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.861+0000 7f7f5de1f800 1 Processor -- start 2024-04-23T19:10:27.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.861+0000 7f7f5de1f800 1 -- start start 2024-04-23T19:10:27.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.861+0000 7f7f5de1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd87198600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.861+0000 7f7f5de1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd86ff06f0 con 0x55cd870a3a30 2024-04-23T19:10:27.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.861+0000 7f7f5b91e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd87198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.861+0000 7f7f5b91e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd87198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53548/0 (socket says 172.21.15.63:53548) 2024-04-23T19:10:27.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.861+0000 7f7f5b91e640 1 -- 172.21.15.63:0/3146690756 learned_addr learned my addr 172.21.15.63:0/3146690756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.862+0000 7f7f5b91e640 1 -- 172.21.15.63:0/3146690756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd87198bd0 con 0x55cd870a3a30 2024-04-23T19:10:27.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.862+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/3146690756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd87198600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f5401ced0 tx=0x7f7f5404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d891a64125186d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.862+0000 7f7e49e0b640 1 -- 172.21.15.63:0/3146690756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f54057040 con 0x55cd870a3a30 2024-04-23T19:10:27.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.862+0000 7f7e49e0b640 1 -- 172.21.15.63:0/3146690756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f54063070 con 0x55cd870a3a30 2024-04-23T19:10:27.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.862+0000 7f7e49e0b640 1 -- 172.21.15.63:0/3146690756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f54052800 con 0x55cd870a3a30 2024-04-23T19:10:27.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3146690756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 msgr2=0x55cd87198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3146690756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd87198600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f5401ced0 tx=0x7f7f5404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3146690756 shutdown_connections 2024-04-23T19:10:27.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3146690756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd87198600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3146690756 >> 172.21.15.63:0/3146690756 conn(0x55cd8709f630 msgr2=0x55cd87193f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3146690756 shutdown_connections 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3146690756 wait complete. 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 Processor -- start 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 -- start start 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd871a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5de1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd870a51f0 con 0x55cd870a3a30 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5b91e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd871a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5b91e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd871a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53552/0 (socket says 172.21.15.63:53552) 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.863+0000 7f7f5b91e640 1 -- 172.21.15.63:0/3153040332 learned_addr learned my addr 172.21.15.63:0/3153040332 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.864+0000 7f7f5b91e640 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd871a0550 con 0x55cd870a3a30 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.864+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd871a0010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f5401ced0 tx=0x7f7f54002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.864+0000 7f7e43fff640 1 -- 172.21.15.63:0/3153040332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5406d070 con 0x55cd870a3a30 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.864+0000 7f7e43fff640 1 -- 172.21.15.63:0/3153040332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f54052ca0 con 0x55cd870a3a30 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.864+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cd871a07e0 con 0x55cd870a3a30 2024-04-23T19:10:27.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.864+0000 7f7e43fff640 1 -- 172.21.15.63:0/3153040332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f54057050 con 0x55cd870a3a30 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.864+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd871a3490 con 0x55cd870a3a30 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.865+0000 7f7e43fff640 1 -- 172.21.15.63:0/3153040332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7f5405d470 con 0x55cd870a3a30 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.865+0000 7f7e43fff640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e44036e60 0x7f7e44039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.865+0000 7f7e43fff640 1 -- 172.21.15.63:0/3153040332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7f5408de40 con 0x55cd870a3a30 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.865+0000 7f7f5b11d640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e44036e60 0x7f7e44039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.865+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd870a4390 0x55cd871a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.865+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871a7820 con 0x55cd870a4390 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.865+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd870a4390 0x55cd871a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.866+0000 7f7f5b11d640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e44036e60 0x7f7e44039320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f44000f60 tx=0x7f7f440012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.866+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd870a4390 0x55cd871a7250 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.866+0000 7f7f5c11f640 1 -- 172.21.15.63:0/3153040332 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f4c00a4a0 con 0x55cd870a4390 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.866+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871a4800 con 0x55cd870a4390 2024-04-23T19:10:27.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.866+0000 7f7f5c11f640 1 -- 172.21.15.63:0/3153040332 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f4c00a4a0 con 0x55cd870a4390 2024-04-23T19:10:27.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.866+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871a9670 con 0x55cd870a4390 2024-04-23T19:10:27.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.867+0000 7f7f5c11f640 1 -- 172.21.15.63:0/3153040332 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7f4c00a4a0 con 0x55cd870a4390 2024-04-23T19:10:27.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.867+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871a89b0 con 0x55cd870a4390 2024-04-23T19:10:27.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.867+0000 7f7f5c11f640 1 -- 172.21.15.63:0/3153040332 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f4c00a4a0 con 0x55cd870a4390 2024-04-23T19:10:27.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.867+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cd871aaa20 0x55cd871ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.867+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ad350 con 0x55cd871aaa20 2024-04-23T19:10:27.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.867+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cd871aaa20 0x55cd871ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.868+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cd871aaa20 0x55cd871ace10 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.868+0000 7f7f5b91e640 1 -- 172.21.15.63:0/3153040332 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7f5405a040 con 0x55cd871aaa20 2024-04-23T19:10:27.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.868+0000 7f7f5de1f800 1 Processor -- start 2024-04-23T19:10:27.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.868+0000 7f7f5de1f800 1 -- start start 2024-04-23T19:10:27.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.868+0000 7f7f5de1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871bb7c0 0x55cd871bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.868+0000 7f7f5de1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd86f96e30 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7f5c11f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871bb7c0 0x55cd871bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7f5c11f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871bb7c0 0x55cd871bbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53558/0 (socket says 172.21.15.63:53558) 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7f5c11f640 1 -- 172.21.15.63:0/4149739951 learned_addr learned my addr 172.21.15.63:0/4149739951 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7f5c11f640 1 -- 172.21.15.63:0/4149739951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd871bc120 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871bb7c0 0x55cd871bbbe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55cd871a7c00 tx=0x7f7f4c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7e417fa640 1 -- 172.21.15.63:0/4149739951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c013440 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7e417fa640 1 -- 172.21.15.63:0/4149739951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4c0135e0 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cd871bcb90 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7e417fa640 1 -- 172.21.15.63:0/4149739951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c0138c0 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.869+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd871bd090 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.870+0000 7f7e417fa640 1 -- 172.21.15.63:0/4149739951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7f4c011040 con 0x55cd871bb7c0 2024-04-23T19:10:27.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.870+0000 7f7e417fa640 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e34036eb0 0x7f7e34039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.870+0000 7f7e417fa640 1 -- 172.21.15.63:0/4149739951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7f4c01d070 con 0x55cd871bb7c0 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.870+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e34036eb0 0x7f7e34039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5de1f800 1 Processor -- start 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5de1f800 1 -- start start 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e34036eb0 0x7f7e34039370 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55cd871a8d50 tx=0x7f7f5408c1e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5de1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5de1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd87198e90 con 0x55cd871c8ac0 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5c11f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5c11f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53566/0 (socket says 172.21.15.63:53566) 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5c11f640 1 -- 172.21.15.63:0/1457143049 learned_addr learned my addr 172.21.15.63:0/1457143049 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5c11f640 1 -- 172.21.15.63:0/1457143049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd871c94b0 con 0x55cd871c8ac0 2024-04-23T19:10:27.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/1457143049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871c8ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0129a0 tx=0x7f7f4c012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b8cae6b5424a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7e3affd640 1 -- 172.21.15.63:0/1457143049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c02ad80 con 0x55cd871c8ac0 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7e3affd640 1 -- 172.21.15.63:0/1457143049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4c0014f0 con 0x55cd871c8ac0 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.871+0000 7f7e3affd640 1 -- 172.21.15.63:0/1457143049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c0017d0 con 0x55cd871c8ac0 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 -- 172.21.15.63:0/1457143049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 msgr2=0x55cd871c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/1457143049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871c8ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0129a0 tx=0x7f7f4c012a90 comp rx=0 tx=0).stop 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 -- 172.21.15.63:0/1457143049 shutdown_connections 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/1457143049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871c8ee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 -- 172.21.15.63:0/1457143049 >> 172.21.15.63:0/1457143049 conn(0x55cd871c4e20 msgr2=0x55cd871c2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 -- 172.21.15.63:0/1457143049 shutdown_connections 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 -- 172.21.15.63:0/1457143049 wait complete. 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 Processor -- start 2024-04-23T19:10:27.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 -- start start 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.872+0000 7f7f5de1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd871c9c20 con 0x55cd871c8ac0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7f5c11f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7f5c11f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53582/0 (socket says 172.21.15.63:53582) 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 learned_addr learned my addr 172.21.15.63:0/2723637993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd871d05d0 con 0x55cd871c8ac0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871d0090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c04cba0 tx=0x7f7f4c04cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7e397fa640 1 -- 172.21.15.63:0/2723637993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c0125b0 con 0x55cd871c8ac0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7e397fa640 1 -- 172.21.15.63:0/2723637993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4c030070 con 0x55cd871c8ac0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cd871d0860 con 0x55cd871c8ac0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7e397fa640 1 -- 172.21.15.63:0/2723637993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c04e110 con 0x55cd871c8ac0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.873+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd871d3510 con 0x55cd871c8ac0 2024-04-23T19:10:27.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.874+0000 7f7e397fa640 1 -- 172.21.15.63:0/2723637993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7f4c047050 con 0x55cd871c8ac0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.874+0000 7f7e397fa640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e24036e60 0x7f7e24039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.874+0000 7f7e397fa640 1 -- 172.21.15.63:0/2723637993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7f4c00e040 con 0x55cd871c8ac0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.874+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e24036e60 0x7f7e24039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e24036e60 0x7f7e24039320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f5405d940 tx=0x7f7f54083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd871dbda0 0x55cd871de260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871de9b0 con 0x55cd871dbda0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5b11d640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd871dbda0 0x55cd871de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5b11d640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd871dbda0 0x55cd871de260 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f44009050 con 0x55cd871dbda0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5de1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5de1f800 20 realm 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871db1f0 con 0x55cd871dbda0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.875+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f44009050 con 0x55cd871dbda0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871df490 con 0x55cd871dbda0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f44009050 con 0x55cd871dbda0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871dfd60 con 0x55cd871dbda0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f44009050 con 0x55cd871dbda0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e0100 con 0x55cd871dbda0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7f44009050 con 0x55cd871dbda0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 20 searching for the correct realm 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cd871e1fe0 0x55cd871e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x55cd871e1fe0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.876+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cd871e1fe0 0x55cd871e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.877+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cd871e1fe0 0x55cd871e43d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.877+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.877+0000 7f7f5c11f640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7f4c04e490 0x7f7f4c04e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.877+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x7f7f4c04e490 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.877+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7f4c04e490 0x7f7f4c04e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.878+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7f4c04e490 0x7f7f4c04e9c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.878+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.878+0000 7f7f5b91e640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7f54093030 0x7f7f5408d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.878+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x7f7f54093030 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.878+0000 7f7f5b11d640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7f54093030 0x7f7f5408d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.878+0000 7f7f5b11d640 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7f54093030 0x7f7f5408d400 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.879+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f44009050 con 0x7f7f54093030 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.879+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x55cd871dbda0 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.879+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f44009050 con 0x55cd871dbda0 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.879+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x55cd871dbda0 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.879+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.879+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x7f7f54093030 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x7f7f54093030 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x7f7f4c04e490 2024-04-23T19:10:27.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x55cd871e1fe0 2024-04-23T19:10:27.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x55cd871dbda0 2024-04-23T19:10:27.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.880+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.881+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x7f7f54093030 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.881+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.881+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x55cd871dbda0 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.881+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.881+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4a80 con 0x7f7f4c04e490 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.881+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.881+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871e1fe0 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871e1fe0 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f54093030 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871e1fe0 2024-04-23T19:10:27.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.882+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871e1fe0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871dbda0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871dbda0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f54093030 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871e1fe0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.883+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f4c04e490 2024-04-23T19:10:27.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871dbda0 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f54093030 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871e1fe0 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.884+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f54093030 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f4c04e490 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871e1fe0 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f54093030 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.885+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x55cd871dbda0 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f7f540a2060 con 0x7f7f4c04e490 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e0ab0 con 0x55cd871dbda0 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e1350 con 0x55cd871dbda0 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.886+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e06f0 con 0x7f7f54093030 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 zone default found 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 4 Realm: () 2024-04-23T19:10:27.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 zonegroup default 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e4ee0 con 0x7f7f54093030 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e6d70 con 0x55cd871e1fe0 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.887+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e7110 con 0x55cd871e1fe0 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.888+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.888+0000 7f7f5de1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.888+0000 7f7f5de1f800 20 started sync module instance, tier type = 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.888+0000 7f7f5de1f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.888+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e61b0 con 0x7f7f4c04e490 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.889+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.889+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94341223572896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e69c0 con 0x7f7f4c04e490 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.889+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e7f80 con 0x55cd871dbda0 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.890+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94341223572896] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.890+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.890+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94341223578320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e83a0 con 0x55cd871dbda0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.890+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e8d60 con 0x55cd871e1fe0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.891+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94341223578320] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.891+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.891+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94341223582352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e83a0 con 0x55cd871e1fe0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.891+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e97d0 con 0x7f7f4c04e490 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.892+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94341223582352] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.892+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.893+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94341223579376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e8310 con 0x7f7f4c04e490 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.893+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ea690 con 0x55cd871dbda0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.894+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94341223579376] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.894+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.894+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94341223588320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871eaab0 con 0x55cd871dbda0 2024-04-23T19:10:27.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.894+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871eb470 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.895+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.895+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94341223591872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871eb890 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.895+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94341223588320] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.895+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ec330 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.896+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94341223591872] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.896+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.896+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94341223595520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871eb3e0 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.896+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ecc00 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94341223597904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871eb3e0 con 0x7f7f54093030 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 20 add_watcher() i=0 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 20 add_watcher() i=1 2024-04-23T19:10:27.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 20 add_watcher() i=2 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 20 add_watcher() i=3 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 20 add_watcher() i=4 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 20 add_watcher() i=5 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94341223595520] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.897+0000 7f7f5de1f800 20 add_watcher() i=6 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.898+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94341223597904] v30'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.898+0000 7f7f5de1f800 20 add_watcher() i=7 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.898+0000 7f7f5de1f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.898+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ea600 con 0x55cd871dbda0 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.898+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.898+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94341223598960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ea600 con 0x55cd871dbda0 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.899+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94341223598960] v30'74 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.899+0000 7f7e11fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.899+0000 7f7f5de1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.899+0000 7f7f5de1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.899+0000 7f7f5de1f800 5 note: GC not initialized 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7e10fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7f5de1f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7f5de1f800 20 Filter name: none 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7d8a7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7f5de1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cd87222c50 con 0x55cd871dbda0 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.900+0000 7f7f5de1f800 20 rados_obj.operate() r=0 bl.length=556 2024-04-23T19:10:27.912 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.901+0000 7f7e11fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.901+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94341223598960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd87221820 con 0x55cd871dbda0 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.902+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94341223598960] v30'75 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.902+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94341223572896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ecb70 con 0x7f7f4c04e490 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.903+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94341223572896] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.903+0000 7f7f5de1f800 20 remove_watcher() i=0 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.903+0000 7f7f5de1f800 2 removed watcher, disabling cache 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.903+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94341223578320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ecb70 con 0x55cd871dbda0 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.904+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94341223578320] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.904+0000 7f7f5de1f800 20 remove_watcher() i=1 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.904+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94341223582352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871e7ad0 con 0x55cd871e1fe0 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.905+0000 7f7f5c11f640 1 -- 172.21.15.63:0/2723637993 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94341223582352] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4c029070 con 0x55cd871e1fe0 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.905+0000 7f7f5de1f800 20 remove_watcher() i=2 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.905+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94341223579376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ecb70 con 0x7f7f4c04e490 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.906+0000 7f7f5b91e640 1 -- 172.21.15.63:0/2723637993 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94341223579376] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f5405a040 con 0x7f7f4c04e490 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.906+0000 7f7f5de1f800 20 remove_watcher() i=3 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.906+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94341223588320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ecb70 con 0x55cd871dbda0 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.907+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94341223588320] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x55cd871dbda0 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.907+0000 7f7f5de1f800 20 remove_watcher() i=4 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.907+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94341223591872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ea1e0 con 0x7f7f54093030 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.908+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94341223591872] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.908+0000 7f7f5de1f800 20 remove_watcher() i=5 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.908+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94341223595520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871ecb70 con 0x7f7f54093030 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.909+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94341223595520] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.909+0000 7f7f5de1f800 20 remove_watcher() i=6 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.909+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94341223597904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cd871eafc0 con 0x7f7f54093030 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94341223597904] v30'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f4402a040 con 0x7f7f54093030 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 20 remove_watcher() i=7 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7f54093030 msgr2=0x7f7f5408d400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7f54093030 0x7f7f5408d400 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd871dbda0 msgr2=0x55cd871de260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd871dbda0 0x55cd871de260 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7f4c04e490 msgr2=0x7f7f4c04e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7f4c04e490 0x7f7f4c04e9c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cd871e1fe0 msgr2=0x55cd871e43d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cd871e1fe0 0x55cd871e43d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e24036e60 msgr2=0x7f7e24039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e24036e60 0x7f7e24039320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f5405d940 tx=0x7f7f54083000 comp rx=0 tx=0).stop 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 msgr2=0x55cd871d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5b11d640 1 -- 172.21.15.63:0/2723637993 reap_dead start 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871d0090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c04cba0 tx=0x7f7f4c04cbd0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 shutdown_connections 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/2723637993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871c8ac0 0x55cd871d0090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 >> 172.21.15.63:0/2723637993 conn(0x55cd871c4e20 msgr2=0x55cd871c2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 shutdown_connections 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.910+0000 7f7f5de1f800 1 -- 172.21.15.63:0/2723637993 wait complete. 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e34036eb0 msgr2=0x7f7e34039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e34036eb0 0x7f7e34039370 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55cd871a8d50 tx=0x7f7f5408c1e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871bb7c0 msgr2=0x55cd871bbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871bb7c0 0x55cd871bbbe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55cd871a7c00 tx=0x7f7f4c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 shutdown_connections 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e34036eb0 0x7f7e34039370 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/4149739951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd871bb7c0 0x55cd871bbbe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 >> 172.21.15.63:0/4149739951 conn(0x55cd871af860 msgr2=0x55cd871afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 shutdown_connections 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/4149739951 wait complete. 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cd871aaa20 msgr2=0x55cd871ace10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cd871aaa20 0x55cd871ace10 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd870a4390 msgr2=0x55cd871a7250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd870a4390 0x55cd871a7250 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e44036e60 msgr2=0x7f7e44039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e44036e60 0x7f7e44039320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f44000f60 tx=0x7f7f440012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 msgr2=0x55cd871a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd871a0010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f5401ced0 tx=0x7f7f54002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 shutdown_connections 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cd871aaa20 0x55cd871ace10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cd870a4390 0x55cd871a7250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7e44036e60 0x7f7e44039320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 --2- 172.21.15.63:0/3153040332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd870a3a30 0x55cd871a0010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 >> 172.21.15.63:0/3153040332 conn(0x55cd8709f630 msgr2=0x55cd871927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 shutdown_connections 2024-04-23T19:10:27.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:27.911+0000 7f7f5de1f800 1 -- 172.21.15.63:0/3153040332 wait complete. 2024-04-23T19:10:27.923 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:27.923 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:27.924 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:28.008 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.008 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.008+0000 7fc9be12d800 1 Processor -- start 2024-04-23T19:10:28.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9be12d800 1 -- start start 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9be12d800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559420948010 0x559420948430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9be12d800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5594207fb7f0 con 0x559420949270 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9be12d800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5594207eae20 con 0x559420948010 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9b9dd1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559420948010 0x559420948430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9b95d0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x559420949270 0x5594209496f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38620/0 (socket says 172.21.15.63:38620) 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1688670396 learned_addr learned my addr 172.21.15.63:0/1688670396 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1688670396 >> v1:172.21.15.16:6789/0 conn(0x559420949270 legacy=0x5594209496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.009+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1688670396 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559420809b50 con 0x559420948010 2024-04-23T19:10:28.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.011+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/1688670396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420948430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00a470 tx=0x7fc9ac01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d480220e7e7aae27 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.011+0000 7fc9b8dcf640 1 -- 172.21.15.63:0/1688670396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9ac057070 con 0x559420948010 2024-04-23T19:10:28.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.011+0000 7fc9b8dcf640 1 -- 172.21.15.63:0/1688670396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ac052570 con 0x559420948010 2024-04-23T19:10:28.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.011+0000 7fc9b8dcf640 1 -- 172.21.15.63:0/1688670396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9ac052870 con 0x559420948010 2024-04-23T19:10:28.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.012+0000 7fc9be12d800 1 -- 172.21.15.63:0/1688670396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 msgr2=0x559420948430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.012+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1688670396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420948430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00a470 tx=0x7fc9ac01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:28.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.012+0000 7fc9be12d800 1 -- 172.21.15.63:0/1688670396 shutdown_connections 2024-04-23T19:10:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.012+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1688670396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420948430 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.012+0000 7fc9be12d800 1 -- 172.21.15.63:0/1688670396 >> 172.21.15.63:0/1688670396 conn(0x559420943630 msgr2=0x559420945a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.012+0000 7fc9be12d800 1 -- 172.21.15.63:0/1688670396 shutdown_connections 2024-04-23T19:10:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.012+0000 7fc9be12d800 1 -- 172.21.15.63:0/1688670396 wait complete. 2024-04-23T19:10:28.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.025+0000 7fc9be12d800 1 Processor -- start 2024-04-23T19:10:28.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.025+0000 7fc9be12d800 1 -- start start 2024-04-23T19:10:28.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.025+0000 7fc9be12d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a3ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.025+0000 7fc9be12d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594208946f0 con 0x559420948010 2024-04-23T19:10:28.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.025+0000 7fc9b9dd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a3ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.025+0000 7fc9b9dd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a3ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53600/0 (socket says 172.21.15.63:53600) 2024-04-23T19:10:28.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.025+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/728288597 learned_addr learned my addr 172.21.15.63:0/728288597 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.026+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/728288597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559420a3cff0 con 0x559420948010 2024-04-23T19:10:28.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.026+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/728288597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a3ca20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac055040 tx=0x7fc9ac04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1ba2e2985a0ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.026+0000 7fc8a7e07640 1 -- 172.21.15.63:0/728288597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9ac057040 con 0x559420948010 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.026+0000 7fc8a7e07640 1 -- 172.21.15.63:0/728288597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ac052990 con 0x559420948010 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.026+0000 7fc8a7e07640 1 -- 172.21.15.63:0/728288597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9ac052c70 con 0x559420948010 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 -- 172.21.15.63:0/728288597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 msgr2=0x559420a3ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 --2- 172.21.15.63:0/728288597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a3ca20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac055040 tx=0x7fc9ac04c000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 -- 172.21.15.63:0/728288597 shutdown_connections 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 --2- 172.21.15.63:0/728288597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a3ca20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 -- 172.21.15.63:0/728288597 >> 172.21.15.63:0/728288597 conn(0x559420943630 msgr2=0x559420a33f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 -- 172.21.15.63:0/728288597 shutdown_connections 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 -- 172.21.15.63:0/728288597 wait complete. 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 Processor -- start 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 -- start start 2024-04-23T19:10:28.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a44470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.027+0000 7fc9be12d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594208bb7e0 con 0x559420948010 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc9b9dd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc9b9dd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a44470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53614/0 (socket says 172.21.15.63:53614) 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/3169563611 learned_addr learned my addr 172.21.15.63:0/3169563611 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559420a449b0 con 0x559420948010 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a44470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00a470 tx=0x7fc9ac003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc8a6604640 1 -- 172.21.15.63:0/3169563611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9ac073070 con 0x559420948010 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc8a6604640 1 -- 172.21.15.63:0/3169563611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ac05d570 con 0x559420948010 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc8a6604640 1 -- 172.21.15.63:0/3169563611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9ac057070 con 0x559420948010 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559420a44c40 con 0x559420948010 2024-04-23T19:10:28.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.028+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559420a478f0 con 0x559420948010 2024-04-23T19:10:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.029+0000 7fc8a6604640 1 -- 172.21.15.63:0/3169563611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc9ac076070 con 0x559420948010 2024-04-23T19:10:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.029+0000 7fc8a6604640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc8a0036e60 0x7fc8a0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.029+0000 7fc8a6604640 1 -- 172.21.15.63:0/3169563611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fc9ac08e480 con 0x559420948010 2024-04-23T19:10:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.029+0000 7fc9b95d0640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc8a0036e60 0x7fc8a0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.029+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420949270 0x559420a4b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.029+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a4bc80 con 0x559420949270 2024-04-23T19:10:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.029+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420949270 0x559420a4b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.030+0000 7fc9b95d0640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc8a0036e60 0x7fc8a0039320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000f60 tx=0x7fc9b00012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.030+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420949270 0x559420a4b6b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.030+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/3169563611 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420949270 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.030+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a48c80 con 0x559420949270 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.030+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/3169563611 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420949270 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.030+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a4da80 con 0x559420949270 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.031+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/3169563611 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420949270 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.031+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a4cdc0 con 0x559420949270 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.031+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/3169563611 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420949270 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.031+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559420a4ee30 0x559420a51220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.031+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a517a0 con 0x559420a4ee30 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.031+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559420a4ee30 0x559420a51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.032+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559420a4ee30 0x559420a51220 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.032+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/3169563611 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc9ac075030 con 0x559420a4ee30 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.032+0000 7fc9be12d800 1 Processor -- start 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9be12d800 1 -- start start 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9be12d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a5fb80 0x559420a5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9be12d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55942083ae30 con 0x559420a5fb80 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9ba5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a5fb80 0x559420a5ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9ba5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a5fb80 0x559420a5ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53620/0 (socket says 172.21.15.63:53620) 2024-04-23T19:10:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/372004579 learned_addr learned my addr 172.21.15.63:0/372004579 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/372004579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559420a604e0 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a5fb80 0x559420a5ffa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x559420a4c040 tx=0x7fc9b40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.033+0000 7fc89f7fe640 1 -- 172.21.15.63:0/372004579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b4013440 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.034+0000 7fc89f7fe640 1 -- 172.21.15.63:0/372004579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b40135e0 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.034+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559420a60f50 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.034+0000 7fc89f7fe640 1 -- 172.21.15.63:0/372004579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b40138c0 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.034+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559420a61450 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.034+0000 7fc89f7fe640 1 -- 172.21.15.63:0/372004579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc9b4011040 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.034+0000 7fc89f7fe640 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc894036e60 0x7fc894039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc89f7fe640 1 -- 172.21.15.63:0/372004579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fc9b401d070 con 0x559420a5fb80 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc894036e60 0x7fc894039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9be12d800 1 Processor -- start 2024-04-23T19:10:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc894036e60 0x7fc894039320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac006e60 tx=0x7fc9ac006f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9be12d800 1 -- start start 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9be12d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9be12d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559420a3d2b0 con 0x559420a6cf30 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9ba5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9ba5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a6d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53636/0 (socket says 172.21.15.63:53636) 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.035+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/2137382068 learned_addr learned my addr 172.21.15.63:0/2137382068 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/2137382068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559420a6d920 con 0x559420a6cf30 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/2137382068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a6d350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc9b404cd50 tx=0x7fc9b404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9517408e026e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc89d7fa640 1 -- 172.21.15.63:0/2137382068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b4015560 con 0x559420a6cf30 2024-04-23T19:10:28.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc89d7fa640 1 -- 172.21.15.63:0/2137382068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b402a350 con 0x559420a6cf30 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc89d7fa640 1 -- 172.21.15.63:0/2137382068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b4012ca0 con 0x559420a6cf30 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9be12d800 1 -- 172.21.15.63:0/2137382068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 msgr2=0x559420a6d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9be12d800 1 --2- 172.21.15.63:0/2137382068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a6d350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc9b404cd50 tx=0x7fc9b404d000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9be12d800 1 -- 172.21.15.63:0/2137382068 shutdown_connections 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9be12d800 1 --2- 172.21.15.63:0/2137382068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a6d350 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9be12d800 1 -- 172.21.15.63:0/2137382068 >> 172.21.15.63:0/2137382068 conn(0x559420a69200 msgr2=0x559420a67260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9be12d800 1 -- 172.21.15.63:0/2137382068 shutdown_connections 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.036+0000 7fc9be12d800 1 -- 172.21.15.63:0/2137382068 wait complete. 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9be12d800 1 Processor -- start 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9be12d800 1 -- start start 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9be12d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9be12d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559420a6e0b0 con 0x559420a6cf30 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9ba5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9ba5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53650/0 (socket says 172.21.15.63:53650) 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 learned_addr learned my addr 172.21.15.63:0/1789643896 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.037+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559420a74920 con 0x559420a6cf30 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.038+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a743e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc9b404d040 tx=0x7fc9b404cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.038+0000 7fc88b7fe640 1 -- 172.21.15.63:0/1789643896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b402a8e0 con 0x559420a6cf30 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.038+0000 7fc88b7fe640 1 -- 172.21.15.63:0/1789643896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b4013af0 con 0x559420a6cf30 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.038+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559420a74bb0 con 0x559420a6cf30 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.038+0000 7fc88b7fe640 1 -- 172.21.15.63:0/1789643896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b4013dd0 con 0x559420a6cf30 2024-04-23T19:10:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.038+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559420a77860 con 0x559420a6cf30 2024-04-23T19:10:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.038+0000 7fc88b7fe640 1 -- 172.21.15.63:0/1789643896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc9b402b3a0 con 0x559420a6cf30 2024-04-23T19:10:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc88b7fe640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc884036e10 0x7fc8840392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc88b7fe640 1 -- 172.21.15.63:0/1789643896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fc9b4032070 con 0x559420a6cf30 2024-04-23T19:10:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc884036e10 0x7fc8840392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420a7fee0 0x559420a823a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a82af0 con 0x559420a7fee0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc884036e10 0x7fc8840392d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x559420a51b80 tx=0x7fc9ac089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.039+0000 7fc9b95d0640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420a7fee0 0x559420a823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9b95d0640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420a7fee0 0x559420a823a0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc9b0009050 con 0x559420a7fee0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 realm 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a7f330 con 0x559420a7fee0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc9b0009050 con 0x559420a7fee0 2024-04-23T19:10:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a835d0 con 0x559420a7fee0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc9b0009050 con 0x559420a7fee0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a83ea0 con 0x559420a7fee0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc9b0009050 con 0x559420a7fee0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.040+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a84240 con 0x559420a7fee0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.041+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc9b0009050 con 0x559420a7fee0 2024-04-23T19:10:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.041+0000 7fc9be12d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.041+0000 7fc9be12d800 20 searching for the correct realm 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.041+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559420a86120 0x559420a88510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.041+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.041+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559420a86120 0x559420a88510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.041+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559420a86120 0x559420a88510 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9ba5d2640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc9b402c430 0x7fc9b4045460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9b402c430 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc9b402c430 0x7fc9b4045460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc9b402c430 0x7fc9b4045460 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9b9dd1640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc9ac09c040 0x7fc9ac05ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.042+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.043+0000 7fc9b95d0640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc9ac09c040 0x7fc9ac05ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.043+0000 7fc9b95d0640 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc9ac09c040 0x7fc9ac05ca60 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.043+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b0009050 con 0x7fc9ac09c040 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.043+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.043+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b0009050 con 0x559420a7fee0 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.043+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.044+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.044+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.044+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.044+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.044+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.044+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9b402c430 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.045+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.045+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.045+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.045+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.045+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.045+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9b402c430 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.046+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.047+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.048+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9b402c430 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.049+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.050+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.050+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9b402c430 2024-04-23T19:10:28.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.050+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.050+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a86120 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.050+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.050+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9ac09c040 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x559420a7fee0 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559420a88bc0 con 0x7fc9b402c430 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.051+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a84bc0 con 0x559420a7fee0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a85460 con 0x559420a7fee0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a84800 con 0x7fc9ac09c040 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 zone default found 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 4 Realm: () 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 zonegroup default 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.052+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a89020 con 0x7fc9ac09c040 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a8aeb0 con 0x559420a86120 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a8b250 con 0x559420a86120 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 20 started sync module instance, tier type = 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.053+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8a2f0 con 0x7fc9b402c430 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.054+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.054+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94094691444496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8baa0 con 0x7fc9b402c430 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.054+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8c2f0 con 0x559420a7fee0 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.055+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94094691444496] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.056+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.056+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94094691448304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8c2f0 con 0x559420a7fee0 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.056+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8cd70 con 0x559420a86120 2024-04-23T19:10:28.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.057+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94094691448304] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.057+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.057+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94094691453024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8c2f0 con 0x559420a86120 2024-04-23T19:10:28.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.057+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8d810 con 0x7fc9b402c430 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.058+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94094691453024] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.058+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.058+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94094691450432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8c260 con 0x7fc9b402c430 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.058+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8e6d0 con 0x559420a7fee0 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.059+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94094691450432] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.059+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.059+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94094691459616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8eaf0 con 0x559420a7fee0 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.059+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8f4b0 con 0x7fc9ac09c040 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.060+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94094691459616] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.060+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.060+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94094691461152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8f4b0 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.060+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8fe70 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.061+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94094691461152] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.061+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.061+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94094691463648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8fe70 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.061+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a90830 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94094691463648] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94094691466144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a90830 con 0x7fc9ac09c040 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 20 add_watcher() i=0 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 20 add_watcher() i=1 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 20 add_watcher() i=2 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 20 add_watcher() i=3 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 20 add_watcher() i=4 2024-04-23T19:10:28.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 20 add_watcher() i=5 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.062+0000 7fc9be12d800 20 add_watcher() i=6 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.063+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94094691466144] v30'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.063+0000 7fc9be12d800 20 add_watcher() i=7 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.063+0000 7fc9be12d800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.063+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420a93c30 con 0x559420a7fee0 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.064+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.064+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94094691468624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a91180 con 0x559420a7fee0 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94094691468624] v30'76 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc8637fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9be12d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9be12d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9be12d800 5 note: GC not initialized 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc8627fc640 20 reqs_thread_entry: start 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9be12d800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9be12d800 20 Filter name: none 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc7dbfff640 20 reqs_thread_entry: start 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9be12d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.065+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559420ac6c80 con 0x559420a7fee0 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.066+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.066+0000 7fc9be12d800 20 rados_obj.operate() r=0 bl.length=556 2024-04-23T19:10:28.076 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.067+0000 7fc8637fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.067+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94094691468624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420ac5850 con 0x559420a7fee0 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.068+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94094691468624] v30'77 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.068+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94094691444496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a90520 con 0x7fc9b402c430 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.068+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94094691444496] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.069+0000 7fc9be12d800 20 remove_watcher() i=0 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.069+0000 7fc9be12d800 2 removed watcher, disabling cache 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.069+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94094691448304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8a710 con 0x559420a7fee0 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.070+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94094691448304] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.070+0000 7fc9be12d800 20 remove_watcher() i=1 2024-04-23T19:10:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.070+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94094691453024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8a710 con 0x559420a86120 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.071+0000 7fc9ba5d2640 1 -- 172.21.15.63:0/1789643896 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94094691453024] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b400a4a0 con 0x559420a86120 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.071+0000 7fc9be12d800 20 remove_watcher() i=2 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.071+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94094691450432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8a710 con 0x7fc9b402c430 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.072+0000 7fc9b9dd1640 1 -- 172.21.15.63:0/1789643896 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94094691450432] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9ac075030 con 0x7fc9b402c430 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.072+0000 7fc9be12d800 20 remove_watcher() i=3 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.072+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94094691459616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8be40 con 0x559420a7fee0 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.073+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94094691459616] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x559420a7fee0 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.073+0000 7fc9be12d800 20 remove_watcher() i=4 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.073+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94094691461152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8a710 con 0x7fc9ac09c040 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.074+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94094691461152] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.074+0000 7fc9be12d800 20 remove_watcher() i=5 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.074+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94094691463648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8e220 con 0x7fc9ac09c040 2024-04-23T19:10:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.075+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94094691463648] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.075+0000 7fc9be12d800 20 remove_watcher() i=6 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.075+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94094691466144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559420a8be40 con 0x7fc9ac09c040 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94094691466144] v30'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc9b002a040 con 0x7fc9ac09c040 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 20 remove_watcher() i=7 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc9ac09c040 msgr2=0x7fc9ac05ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc9ac09c040 0x7fc9ac05ca60 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420a7fee0 msgr2=0x559420a823a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420a7fee0 0x559420a823a0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc9b402c430 msgr2=0x7fc9b4045460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc9b402c430 0x7fc9b4045460 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559420a86120 msgr2=0x559420a88510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559420a86120 0x559420a88510 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc884036e10 msgr2=0x7fc8840392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc884036e10 0x7fc8840392d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x559420a51b80 tx=0x7fc9ac089000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 msgr2=0x559420a743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a743e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc9b404d040 tx=0x7fc9b404cce0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9b95d0640 1 -- 172.21.15.63:0/1789643896 reap_dead start 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 shutdown_connections 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 --2- 172.21.15.63:0/1789643896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a6cf30 0x559420a743e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 >> 172.21.15.63:0/1789643896 conn(0x559420a69200 msgr2=0x559420a66860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.076+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 shutdown_connections 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/1789643896 wait complete. 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc894036e60 msgr2=0x7fc894039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc894036e60 0x7fc894039320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac006e60 tx=0x7fc9ac006f50 comp rx=0 tx=0).stop 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a5fb80 msgr2=0x559420a5ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a5fb80 0x559420a5ffa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x559420a4c040 tx=0x7fc9b40178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 shutdown_connections 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc894036e60 0x7fc894039320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/372004579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420a5fb80 0x559420a5ffa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 >> 172.21.15.63:0/372004579 conn(0x559420a53c20 msgr2=0x559420a54020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 shutdown_connections 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/372004579 wait complete. 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559420a4ee30 msgr2=0x559420a51220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559420a4ee30 0x559420a51220 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420949270 msgr2=0x559420a4b6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420949270 0x559420a4b6b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc8a0036e60 msgr2=0x7fc8a0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc8a0036e60 0x7fc8a0039320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000f60 tx=0x7fc9b00012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 msgr2=0x559420a44470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a44470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00a470 tx=0x7fc9ac003020 comp rx=0 tx=0).stop 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 shutdown_connections 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559420a4ee30 0x559420a51220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559420949270 0x559420a4b6b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc8a0036e60 0x7fc8a0039320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 --2- 172.21.15.63:0/3169563611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559420948010 0x559420a44470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 >> 172.21.15.63:0/3169563611 conn(0x559420943630 msgr2=0x559420a32730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 shutdown_connections 2024-04-23T19:10:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.077+0000 7fc9be12d800 1 -- 172.21.15.63:0/3169563611 wait complete. 2024-04-23T19:10:28.088 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:28.088 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-23T19:10:28.088 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-23T19:10:28.088 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-04-23T19:10:28.137 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.137 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.137+0000 7f4208f1f800 1 Processor -- start 2024-04-23T19:10:28.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.137+0000 7f4208f1f800 1 -- start start 2024-04-23T19:10:28.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.137+0000 7f4208f1f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558f6ab22030 0x558f6ab22450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.137+0000 7f4208f1f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558f6a9d57f0 con 0x558f6ab23290 2024-04-23T19:10:28.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.137+0000 7f4208f1f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x558f6a9c4e20 con 0x558f6ab22030 2024-04-23T19:10:28.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.138+0000 7f4204bc3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558f6ab22030 0x558f6ab22450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.138+0000 7f42043c2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x558f6ab23290 0x558f6ab23710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38628/0 (socket says 172.21.15.63:38628) 2024-04-23T19:10:28.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.138+0000 7f42043c2640 1 -- 172.21.15.63:0/970824515 learned_addr learned my addr 172.21.15.63:0/970824515 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:28.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.138+0000 7f4204bc3640 1 -- 172.21.15.63:0/970824515 >> v1:172.21.15.16:6789/0 conn(0x558f6ab23290 legacy=0x558f6ab23710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.138+0000 7f4204bc3640 1 -- 172.21.15.63:0/970824515 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6a9e3b50 con 0x558f6ab22030 2024-04-23T19:10:28.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.140+0000 7f4204bc3640 1 --2- 172.21.15.63:0/970824515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ab22450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f41f400a470 tx=0x7f41f401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca17aecd894f0fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.140+0000 7f4203bc1640 1 -- 172.21.15.63:0/970824515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f4057070 con 0x558f6ab22030 2024-04-23T19:10:28.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.140+0000 7f4203bc1640 1 -- 172.21.15.63:0/970824515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4052570 con 0x558f6ab22030 2024-04-23T19:10:28.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.140+0000 7f4203bc1640 1 -- 172.21.15.63:0/970824515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f4052870 con 0x558f6ab22030 2024-04-23T19:10:28.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.140+0000 7f4208f1f800 1 -- 172.21.15.63:0/970824515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 msgr2=0x558f6ab22450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.140+0000 7f4208f1f800 1 --2- 172.21.15.63:0/970824515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ab22450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f41f400a470 tx=0x7f41f401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:28.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.141+0000 7f4208f1f800 1 -- 172.21.15.63:0/970824515 shutdown_connections 2024-04-23T19:10:28.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.141+0000 7f4208f1f800 1 --2- 172.21.15.63:0/970824515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ab22450 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.141+0000 7f4208f1f800 1 -- 172.21.15.63:0/970824515 >> 172.21.15.63:0/970824515 conn(0x558f6ab1d650 msgr2=0x558f6ab1fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.141+0000 7f4208f1f800 1 -- 172.21.15.63:0/970824515 shutdown_connections 2024-04-23T19:10:28.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.141+0000 7f4208f1f800 1 -- 172.21.15.63:0/970824515 wait complete. 2024-04-23T19:10:28.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.153+0000 7f4208f1f800 1 Processor -- start 2024-04-23T19:10:28.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.153+0000 7f4208f1f800 1 -- start start 2024-04-23T19:10:28.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.153+0000 7f4208f1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6aa96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.153+0000 7f4208f1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f6aa6e710 con 0x558f6ab22030 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.154+0000 7f4204bc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6aa96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.154+0000 7f4204bc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6aa96100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53666/0 (socket says 172.21.15.63:53666) 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.154+0000 7f4204bc3640 1 -- 172.21.15.63:0/1161545274 learned_addr learned my addr 172.21.15.63:0/1161545274 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.154+0000 7f4204bc3640 1 -- 172.21.15.63:0/1161545274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6ac17570 con 0x558f6ab22030 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.154+0000 7f4204bc3640 1 --2- 172.21.15.63:0/1161545274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6aa96100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f41f401ced0 tx=0x7f41f404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988ed635b0e9f5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f40f2e0d640 1 -- 172.21.15.63:0/1161545274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f4057040 con 0x558f6ab22030 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f40f2e0d640 1 -- 172.21.15.63:0/1161545274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f40529a0 con 0x558f6ab22030 2024-04-23T19:10:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f40f2e0d640 1 -- 172.21.15.63:0/1161545274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f4052cc0 con 0x558f6ab22030 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f4208f1f800 1 -- 172.21.15.63:0/1161545274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 msgr2=0x558f6aa96100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1161545274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6aa96100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f41f401ced0 tx=0x7f41f404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f4208f1f800 1 -- 172.21.15.63:0/1161545274 shutdown_connections 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1161545274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6aa96100 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f4208f1f800 1 -- 172.21.15.63:0/1161545274 >> 172.21.15.63:0/1161545274 conn(0x558f6ab1d650 msgr2=0x558f6ac0f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f4208f1f800 1 -- 172.21.15.63:0/1161545274 shutdown_connections 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.155+0000 7f4208f1f800 1 -- 172.21.15.63:0/1161545274 wait complete. 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4208f1f800 1 Processor -- start 2024-04-23T19:10:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4208f1f800 1 -- start start 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4208f1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ac1c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4208f1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f6ab2b700 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4204bc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ac1c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4204bc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ac1c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53680/0 (socket says 172.21.15.63:53680) 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4204bc3640 1 -- 172.21.15.63:0/61784162 learned_addr learned my addr 172.21.15.63:0/61784162 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.156+0000 7f4204bc3640 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6ac1c800 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.157+0000 7f4204bc3640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ac1c2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f41f4000c00 tx=0x7f41f4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.157+0000 7f40f160a640 1 -- 172.21.15.63:0/61784162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f4073070 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.157+0000 7f40f160a640 1 -- 172.21.15.63:0/61784162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f405d570 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.157+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f6ac1ca90 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.157+0000 7f40f160a640 1 -- 172.21.15.63:0/61784162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f4057070 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.157+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6ac1f740 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.157+0000 7f40f160a640 1 -- 172.21.15.63:0/61784162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f41f4076070 con 0x558f6ab22030 2024-04-23T19:10:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f40f160a640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40ec036e60 0x7f40ec039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f40f160a640 1 -- 172.21.15.63:0/61784162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f41f408e440 con 0x558f6ab22030 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f42043c2640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40ec036e60 0x7f40ec039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ab23290 0x558f6ac23500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac23ae0 con 0x558f6ab23290 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f42053c4640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ab23290 0x558f6ac23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f42043c2640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40ec036e60 0x7f40ec039320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000f60 tx=0x7f41f80012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.158+0000 7f42053c4640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ab23290 0x558f6ac23500 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.159+0000 7f42053c4640 1 -- 172.21.15.63:0/61784162 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ab23290 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.159+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac20ad0 con 0x558f6ab23290 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.159+0000 7f42053c4640 1 -- 172.21.15.63:0/61784162 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ab23290 2024-04-23T19:10:28.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.159+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac25920 con 0x558f6ab23290 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.159+0000 7f42053c4640 1 -- 172.21.15.63:0/61784162 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ab23290 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.159+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac24c60 con 0x558f6ab23290 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.160+0000 7f42053c4640 1 -- 172.21.15.63:0/61784162 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ab23290 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.160+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f6ac26cd0 0x558f6ac290c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.160+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac29640 con 0x558f6ac26cd0 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.160+0000 7f4204bc3640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f6ac26cd0 0x558f6ac290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.160+0000 7f4204bc3640 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f6ac26cd0 0x558f6ac290c0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f4204bc3640 1 -- 172.21.15.63:0/61784162 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f41f4075030 con 0x558f6ac26cd0 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f4208f1f800 1 Processor -- start 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f4208f1f800 1 -- start start 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f4208f1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac15120 0x558f6ac13590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f4208f1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f6aa14e30 con 0x558f6ac15120 2024-04-23T19:10:28.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f42053c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac15120 0x558f6ac13590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f42053c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac15120 0x558f6ac13590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53692/0 (socket says 172.21.15.63:53692) 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f42053c4640 1 -- 172.21.15.63:0/3890576683 learned_addr learned my addr 172.21.15.63:0/3890576683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.161+0000 7f42053c4640 1 -- 172.21.15.63:0/3890576683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6ac13ad0 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.162+0000 7f42053c4640 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac15120 0x558f6ac13590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x558f6ac23ee0 tx=0x7f41fc0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.162+0000 7f40ea7fc640 1 -- 172.21.15.63:0/3890576683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41fc013440 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.162+0000 7f40ea7fc640 1 -- 172.21.15.63:0/3890576683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc0135e0 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.162+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f6ac13d60 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.162+0000 7f40ea7fc640 1 -- 172.21.15.63:0/3890576683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41fc0138c0 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.162+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6ac14260 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f40ea7fc640 1 -- 172.21.15.63:0/3890576683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f41fc011040 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f40ea7fc640 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40dc036e60 0x7f40dc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f40ea7fc640 1 -- 172.21.15.63:0/3890576683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f41fc01d070 con 0x558f6ac15120 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f4204bc3640 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40dc036e60 0x7f40dc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f4208f1f800 1 Processor -- start 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f4208f1f800 1 -- start start 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f4204bc3640 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40dc036e60 0x7f40dc039320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41f408caf0 tx=0x7f41f408c760 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f4208f1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac47370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f4208f1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f6ac17800 con 0x558f6ac46f50 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f42053c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac47370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f42053c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac47370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53702/0 (socket says 172.21.15.63:53702) 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.163+0000 7f42053c4640 1 -- 172.21.15.63:0/1756287362 learned_addr learned my addr 172.21.15.63:0/1756287362 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f42053c4640 1 -- 172.21.15.63:0/1756287362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6ac47940 con 0x558f6ac46f50 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f42053c4640 1 --2- 172.21.15.63:0/1756287362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac47370 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f41fc04cd50 tx=0x7f41fc04d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19faecd7e9bb5abd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f40dbfff640 1 -- 172.21.15.63:0/1756287362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41fc015560 con 0x558f6ac46f50 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f40dbfff640 1 -- 172.21.15.63:0/1756287362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc02a350 con 0x558f6ac46f50 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f40dbfff640 1 -- 172.21.15.63:0/1756287362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41fc012ca0 con 0x558f6ac46f50 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f4208f1f800 1 -- 172.21.15.63:0/1756287362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 msgr2=0x558f6ac47370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1756287362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac47370 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f41fc04cd50 tx=0x7f41fc04d000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f4208f1f800 1 -- 172.21.15.63:0/1756287362 shutdown_connections 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1756287362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac47370 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f4208f1f800 1 -- 172.21.15.63:0/1756287362 >> 172.21.15.63:0/1756287362 conn(0x558f6ac432b0 msgr2=0x558f6ac41310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.164+0000 7f4208f1f800 1 -- 172.21.15.63:0/1756287362 shutdown_connections 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f4208f1f800 1 -- 172.21.15.63:0/1756287362 wait complete. 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f4208f1f800 1 Processor -- start 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f4208f1f800 1 -- start start 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f4208f1f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac4e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f4208f1f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f6ac480b0 con 0x558f6ac46f50 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f42053c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac4e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f42053c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac4e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53716/0 (socket says 172.21.15.63:53716) 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 learned_addr learned my addr 172.21.15.63:0/1068250759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.165+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6ac4ea20 con 0x558f6ac46f50 2024-04-23T19:10:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.166+0000 7f42053c4640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac4e4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f41fc04bc20 tx=0x7f41fc04cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.166+0000 7f40da7fc640 1 -- 172.21.15.63:0/1068250759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41fc015950 con 0x558f6ac46f50 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.166+0000 7f40da7fc640 1 -- 172.21.15.63:0/1068250759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc02a8e0 con 0x558f6ac46f50 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.166+0000 7f40da7fc640 1 -- 172.21.15.63:0/1068250759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41fc013c20 con 0x558f6ac46f50 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.166+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f6ac4ecb0 con 0x558f6ac46f50 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.166+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6ac51960 con 0x558f6ac46f50 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.166+0000 7f40da7fc640 1 -- 172.21.15.63:0/1068250759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f41fc04eb20 con 0x558f6ac46f50 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.167+0000 7f40da7fc640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40d0036e60 0x7f40d0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.167+0000 7f40da7fc640 1 -- 172.21.15.63:0/1068250759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f41fc00e040 con 0x558f6ac46f50 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.167+0000 7f4204bc3640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40d0036e60 0x7f40d0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.167+0000 7f4204bc3640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40d0036e60 0x7f40d0039320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41f4006a00 tx=0x7f41f4088000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.167+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.167+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ac5a070 0x558f6ac5c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.167+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac5cc80 con 0x558f6ac5a070 2024-04-23T19:10:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f42043c2640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ac5a070 0x558f6ac5c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f42043c2640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ac5a070 0x558f6ac5c530 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f41f8009050 con 0x558f6ac5a070 2024-04-23T19:10:28.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 20 realm 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac594c0 con 0x558f6ac5a070 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f41f8009050 con 0x558f6ac5a070 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac5d770 con 0x558f6ac5a070 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.168+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f41f8009050 con 0x558f6ac5a070 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac5e040 con 0x558f6ac5a070 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f41f8009050 con 0x558f6ac5a070 2024-04-23T19:10:28.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac5e3e0 con 0x558f6ac5a070 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f41f8009050 con 0x558f6ac5a070 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 20 searching for the correct realm 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f6ac602c0 0x558f6ac626b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x558f6ac602c0 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f42053c4640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f6ac602c0 0x558f6ac626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.169+0000 7f42053c4640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f6ac602c0 0x558f6ac626b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.170+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.170+0000 7f42053c4640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f41fc001540 0x7f41fc02b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.170+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x7f41fc001540 2024-04-23T19:10:28.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.170+0000 7f4204bc3640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f41fc001540 0x7f41fc02b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.170+0000 7f4204bc3640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f41fc001540 0x7f41fc02b3a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.171+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.171+0000 7f4204bc3640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f41f4078060 0x7f41f405ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.171+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x7f41f4078060 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.171+0000 7f42043c2640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f41f4078060 0x7f41f405ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.171+0000 7f42043c2640 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f41f4078060 0x7f41f405ea40 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.171+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f8009050 con 0x7f41f4078060 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.171+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x558f6ac5a070 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.172+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f8009050 con 0x558f6ac5a070 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.172+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x558f6ac5a070 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.172+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.172+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x7f41f4078060 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.172+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.172+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x7f41f4078060 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x7f41fc001540 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x558f6ac602c0 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x558f6ac5a070 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.173+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x7f41f4078060 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x558f6ac5a070 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac62d60 con 0x7f41fc001540 2024-04-23T19:10:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.174+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.175+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.175+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41f4078060 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.175+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.175+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.175+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.175+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.175+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.176+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac5a070 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.176+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.176+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac5a070 2024-04-23T19:10:28.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.176+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.176+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41f4078060 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac602c0 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41fc001540 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac5a070 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.177+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41f4078060 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.178+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.178+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac602c0 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.178+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.178+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41f4078060 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.178+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.178+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41fc001540 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.178+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.179+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac602c0 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.179+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.179+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41f4078060 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.179+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.179+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x558f6ac5a070 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.179+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.179+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f41f40a8060 con 0x7f41fc001540 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac5ed60 con 0x558f6ac5a070 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac5f600 con 0x558f6ac5a070 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.180+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac631e0 con 0x7f41f4078060 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 zone default found 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 4 Realm: () 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 zonegroup default 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac63690 con 0x7f41f4078060 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac65070 con 0x558f6ac602c0 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.181+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.182+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac65530 con 0x558f6ac602c0 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.182+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.182+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.182+0000 7f4208f1f800 20 started sync module instance, tier type = 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.182+0000 7f4208f1f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.182+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac64500 con 0x7f41fc001540 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.183+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.183+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94074460063984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac64160 con 0x7f41fc001540 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.183+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac66460 con 0x558f6ac5a070 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.184+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94074460063984] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.184+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.184+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94074460069808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac66880 con 0x558f6ac5a070 2024-04-23T19:10:28.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.184+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac67240 con 0x558f6ac602c0 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.185+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94074460069808] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.185+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.185+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94074460073840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac66880 con 0x558f6ac602c0 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.185+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac67cd0 con 0x7f41fc001540 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.186+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94074460073840] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.186+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.187+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94074460070864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac667f0 con 0x7f41fc001540 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.187+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac68b90 con 0x558f6ac5a070 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.188+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94074460070864] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.188+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.188+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94074460079840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac68b00 con 0x558f6ac5a070 2024-04-23T19:10:28.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.188+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac69a50 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.189+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.189+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94074460083616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac69e70 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.189+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac6a940 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.189+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94074460079840] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.189+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94074460083616] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.190+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.190+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94074460087344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac699c0 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.190+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac6b1e0 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94074460089408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac6b510 con 0x7f41f4078060 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=0 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=1 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=2 2024-04-23T19:10:28.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=3 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=4 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=5 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94074460087344] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=6 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94074460089408] v30'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 20 add_watcher() i=7 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.191+0000 7f4208f1f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.192+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac6e1a0 con 0x558f6ac5a070 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.192+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.192+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94074460082048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac6b510 con 0x558f6ac5a070 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.193+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94074460082048] v30'78 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.193+0000 7f40bafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.193+0000 7f4208f1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.193+0000 7f4208f1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.193+0000 7f4208f1f800 5 note: GC not initialized 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.193+0000 7f40b9fe3640 20 reqs_thread_entry: start 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.193+0000 7f4208f1f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 20 Filter name: none 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f402b7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6aca16e0 con 0x558f6ac5a070 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 20 rados_obj.operate() r=0 bl.length=556 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6aca22f0 con 0x7f41fc001540 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.194+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~673 in=673b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6aca7f60 con 0x558f6ac5a070 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.195+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~673] v30'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f4208f1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f4208f1f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f4208f1f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94074460338704 in=861b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6aca4be0 con 0x558f6ac5a070 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94074460079840 notify 128849018887 ret 0) ==== 891+0+0 (crc 0 0 0) 0x558f6a9d57f0 con 0x558f6ac5a070 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f40dbfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018887 cookie 94074460079840 notifier 4387 bl.length()=849 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f40dbfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f40dbfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f40dbfff640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f40c8003ea0 con 0x558f6ac5a070 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(65 notify.4 [notify cookie 94074460338704 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94074460338704 notify 128849018887 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f41f801b8b0 con 0x558f6ac5a070 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.196+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6aca7a10 con 0x7f41fc001540 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.197+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v30'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.197+0000 7f4208f1f800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.197+0000 7f4208f1f800 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.197+0000 7f4208f1f800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.197+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94074460338704 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f6aca4be0 con 0x7f41fc001540 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== watch-notify(notify (1) cookie 94074460063984 notify 128849018883 ret 0) ==== 208+0+0 (crc 0 0 0) 0x7f41f40a2040 con 0x7f41fc001540 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f40db7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018883 cookie 94074460063984 notifier 4387 bl.length()=166 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f40db7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f40db7fe640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.0 [notify cookie 94074460338704 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f40db7fe640 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f4038003400 con 0x7f41fc001540 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.198+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== watch-notify(notify_complete (2) cookie 94074460338704 notify 128849018883 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558f6ac1f740 con 0x7f41fc001540 2024-04-23T19:10:28.209 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-23T19:10:28.195295Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:28.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.199+0000 7f40bafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.199+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94074460082048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6aca0a00 con 0x558f6ac5a070 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.200+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94074460082048] v30'79 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.200+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94074460063984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac68f80 con 0x7f41fc001540 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.201+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94074460063984] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.201+0000 7f4208f1f800 20 remove_watcher() i=0 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.201+0000 7f4208f1f800 2 removed watcher, disabling cache 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.201+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94074460069808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac68f80 con 0x558f6ac5a070 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.202+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94074460069808] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.202+0000 7f4208f1f800 20 remove_watcher() i=1 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.202+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94074460073840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac65fb0 con 0x558f6ac602c0 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.203+0000 7f42053c4640 1 -- 172.21.15.63:0/1068250759 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94074460073840] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41fc00a4a0 con 0x558f6ac602c0 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.203+0000 7f4208f1f800 20 remove_watcher() i=2 2024-04-23T19:10:28.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.203+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94074460070864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac68f80 con 0x7f41fc001540 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.204+0000 7f4204bc3640 1 -- 172.21.15.63:0/1068250759 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94074460070864] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f4098070 con 0x7f41fc001540 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.204+0000 7f4208f1f800 20 remove_watcher() i=3 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.204+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94074460079840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac66f70 con 0x558f6ac5a070 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.205+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94074460079840] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x558f6ac5a070 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.205+0000 7f4208f1f800 20 remove_watcher() i=4 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.205+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94074460083616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac68f80 con 0x7f41f4078060 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.206+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94074460083616] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.206+0000 7f4208f1f800 20 remove_watcher() i=5 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.206+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94074460087344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac686e0 con 0x7f41f4078060 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.207+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94074460087344] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.207+0000 7f4208f1f800 20 remove_watcher() i=6 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.207+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94074460089408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f6ac66f70 con 0x7f41f4078060 2024-04-23T19:10:28.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94074460089408] v30'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f41f802a040 con 0x7f41f4078060 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 20 remove_watcher() i=7 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f41f4078060 msgr2=0x7f41f405ea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f41f4078060 0x7f41f405ea40 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ac5a070 msgr2=0x558f6ac5c530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ac5a070 0x558f6ac5c530 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f41fc001540 msgr2=0x7f41fc02b3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f41fc001540 0x7f41fc02b3a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f6ac602c0 msgr2=0x558f6ac626b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.208+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f6ac602c0 0x558f6ac626b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40d0036e60 msgr2=0x7f40d0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40d0036e60 0x7f40d0039320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41f4006a00 tx=0x7f41f4088000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 msgr2=0x558f6ac4e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/1068250759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac46f50 0x558f6ac4e4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f41fc04bc20 tx=0x7f41fc04cce0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f42043c2640 1 -- 172.21.15.63:0/1068250759 reap_dead start 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 shutdown_connections 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 >> 172.21.15.63:0/1068250759 conn(0x558f6ac432b0 msgr2=0x558f6ac40910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 shutdown_connections 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/1068250759 wait complete. 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40dc036e60 msgr2=0x7f40dc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40dc036e60 0x7f40dc039320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41f408caf0 tx=0x7f41f408c760 comp rx=0 tx=0).stop 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac15120 msgr2=0x558f6ac13590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac15120 0x558f6ac13590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x558f6ac23ee0 tx=0x7f41fc0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 shutdown_connections 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40dc036e60 0x7f40dc039320 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/3890576683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ac15120 0x558f6ac13590 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 >> 172.21.15.63:0/3890576683 conn(0x558f6ac2bb00 msgr2=0x558f6ac2bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 shutdown_connections 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/3890576683 wait complete. 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f6ac26cd0 msgr2=0x558f6ac290c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f6ac26cd0 0x558f6ac290c0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ab23290 msgr2=0x558f6ac23500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.209+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ab23290 0x558f6ac23500 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40ec036e60 msgr2=0x7f40ec039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40ec036e60 0x7f40ec039320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000f60 tx=0x7f41f80012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 msgr2=0x558f6ac1c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ac1c2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f41f4000c00 tx=0x7f41f4003020 comp rx=0 tx=0).stop 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 shutdown_connections 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f6ac26cd0 0x558f6ac290c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f6ab23290 0x558f6ac23500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f40ec036e60 0x7f40ec039320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 --2- 172.21.15.63:0/61784162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f6ab22030 0x558f6ac1c2c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 >> 172.21.15.63:0/61784162 conn(0x558f6ab1d650 msgr2=0x558f6ac0d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 shutdown_connections 2024-04-23T19:10:28.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.210+0000 7f4208f1f800 1 -- 172.21.15.63:0/61784162 wait complete. 2024-04-23T19:10:28.220 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-23T19:10:28.195295Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:28.220 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:10:28.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:10:28.220 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:10:28.264 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.264 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.264+0000 7f48c9c3f800 1 Processor -- start 2024-04-23T19:10:28.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c9c3f800 1 -- start start 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c9c3f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5622733ee6a0 0x5622733f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c9c3f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56227329c7f0 con 0x5622733eab90 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c9c3f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x56227328be20 con 0x5622733ee6a0 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c6f3d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5622733ee6a0 0x5622733f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c773e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5622733eab90 0x5622733ecf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38636/0 (socket says 172.21.15.63:38636) 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c773e640 1 -- 172.21.15.63:0/3775497504 learned_addr learned my addr 172.21.15.63:0/3775497504 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c6f3d640 1 -- 172.21.15.63:0/3775497504 >> v1:172.21.15.16:6789/0 conn(0x5622733eab90 legacy=0x5622733ecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.265+0000 7f48c6f3d640 1 -- 172.21.15.63:0/3775497504 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622732aab50 con 0x5622733ee6a0 2024-04-23T19:10:28.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.267+0000 7f48c6f3d640 1 --2- 172.21.15.63:0/3775497504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ee6a0 0x5622733f0a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f48b000a470 tx=0x7f48b001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177098a2e4a7e1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.267+0000 7f48c673c640 1 -- 172.21.15.63:0/3775497504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b0057070 con 0x5622733ee6a0 2024-04-23T19:10:28.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.267+0000 7f48c673c640 1 -- 172.21.15.63:0/3775497504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b0052570 con 0x5622733ee6a0 2024-04-23T19:10:28.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.267+0000 7f48c673c640 1 -- 172.21.15.63:0/3775497504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b0052870 con 0x5622733ee6a0 2024-04-23T19:10:28.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.268+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3775497504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ee6a0 msgr2=0x5622733f0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.268+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/3775497504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ee6a0 0x5622733f0a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f48b000a470 tx=0x7f48b001ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:28.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.268+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3775497504 shutdown_connections 2024-04-23T19:10:28.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.268+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/3775497504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ee6a0 0x5622733f0a90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.268+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3775497504 >> 172.21.15.63:0/3775497504 conn(0x5622733e4600 msgr2=0x5622733e6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.268+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3775497504 shutdown_connections 2024-04-23T19:10:28.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.268+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3775497504 wait complete. 2024-04-23T19:10:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c9c3f800 1 Processor -- start 2024-04-23T19:10:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c9c3f800 1 -- start start 2024-04-23T19:10:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c9c3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734db380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c9c3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5622733356f0 con 0x5622733eab90 2024-04-23T19:10:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c773e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c773e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734db380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53734/0 (socket says 172.21.15.63:53734) 2024-04-23T19:10:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c773e640 1 -- 172.21.15.63:0/3532192298 learned_addr learned my addr 172.21.15.63:0/3532192298 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.281+0000 7f48c773e640 1 -- 172.21.15.63:0/3532192298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622734db950 con 0x5622733eab90 2024-04-23T19:10:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.282+0000 7f48c773e640 1 --2- 172.21.15.63:0/3532192298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734db380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f48b8001de0 tx=0x7f48b800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448f85f3c3099fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.282+0000 7f47b560a640 1 -- 172.21.15.63:0/3532192298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b8012020 con 0x5622733eab90 2024-04-23T19:10:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.282+0000 7f47b560a640 1 -- 172.21.15.63:0/3532192298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b800e5a0 con 0x5622733eab90 2024-04-23T19:10:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.282+0000 7f47b560a640 1 -- 172.21.15.63:0/3532192298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b800e8c0 con 0x5622733eab90 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.282+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3532192298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 msgr2=0x5622734db380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.282+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/3532192298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734db380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f48b8001de0 tx=0x7f48b800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3532192298 shutdown_connections 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/3532192298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734db380 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3532192298 >> 172.21.15.63:0/3532192298 conn(0x5622733e4600 msgr2=0x5622734d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3532192298 shutdown_connections 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3532192298 wait complete. 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 Processor -- start 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 -- start start 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734e2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c9c3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5622733f12d0 con 0x5622733eab90 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c773e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734e2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c773e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734e2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53740/0 (socket says 172.21.15.63:53740) 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.283+0000 7f48c773e640 1 -- 172.21.15.63:0/109575324 learned_addr learned my addr 172.21.15.63:0/109575324 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.284+0000 7f48c773e640 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622734e3400 con 0x5622733eab90 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.284+0000 7f48c773e640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734e2ec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c010 tx=0x7f48b8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.284+0000 7f47b3e07640 1 -- 172.21.15.63:0/109575324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b8012070 con 0x5622733eab90 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.284+0000 7f47b3e07640 1 -- 172.21.15.63:0/109575324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b801a7a0 con 0x5622733eab90 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.284+0000 7f47b3e07640 1 -- 172.21.15.63:0/109575324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b801aa80 con 0x5622733eab90 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.284+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5622734e3690 con 0x5622733eab90 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.284+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622734e6340 con 0x5622733eab90 2024-04-23T19:10:28.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f47b3e07640 1 -- 172.21.15.63:0/109575324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f48b8015020 con 0x5622733eab90 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f47b3e07640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47ac036e60 0x7f47ac039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f47b3e07640 1 -- 172.21.15.63:0/109575324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f48b8020070 con 0x5622733eab90 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f48c6f3d640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47ac036e60 0x7f47ac039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5622733ee6a0 0x5622734ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5622734ea640 con 0x5622733ee6a0 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5622733ee6a0 0x5622734ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.285+0000 7f48c6f3d640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47ac036e60 0x7f47ac039320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f48b0054000 tx=0x7f48b004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.286+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5622733ee6a0 0x5622734ea100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.286+0000 7f48c7f3f640 1 -- 172.21.15.63:0/109575324 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48c000a4a0 con 0x5622733ee6a0 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.286+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5622734e76b0 con 0x5622733ee6a0 2024-04-23T19:10:28.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.286+0000 7f48c7f3f640 1 -- 172.21.15.63:0/109575324 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f48c000a4a0 con 0x5622733ee6a0 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.286+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5622734ec460 con 0x5622733ee6a0 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.286+0000 7f48c7f3f640 1 -- 172.21.15.63:0/109575324 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f48c000a4a0 con 0x5622733ee6a0 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.286+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5622734eb7a0 con 0x5622733ee6a0 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.287+0000 7f48c7f3f640 1 -- 172.21.15.63:0/109575324 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f48c000a4a0 con 0x5622733ee6a0 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.287+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5622734ed810 0x5622734efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.287+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5622734f0180 con 0x5622734ed810 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.287+0000 7f48c773e640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5622734ed810 0x5622734efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.287+0000 7f48c773e640 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5622734ed810 0x5622734efc00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c773e640 1 -- 172.21.15.63:0/109575324 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f48b802b070 con 0x5622734ed810 2024-04-23T19:10:28.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c9c3f800 1 Processor -- start 2024-04-23T19:10:28.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c9c3f800 1 -- start start 2024-04-23T19:10:28.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c9c3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622734fe5a0 0x5622734fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c9c3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5622732dbe30 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c7f3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622734fe5a0 0x5622734fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c7f3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622734fe5a0 0x5622734fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53746/0 (socket says 172.21.15.63:53746) 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.288+0000 7f48c7f3f640 1 -- 172.21.15.63:0/515481622 learned_addr learned my addr 172.21.15.63:0/515481622 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.289+0000 7f48c7f3f640 1 -- 172.21.15.63:0/515481622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622734fef00 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.289+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622734fe5a0 0x5622734fe9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5622734e3bd0 tx=0x7f48c00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.289+0000 7f47b1602640 1 -- 172.21.15.63:0/515481622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c0013440 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.289+0000 7f47b1602640 1 -- 172.21.15.63:0/515481622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c00135e0 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.289+0000 7f47b1602640 1 -- 172.21.15.63:0/515481622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c00138c0 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.289+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5622734ff970 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.289+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622734ffe70 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.290+0000 7f47b1602640 1 -- 172.21.15.63:0/515481622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f48c0011040 con 0x5622734fe5a0 2024-04-23T19:10:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.290+0000 7f47b1602640 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47a0036e60 0x7f47a0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.290+0000 7f47b1602640 1 -- 172.21.15.63:0/515481622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f48c001d070 con 0x5622734fe5a0 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.290+0000 7f48c773e640 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47a0036e60 0x7f47a0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.290+0000 7f48c9c3f800 1 Processor -- start 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.290+0000 7f48c9c3f800 1 -- start start 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.290+0000 7f48c773e640 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47a0036e60 0x7f47a0039320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f48b804b350 tx=0x7f48b8041040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f48c9c3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622733ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f48c9c3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5622734dbcd0 con 0x5622733ea1b0 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f48c7f3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622733ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f48c7f3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622733ea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53748/0 (socket says 172.21.15.63:53748) 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f48c7f3f640 1 -- 172.21.15.63:0/3362353130 learned_addr learned my addr 172.21.15.63:0/3362353130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f48c7f3f640 1 -- 172.21.15.63:0/3362353130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622733e8a10 con 0x5622733ea1b0 2024-04-23T19:10:28.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/3362353130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622733ea5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f48c004ccb0 tx=0x7f48c0012ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95479476c7af064 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f47aaffd640 1 -- 172.21.15.63:0/3362353130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c0012df0 con 0x5622733ea1b0 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f47aaffd640 1 -- 172.21.15.63:0/3362353130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c0001310 con 0x5622733ea1b0 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.291+0000 7f47aaffd640 1 -- 172.21.15.63:0/3362353130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c0046aa0 con 0x5622733ea1b0 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3362353130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 msgr2=0x5622733ea5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/3362353130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622733ea5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f48c004ccb0 tx=0x7f48c0012ac0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3362353130 shutdown_connections 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/3362353130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622733ea5d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3362353130 >> 172.21.15.63:0/3362353130 conn(0x562273507bd0 msgr2=0x562273505c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3362353130 shutdown_connections 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 -- 172.21.15.63:0/3362353130 wait complete. 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 Processor -- start 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 -- start start 2024-04-23T19:10:28.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622735150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.292+0000 7f48c9c3f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5622733e8eb0 con 0x5622733ea1b0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f48c7f3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622735150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f48c7f3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622735150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53758/0 (socket says 172.21.15.63:53758) 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 learned_addr learned my addr 172.21.15.63:0/176707550 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562273515600 con 0x5622733ea1b0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622735150c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f48c0012c10 tx=0x7f48c004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f47a97fa640 1 -- 172.21.15.63:0/176707550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c0021c60 con 0x5622733ea1b0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f47a97fa640 1 -- 172.21.15.63:0/176707550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c0021e20 con 0x5622733ea1b0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562273515890 con 0x5622733ea1b0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f47a97fa640 1 -- 172.21.15.63:0/176707550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c002a350 con 0x5622733ea1b0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.293+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562273518540 con 0x5622733ea1b0 2024-04-23T19:10:28.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.294+0000 7f47a97fa640 1 -- 172.21.15.63:0/176707550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f48c0040040 con 0x5622733ea1b0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.294+0000 7f47a97fa640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.294+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.294+0000 7f47a97fa640 1 -- 172.21.15.63:0/176707550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f48c000e040 con 0x5622733ea1b0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.294+0000 7f48c773e640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562273520d10 0x5622735231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5622735238f0 con 0x562273520d10 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c6f3d640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562273520d10 0x5622735231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c773e640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f48b8000fd0 tx=0x7f48b8044610 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c6f3d640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562273520d10 0x5622735231a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c9c3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c9c3f800 20 realm 2024-04-23T19:10:28.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273523cc0 con 0x562273520d10 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.295+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273524590 con 0x562273520d10 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273524930 con 0x562273520d10 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273524d50 con 0x562273520d10 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 20 searching for the correct realm 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562273526d50 0x5622735291a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.296+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562273526d50 0x5622735291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.297+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562273526d50 0x5622735291a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.297+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.297+0000 7f48c7f3f640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48c005b040 0x7f48c0046830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.297+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48c005b040 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.297+0000 7f48c773e640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48c005b040 0x7f48c0046830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.298+0000 7f48c773e640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48c005b040 0x7f48c0046830 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.298+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.298+0000 7f48c773e640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f48b8017050 0x7f48b800d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.298+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.298+0000 7f48c6f3d640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f48b8017050 0x7f48b800d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.298+0000 7f48c6f3d640 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f48b8017050 0x7f48b800d8e0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.299+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48c005b040 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.300+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48c005b040 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.301+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.302+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.303+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.303+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.303+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.303+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.303+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.303+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.304+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.304+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48c005b040 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.304+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.304+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.304+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.304+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48c005b040 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.305+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273526d50 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.306+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.306+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48b8017050 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.306+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.306+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x562273520d10 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.306+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.306+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x562273529870 con 0x7f48c005b040 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5622735257f0 con 0x562273520d10 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273526090 con 0x562273520d10 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.307+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273525430 con 0x7f48b8017050 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 zone default found 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 4 Realm: () 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 zonegroup default 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273526090 con 0x7f48b8017050 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56227352b720 con 0x562273526d50 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56227352bac0 con 0x562273526d50 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.308+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.309+0000 7f48c9c3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.309+0000 7f48c9c3f800 20 started sync module instance, tier type = 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.309+0000 7f48c9c3f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.309+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352abc0 con 0x7f48c005b040 2024-04-23T19:10:28.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.310+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.310+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94705963675616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352c3a0 con 0x7f48c005b040 2024-04-23T19:10:28.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.310+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352cbf0 con 0x562273520d10 2024-04-23T19:10:28.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.311+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94705963675616] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.311+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.311+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94705963681600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352d010 con 0x562273520d10 2024-04-23T19:10:28.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.311+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352d860 con 0x562273526d50 2024-04-23T19:10:28.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.312+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94705963681600] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.312+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.312+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94705963683136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352d770 con 0x562273526d50 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.312+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352e290 con 0x7f48c005b040 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.313+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94705963683136] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.313+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.313+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94705963687264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352e680 con 0x7f48c005b040 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.313+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352f0d0 con 0x562273520d10 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.314+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94705963687264] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.315+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.315+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94705963690944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352f090 con 0x562273520d10 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.315+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352fb80 con 0x7f48b8017050 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.316+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.316+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94705963693664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352fb80 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.316+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562273530aa0 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.316+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94705963690944] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.316+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94705963693664] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.317+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.317+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94705963695984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352d1c0 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.317+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562273530f10 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94705963695984] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94705963698784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562273531330 con 0x7f48b8017050 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 20 add_watcher() i=0 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 20 add_watcher() i=1 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 20 add_watcher() i=2 2024-04-23T19:10:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 20 add_watcher() i=3 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 20 add_watcher() i=4 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 20 add_watcher() i=5 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.318+0000 7f48c9c3f800 20 add_watcher() i=6 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.319+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94705963698784] v30'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.319+0000 7f48c9c3f800 20 add_watcher() i=7 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.319+0000 7f48c9c3f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.319+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273531330 con 0x562273520d10 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.319+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.319+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94705963699840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5622735312d0 con 0x562273520d10 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.320+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94705963699840] v30'80 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f477dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f48c9c3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f48c9c3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f48c9c3f800 5 note: GC not initialized 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f477cfe1640 20 reqs_thread_entry: start 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f48c9c3f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f48c9c3f800 20 Filter name: none 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f46f67fc640 20 reqs_thread_entry: start 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f48c9c3f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.321+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562273567210 con 0x562273520d10 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.322+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.322+0000 7f48c9c3f800 20 rados_obj.operate() r=0 bl.length=673 2024-04-23T19:10:28.335 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-23T19:10:28.195295Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:28.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.323+0000 7f477dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.323+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94705963699840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5622735675f0 con 0x562273520d10 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.323+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94705963699840] v30'81 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.323+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94705963675616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562273530e80 con 0x7f48c005b040 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.324+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94705963675616] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.324+0000 7f48c9c3f800 20 remove_watcher() i=0 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.324+0000 7f48c9c3f800 2 removed watcher, disabling cache 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.325+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94705963681600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562273530e80 con 0x562273520d10 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.326+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94705963681600] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.326+0000 7f48c9c3f800 20 remove_watcher() i=1 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.326+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94705963683136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562273530e80 con 0x562273526d50 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.327+0000 7f48c7f3f640 1 -- 172.21.15.63:0/176707550 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94705963683136] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48c0029070 con 0x562273526d50 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.327+0000 7f48c9c3f800 20 remove_watcher() i=2 2024-04-23T19:10:28.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.327+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94705963687264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352cd40 con 0x7f48c005b040 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.328+0000 7f48c773e640 1 -- 172.21.15.63:0/176707550 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94705963687264] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b802b070 con 0x7f48c005b040 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.328+0000 7f48c9c3f800 20 remove_watcher() i=3 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.328+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94705963690944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562273530e80 con 0x562273520d10 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.329+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94705963690944] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x562273520d10 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.329+0000 7f48c9c3f800 20 remove_watcher() i=4 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.329+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94705963693664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352dd60 con 0x7f48b8017050 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.330+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94705963693664] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.330+0000 7f48c9c3f800 20 remove_watcher() i=5 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.330+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94705963695984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352cd40 con 0x7f48b8017050 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.331+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94705963695984] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.331+0000 7f48c9c3f800 20 remove_watcher() i=6 2024-04-23T19:10:28.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.331+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94705963698784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56227352ebc0 con 0x7f48b8017050 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94705963698784] v30'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48b005a040 con 0x7f48b8017050 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 20 remove_watcher() i=7 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f48b8017050 msgr2=0x7f48b800d8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f48b8017050 0x7f48b800d8e0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562273520d10 msgr2=0x5622735231a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562273520d10 0x5622735231a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48c005b040 msgr2=0x7f48c0046830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48c005b040 0x7f48c0046830 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562273526d50 msgr2=0x5622735291a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x562273526d50 0x5622735291a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 msgr2=0x7f4790039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f48b8000fd0 tx=0x7f48b8044610 comp rx=0 tx=0).stop 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c6f3d640 1 -- 172.21.15.63:0/176707550 reap_dead start 2024-04-23T19:10:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 msgr2=0x5622735150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622735150c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f48c0012c10 tx=0x7f48c004cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 shutdown_connections 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/176707550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733ea1b0 0x5622735150c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 >> 172.21.15.63:0/176707550 conn(0x562273507bd0 msgr2=0x562273505280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 shutdown_connections 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/176707550 wait complete. 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47a0036e60 msgr2=0x7f47a0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47a0036e60 0x7f47a0039320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f48b804b350 tx=0x7f48b8041040 comp rx=0 tx=0).stop 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622734fe5a0 msgr2=0x5622734fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622734fe5a0 0x5622734fe9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5622734e3bd0 tx=0x7f48c00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 shutdown_connections 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47a0036e60 0x7f47a0039320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/515481622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622734fe5a0 0x5622734fe9c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.332+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 >> 172.21.15.63:0/515481622 conn(0x5622734f2640 msgr2=0x5622734f2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 shutdown_connections 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/515481622 wait complete. 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5622734ed810 msgr2=0x5622734efc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5622734ed810 0x5622734efc00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5622733ee6a0 msgr2=0x5622734ea100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5622733ee6a0 0x5622734ea100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47ac036e60 msgr2=0x7f47ac039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47ac036e60 0x7f47ac039320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f48b0054000 tx=0x7f48b004e000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 msgr2=0x5622734e2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734e2ec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c010 tx=0x7f48b8000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 shutdown_connections 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5622734ed810 0x5622734efc00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5622733ee6a0 0x5622734ea100 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f47ac036e60 0x7f47ac039320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 --2- 172.21.15.63:0/109575324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622733eab90 0x5622734e2ec0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 >> 172.21.15.63:0/109575324 conn(0x5622733e4600 msgr2=0x5622734d54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 shutdown_connections 2024-04-23T19:10:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.333+0000 7f48c9c3f800 1 -- 172.21.15.63:0/109575324 wait complete. 2024-04-23T19:10:28.347 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-23T19:10:28.195295Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:28.347 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-23T19:10:28.347 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-23T19:10:28.347 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-04-23T19:10:28.393 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.393 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.393+0000 7fd326e87800 1 Processor -- start 2024-04-23T19:10:28.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.393+0000 7fd326e87800 1 -- start start 2024-04-23T19:10:28.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.393+0000 7fd326e87800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558800f3a4a0 0x558800f3c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.394+0000 7fd326e87800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558800de97f0 con 0x558800f36010 2024-04-23T19:10:28.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.394+0000 7fd326e87800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x558800dd8e20 con 0x558800f3a4a0 2024-04-23T19:10:28.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.394+0000 7fd324185640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558800f3a4a0 0x558800f3c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.394+0000 7fd324986640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x558800f36010 0x558800f36430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38642/0 (socket says 172.21.15.63:38642) 2024-04-23T19:10:28.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.394+0000 7fd324986640 1 -- 172.21.15.63:0/233670938 learned_addr learned my addr 172.21.15.63:0/233670938 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:28.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.394+0000 7fd324185640 1 -- 172.21.15.63:0/233670938 >> v1:172.21.15.16:6789/0 conn(0x558800f36010 legacy=0x558800f36430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.394+0000 7fd324185640 1 -- 172.21.15.63:0/233670938 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558800df7b50 con 0x558800f3a4a0 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.396+0000 7fd324185640 1 --2- 172.21.15.63:0/233670938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f3a4a0 0x558800f3c890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd31800a470 tx=0x7fd31801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f077c30f4b7319c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.396+0000 7fd323984640 1 -- 172.21.15.63:0/233670938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd318057070 con 0x558800f3a4a0 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.396+0000 7fd323984640 1 -- 172.21.15.63:0/233670938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd318052570 con 0x558800f3a4a0 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.396+0000 7fd323984640 1 -- 172.21.15.63:0/233670938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd318052870 con 0x558800f3a4a0 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.397+0000 7fd326e87800 1 -- 172.21.15.63:0/233670938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f3a4a0 msgr2=0x558800f3c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.397+0000 7fd326e87800 1 --2- 172.21.15.63:0/233670938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f3a4a0 0x558800f3c890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd31800a470 tx=0x7fd31801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.397+0000 7fd326e87800 1 -- 172.21.15.63:0/233670938 shutdown_connections 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.397+0000 7fd326e87800 1 --2- 172.21.15.63:0/233670938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f3a4a0 0x558800f3c890 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.397+0000 7fd326e87800 1 -- 172.21.15.63:0/233670938 >> 172.21.15.63:0/233670938 conn(0x558800f31630 msgr2=0x558800f33a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.397+0000 7fd326e87800 1 -- 172.21.15.63:0/233670938 shutdown_connections 2024-04-23T19:10:28.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.397+0000 7fd326e87800 1 -- 172.21.15.63:0/233670938 wait complete. 2024-04-23T19:10:28.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.409+0000 7fd326e87800 1 Processor -- start 2024-04-23T19:10:28.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.409+0000 7fd326e87800 1 -- start start 2024-04-23T19:10:28.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd326e87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558800eaa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd326e87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558800e826f0 con 0x558800f36010 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd324986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558800eaa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd324986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558800eaa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53776/0 (socket says 172.21.15.63:53776) 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd324986640 1 -- 172.21.15.63:0/4045633696 learned_addr learned my addr 172.21.15.63:0/4045633696 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd324986640 1 -- 172.21.15.63:0/4045633696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880102b050 con 0x558800f36010 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd324986640 1 --2- 172.21.15.63:0/4045633696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558800eaa0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd314001d80 tx=0x7fd314001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c9739050606745 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd212e0d640 1 -- 172.21.15.63:0/4045633696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd314013030 con 0x558800f36010 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.410+0000 7fd212e0d640 1 -- 172.21.15.63:0/4045633696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31400e470 con 0x558800f36010 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd212e0d640 1 -- 172.21.15.63:0/4045633696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd31400e810 con 0x558800f36010 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd326e87800 1 -- 172.21.15.63:0/4045633696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 msgr2=0x558800eaa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd326e87800 1 --2- 172.21.15.63:0/4045633696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558800eaa0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd314001d80 tx=0x7fd314001e70 comp rx=0 tx=0).stop 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd326e87800 1 -- 172.21.15.63:0/4045633696 shutdown_connections 2024-04-23T19:10:28.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd326e87800 1 --2- 172.21.15.63:0/4045633696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558800eaa0e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd326e87800 1 -- 172.21.15.63:0/4045633696 >> 172.21.15.63:0/4045633696 conn(0x558800f31630 msgr2=0x558801022b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd326e87800 1 -- 172.21.15.63:0/4045633696 shutdown_connections 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.411+0000 7fd326e87800 1 -- 172.21.15.63:0/4045633696 wait complete. 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd326e87800 1 Processor -- start 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd326e87800 1 -- start start 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd326e87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558801028940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd326e87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558800f3d160 con 0x558800f36010 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd324986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558801028940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd324986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558801028940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53780/0 (socket says 172.21.15.63:53780) 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd324986640 1 -- 172.21.15.63:0/32391087 learned_addr learned my addr 172.21.15.63:0/32391087 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd324986640 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558801027030 con 0x558800f36010 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.412+0000 7fd324986640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558801028940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd314004000 tx=0x7fd314000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.413+0000 7fd21160a640 1 -- 172.21.15.63:0/32391087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd314027070 con 0x558800f36010 2024-04-23T19:10:28.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.413+0000 7fd21160a640 1 -- 172.21.15.63:0/32391087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd314001400 con 0x558800f36010 2024-04-23T19:10:28.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.413+0000 7fd21160a640 1 -- 172.21.15.63:0/32391087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd314019a70 con 0x558800f36010 2024-04-23T19:10:28.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.413+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588010272c0 con 0x558800f36010 2024-04-23T19:10:28.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.413+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588010277e0 con 0x558800f36010 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.413+0000 7fd21160a640 1 -- 172.21.15.63:0/32391087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd31402a070 con 0x558800f36010 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd21160a640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd20c036e60 0x7fd20c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd21160a640 1 -- 172.21.15.63:0/32391087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd31401f070 con 0x558800f36010 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd324185640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd20c036e60 0x7fd20c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558800f3a4a0 0x558801039350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588010398e0 con 0x558800f3a4a0 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd325187640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558800f3a4a0 0x558801039350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd324185640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd20c036e60 0x7fd20c039320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd318054000 tx=0x7fd31804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd325187640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558800f3a4a0 0x558801039350 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd325187640 1 -- 172.21.15.63:0/32391087 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd31c00a4a0 con 0x558800f3a4a0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.414+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801027b70 con 0x558800f3a4a0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.415+0000 7fd325187640 1 -- 172.21.15.63:0/32391087 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd31c00a4a0 con 0x558800f3a4a0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.415+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55880103b720 con 0x558800f3a4a0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.415+0000 7fd325187640 1 -- 172.21.15.63:0/32391087 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd31c00a4a0 con 0x558800f3a4a0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.415+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55880103aa60 con 0x558800f3a4a0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.415+0000 7fd325187640 1 -- 172.21.15.63:0/32391087 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd31c00a4a0 con 0x558800f3a4a0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.415+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55880103cad0 0x55880103eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.415+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55880103f440 con 0x55880103cad0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.416+0000 7fd324986640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55880103cad0 0x55880103eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.416+0000 7fd324986640 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55880103cad0 0x55880103eec0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.416+0000 7fd324986640 1 -- 172.21.15.63:0/32391087 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd314029030 con 0x55880103cad0 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd326e87800 1 Processor -- start 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd326e87800 1 -- start start 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd326e87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880104d7f0 0x55880104dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd326e87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558800e28e30 con 0x55880104d7f0 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd325187640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880104d7f0 0x55880104dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd325187640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880104d7f0 0x55880104dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53792/0 (socket says 172.21.15.63:53792) 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd325187640 1 -- 172.21.15.63:0/2090096509 learned_addr learned my addr 172.21.15.63:0/2090096509 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.417+0000 7fd325187640 1 -- 172.21.15.63:0/2090096509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880104e150 con 0x55880104d7f0 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.418+0000 7fd325187640 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880104d7f0 0x55880104dc10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x558801028680 tx=0x7fd31c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.418+0000 7fd20a7fc640 1 -- 172.21.15.63:0/2090096509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd31c013440 con 0x55880104d7f0 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.418+0000 7fd20a7fc640 1 -- 172.21.15.63:0/2090096509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c0135e0 con 0x55880104d7f0 2024-04-23T19:10:28.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.418+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55880104ebc0 con 0x55880104d7f0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.418+0000 7fd20a7fc640 1 -- 172.21.15.63:0/2090096509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd31c0138c0 con 0x55880104d7f0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.418+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880104f0c0 con 0x55880104d7f0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.418+0000 7fd20a7fc640 1 -- 172.21.15.63:0/2090096509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd31c011040 con 0x55880104d7f0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd20a7fc640 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1fc036e60 0x7fd1fc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd20a7fc640 1 -- 172.21.15.63:0/2090096509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd31c01d070 con 0x55880104d7f0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd324986640 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1fc036e60 0x7fd1fc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd326e87800 1 Processor -- start 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd324986640 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1fc036e60 0x7fd1fc039320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55880103f820 tx=0x7fd314017f30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd326e87800 1 -- start start 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd326e87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x55880105af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd326e87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55880102b3a0 con 0x55880105ab10 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd325187640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x55880105af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd325187640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x55880105af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53808/0 (socket says 172.21.15.63:53808) 2024-04-23T19:10:28.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.419+0000 7fd325187640 1 -- 172.21.15.63:0/1792812127 learned_addr learned my addr 172.21.15.63:0/1792812127 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd325187640 1 -- 172.21.15.63:0/1792812127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880105b500 con 0x55880105ab10 2024-04-23T19:10:28.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd325187640 1 --2- 172.21.15.63:0/1792812127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x55880105af30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd31c04cce0 tx=0x7fd31c001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bcaf6b1e502aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd1fbfff640 1 -- 172.21.15.63:0/1792812127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd31c012cc0 con 0x55880105ab10 2024-04-23T19:10:28.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd1fbfff640 1 -- 172.21.15.63:0/1792812127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c001310 con 0x55880105ab10 2024-04-23T19:10:28.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd1fbfff640 1 -- 172.21.15.63:0/1792812127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd31c012e60 con 0x55880105ab10 2024-04-23T19:10:28.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd326e87800 1 -- 172.21.15.63:0/1792812127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 msgr2=0x55880105af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd326e87800 1 --2- 172.21.15.63:0/1792812127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x55880105af30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd31c04cce0 tx=0x7fd31c001f90 comp rx=0 tx=0).stop 2024-04-23T19:10:28.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd326e87800 1 -- 172.21.15.63:0/1792812127 shutdown_connections 2024-04-23T19:10:28.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd326e87800 1 --2- 172.21.15.63:0/1792812127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x55880105af30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd326e87800 1 -- 172.21.15.63:0/1792812127 >> 172.21.15.63:0/1792812127 conn(0x558801056e70 msgr2=0x558801054ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd326e87800 1 -- 172.21.15.63:0/1792812127 shutdown_connections 2024-04-23T19:10:28.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.420+0000 7fd326e87800 1 -- 172.21.15.63:0/1792812127 wait complete. 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd326e87800 1 Processor -- start 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd326e87800 1 -- start start 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd326e87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x558801062100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd326e87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55880105bc70 con 0x55880105ab10 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd325187640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x558801062100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd325187640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x558801062100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53824/0 (socket says 172.21.15.63:53824) 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 learned_addr learned my addr 172.21.15.63:0/4271227727 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558801062640 con 0x55880105ab10 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.421+0000 7fd325187640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x558801062100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd31c001df0 tx=0x7fd31c04cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.422+0000 7fd1fa7fc640 1 -- 172.21.15.63:0/4271227727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd31c021c60 con 0x55880105ab10 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.422+0000 7fd1fa7fc640 1 -- 172.21.15.63:0/4271227727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c021e00 con 0x55880105ab10 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.422+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588010628d0 con 0x55880105ab10 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.422+0000 7fd1fa7fc640 1 -- 172.21.15.63:0/4271227727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd31c02a350 con 0x55880105ab10 2024-04-23T19:10:28.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.422+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558801065580 con 0x55880105ab10 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd1fa7fc640 1 -- 172.21.15.63:0/4271227727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd31c040040 con 0x55880105ab10 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd1fa7fc640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1f0036e60 0x7fd1f0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd1fa7fc640 1 -- 172.21.15.63:0/4271227727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd31c033030 con 0x55880105ab10 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd324986640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1f0036e60 0x7fd1f0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd324986640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1f0036e60 0x7fd1f0039320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd314044730 tx=0x7fd314035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55880106dbf0 0x5588010700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801070800 con 0x55880106dbf0 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.423+0000 7fd324185640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55880106dbf0 0x5588010700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd324185640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55880106dbf0 0x5588010700b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 20 realm 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801070bd0 con 0x55880106dbf0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588010714a0 con 0x55880106dbf0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.424+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801071840 con 0x55880106dbf0 2024-04-23T19:10:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd326e87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801072090 con 0x55880106dbf0 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd326e87800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd326e87800 20 searching for the correct realm 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558801073e00 0x5588010761f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd325187640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558801073e00 0x5588010761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.425+0000 7fd325187640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558801073e00 0x5588010761f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.426+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.426+0000 7fd325187640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd31c0123a0 0x7fd31c022630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.426+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31c0123a0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.426+0000 7fd324986640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd31c0123a0 0x7fd31c022630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.426+0000 7fd324986640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd31c0123a0 0x7fd31c022630 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.427+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.427+0000 7fd324986640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd31402c060 0x7fd314040620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.427+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.427+0000 7fd324185640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd31402c060 0x7fd314040620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.427+0000 7fd324185640 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd31402c060 0x7fd314040620 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.427+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.427+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.428+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31c0123a0 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.429+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.429+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.429+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.429+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.429+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.429+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31c0123a0 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.430+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.431+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.432+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.432+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.432+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.432+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.432+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.432+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31c0123a0 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.433+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31c0123a0 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x558801073e00 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31402c060 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.434+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x55880106dbf0 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558801076780 con 0x7fd31c0123a0 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801072a10 con 0x55880106dbf0 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588010732b0 con 0x55880106dbf0 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.435+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588010732b0 con 0x7fd31402c060 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 zone default found 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 4 Realm: () 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 zonegroup default 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801076c00 con 0x7fd31402c060 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801073030 con 0x558801073e00 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.436+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801078fe0 con 0x558801073e00 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.437+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.437+0000 7fd326e87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.437+0000 7fd326e87800 20 started sync module instance, tier type = 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.437+0000 7fd326e87800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.437+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558801077a80 con 0x7fd31c0123a0 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.438+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.438+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94042621179344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558801079380 con 0x7fd31c0123a0 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.438+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558801079bd0 con 0x55880106dbf0 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.439+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.439+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94042621188352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107a3b0 con 0x55880106dbf0 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.439+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94042621179344] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.439+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558801077ec0 con 0x558801073e00 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.440+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94042621188352] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.440+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.440+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94042621189632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107ae50 con 0x558801073e00 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.440+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107b810 con 0x7fd31c0123a0 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.441+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.441+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94042621189632] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.441+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94042621192608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107adc0 con 0x7fd31c0123a0 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.441+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107c680 con 0x55880106dbf0 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.442+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94042621192608] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.442+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.442+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94042621197232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107c680 con 0x55880106dbf0 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.442+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107d090 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.443+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.443+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94042621201376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107d000 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.443+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107dff0 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.443+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94042621197232] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.444+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94042621201376] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.444+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.444+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94042621203584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107dad0 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.444+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107e4a0 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94042621203584] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94042621198800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107e4a0 con 0x7fd31402c060 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 20 add_watcher() i=0 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 20 add_watcher() i=1 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 20 add_watcher() i=2 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 20 add_watcher() i=3 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 20 add_watcher() i=4 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 20 add_watcher() i=5 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.445+0000 7fd326e87800 20 add_watcher() i=6 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.446+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94042621198800] v30'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.446+0000 7fd326e87800 20 add_watcher() i=7 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.446+0000 7fd326e87800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.446+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558801081820 con 0x55880106dbf0 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.446+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.447+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94042621206880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55880107ea30 con 0x55880106dbf0 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.447+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94042621206880] v30'82 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd1dafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 5 note: GC not initialized 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd1d9fe3640 20 reqs_thread_entry: start 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 20 Filter name: none 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd14b7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.448+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588010b4ce0 con 0x55880106dbf0 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.449+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.449+0000 7fd326e87800 20 rados_obj.operate() r=0 bl.length=673 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.449+0000 7fd326e87800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.449+0000 7fd326e87800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.449+0000 7fd326e87800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.449+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010bb5f0 con 0x55880106dbf0 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~582] v30'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd326e87800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd326e87800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd326e87800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94042621456480 in=770b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588010ba180 con 0x55880106dbf0 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94042621197232 notify 128849018888 ret 0) ==== 800+0+0 (crc 0 0 0) 0x558800df7b50 con 0x55880106dbf0 2024-04-23T19:10:28.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd1fbfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018888 cookie 94042621197232 notifier 4401 bl.length()=758 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd1fbfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd1fbfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd1fbfff640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fd1e8003ea0 con 0x55880106dbf0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(64 notify.4 [notify cookie 94042621456480 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.450+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.451+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94042621456480 notify 128849018888 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fd31805dd50 con 0x55880106dbf0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.451+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010bb060 con 0x55880106dbf0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(66 foo:foo1 [delete] v30'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd326e87800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94042621456480 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x7fd31c0123a0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== watch-notify(notify (1) cookie 94042621179344 notify 128849018884 ret 0) ==== 201+0+0 (crc 0 0 0) 0x5588010277e0 con 0x7fd31c0123a0 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd1fb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018884 cookie 94042621179344 notifier 4401 bl.length()=159 2024-04-23T19:10:28.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.0 [notify cookie 94042621456480 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd1fb7fe640 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fd1580032c0 con 0x7fd31c0123a0 2024-04-23T19:10:28.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.452+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== watch-notify(notify_complete (2) cookie 94042621456480 notify 128849018884 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5588010272c0 con 0x7fd31c0123a0 2024-04-23T19:10:28.465 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-23T19:10:28.195295Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:28.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.453+0000 7fd1dafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.453+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94042621206880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x55880106dbf0 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.454+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94042621206880] v30'83 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.454+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94042621179344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x7fd31c0123a0 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.455+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94042621179344] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.455+0000 7fd326e87800 20 remove_watcher() i=0 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.455+0000 7fd326e87800 2 removed watcher, disabling cache 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.455+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94042621188352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x55880106dbf0 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.456+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94042621188352] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.456+0000 7fd326e87800 20 remove_watcher() i=1 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.456+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94042621189632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x558801073e00 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.457+0000 7fd325187640 1 -- 172.21.15.63:0/4271227727 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94042621189632] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31c029070 con 0x558801073e00 2024-04-23T19:10:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.457+0000 7fd326e87800 20 remove_watcher() i=2 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.457+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94042621192608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x7fd31c0123a0 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.458+0000 7fd324986640 1 -- 172.21.15.63:0/4271227727 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94042621192608] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd314029030 con 0x7fd31c0123a0 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.458+0000 7fd326e87800 20 remove_watcher() i=3 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.458+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94042621197232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x55880106dbf0 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.459+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94042621197232] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x55880106dbf0 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.459+0000 7fd326e87800 20 remove_watcher() i=4 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.459+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94042621201376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x7fd31402c060 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.461+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94042621201376] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.461+0000 7fd326e87800 20 remove_watcher() i=5 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.461+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94042621203584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x7fd31402c060 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.461+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94042621203584] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.461+0000 7fd326e87800 20 remove_watcher() i=6 2024-04-23T19:10:28.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.461+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94042621198800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588010b9cd0 con 0x7fd31402c060 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.462+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94042621198800] v30'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31805a040 con 0x7fd31402c060 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.462+0000 7fd326e87800 20 remove_watcher() i=7 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd31402c060 msgr2=0x7fd314040620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd31402c060 0x7fd314040620 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55880106dbf0 msgr2=0x5588010700b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55880106dbf0 0x5588010700b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd31c0123a0 msgr2=0x7fd31c022630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd31c0123a0 0x7fd31c022630 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558801073e00 msgr2=0x5588010761f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558801073e00 0x5588010761f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1f0036e60 msgr2=0x7fd1f0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1f0036e60 0x7fd1f0039320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd314044730 tx=0x7fd314035000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 msgr2=0x558801062100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd324185640 1 -- 172.21.15.63:0/4271227727 reap_dead start 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x558801062100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd31c001df0 tx=0x7fd31c04cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 shutdown_connections 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/4271227727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880105ab10 0x558801062100 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 >> 172.21.15.63:0/4271227727 conn(0x558801056e70 msgr2=0x5588010544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 shutdown_connections 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/4271227727 wait complete. 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1fc036e60 msgr2=0x7fd1fc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1fc036e60 0x7fd1fc039320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55880103f820 tx=0x7fd314017f30 comp rx=0 tx=0).stop 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880104d7f0 msgr2=0x55880104dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880104d7f0 0x55880104dc10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x558801028680 tx=0x7fd31c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 shutdown_connections 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd1fc036e60 0x7fd1fc039320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 --2- 172.21.15.63:0/2090096509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55880104d7f0 0x55880104dc10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 >> 172.21.15.63:0/2090096509 conn(0x558801041890 msgr2=0x558801041c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 shutdown_connections 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.463+0000 7fd326e87800 1 -- 172.21.15.63:0/2090096509 wait complete. 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55880103cad0 msgr2=0x55880103eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55880103cad0 0x55880103eec0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558800f3a4a0 msgr2=0x558801039350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558800f3a4a0 0x558801039350 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd20c036e60 msgr2=0x7fd20c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd20c036e60 0x7fd20c039320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd318054000 tx=0x7fd31804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 msgr2=0x558801028940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558801028940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd314004000 tx=0x7fd314000bc0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 shutdown_connections 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55880103cad0 0x55880103eec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558800f3a4a0 0x558801039350 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd20c036e60 0x7fd20c039320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 --2- 172.21.15.63:0/32391087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558800f36010 0x558801028940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 >> 172.21.15.63:0/32391087 conn(0x558800f31630 msgr2=0x558801021360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 shutdown_connections 2024-04-23T19:10:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.464+0000 7fd326e87800 1 -- 172.21.15.63:0/32391087 wait complete. 2024-04-23T19:10:28.477 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-23T19:10:28.195295Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:28.477 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-23T19:10:28.477 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-23T19:10:28.477 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-04-23T19:10:28.521 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.521 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5ae146800 1 Processor -- start 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5ae146800 1 -- start start 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5ae146800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559837baca20 0x559837bace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5ae146800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559837a607f0 con 0x559837bad380 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5ae146800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x559837a4fe20 con 0x559837baca20 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5abc45640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559837baca20 0x559837bace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5abc45640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559837baca20 0x559837bace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53834/0 (socket says 172.21.15.63:53834) 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5ab444640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x559837bad380 0x559837bad830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38654/0 (socket says 172.21.15.63:38654) 2024-04-23T19:10:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.522+0000 7fd5abc45640 1 -- 172.21.15.63:0/767070980 learned_addr learned my addr 172.21.15.63:0/767070980 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.523+0000 7fd5abc45640 1 -- 172.21.15.63:0/767070980 >> v1:172.21.15.16:6789/0 conn(0x559837bad380 legacy=0x559837bad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.523+0000 7fd5abc45640 1 -- 172.21.15.63:0/767070980 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559837a6eb50 con 0x559837baca20 2024-04-23T19:10:28.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5abc45640 1 --2- 172.21.15.63:0/767070980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837bace40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00a520 tx=0x7fd59c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cd7b28ab6eaa7d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5aac43640 1 -- 172.21.15.63:0/767070980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c057070 con 0x559837baca20 2024-04-23T19:10:28.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5aac43640 1 -- 172.21.15.63:0/767070980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59c052570 con 0x559837baca20 2024-04-23T19:10:28.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5aac43640 1 -- 172.21.15.63:0/767070980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c052870 con 0x559837baca20 2024-04-23T19:10:28.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5ae146800 1 -- 172.21.15.63:0/767070980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 msgr2=0x559837bace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5ae146800 1 --2- 172.21.15.63:0/767070980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837bace40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00a520 tx=0x7fd59c01d010 comp rx=0 tx=0).stop 2024-04-23T19:10:28.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5ae146800 1 -- 172.21.15.63:0/767070980 shutdown_connections 2024-04-23T19:10:28.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5ae146800 1 --2- 172.21.15.63:0/767070980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837bace40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5ae146800 1 -- 172.21.15.63:0/767070980 >> 172.21.15.63:0/767070980 conn(0x559837ba8600 msgr2=0x559837baaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5ae146800 1 -- 172.21.15.63:0/767070980 shutdown_connections 2024-04-23T19:10:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.525+0000 7fd5ae146800 1 -- 172.21.15.63:0/767070980 wait complete. 2024-04-23T19:10:28.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.538+0000 7fd5ae146800 1 Processor -- start 2024-04-23T19:10:28.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.538+0000 7fd5ae146800 1 -- start start 2024-04-23T19:10:28.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.538+0000 7fd5ae146800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.539+0000 7fd5ae146800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559837af96f0 con 0x559837baca20 2024-04-23T19:10:28.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.539+0000 7fd5abc45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.539+0000 7fd5abc45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53840/0 (socket says 172.21.15.63:53840) 2024-04-23T19:10:28.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.539+0000 7fd5abc45640 1 -- 172.21.15.63:0/2553169056 learned_addr learned my addr 172.21.15.63:0/2553169056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.539+0000 7fd5abc45640 1 -- 172.21.15.63:0/2553169056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559837ca1bb0 con 0x559837baca20 2024-04-23T19:10:28.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.539+0000 7fd5abc45640 1 --2- 172.21.15.63:0/2553169056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca15e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd59c01d050 tx=0x7fd59c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaed4e5b57b43578 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd499e0b640 1 -- 172.21.15.63:0/2553169056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c057040 con 0x559837baca20 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd499e0b640 1 -- 172.21.15.63:0/2553169056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59c052850 con 0x559837baca20 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd499e0b640 1 -- 172.21.15.63:0/2553169056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c052b70 con 0x559837baca20 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd5ae146800 1 -- 172.21.15.63:0/2553169056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 msgr2=0x559837ca15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd5ae146800 1 --2- 172.21.15.63:0/2553169056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca15e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd59c01d050 tx=0x7fd59c04c000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd5ae146800 1 -- 172.21.15.63:0/2553169056 shutdown_connections 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd5ae146800 1 --2- 172.21.15.63:0/2553169056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca15e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd5ae146800 1 -- 172.21.15.63:0/2553169056 >> 172.21.15.63:0/2553169056 conn(0x559837ba8600 msgr2=0x559837c9cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd5ae146800 1 -- 172.21.15.63:0/2553169056 shutdown_connections 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.540+0000 7fd5ae146800 1 -- 172.21.15.63:0/2553169056 wait complete. 2024-04-23T19:10:28.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5ae146800 1 Processor -- start 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5ae146800 1 -- start start 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5ae146800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5ae146800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559837bae1e0 con 0x559837baca20 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5abc45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5abc45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53856/0 (socket says 172.21.15.63:53856) 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5abc45640 1 -- 172.21.15.63:0/1430349069 learned_addr learned my addr 172.21.15.63:0/1430349069 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5abc45640 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559837ca9630 con 0x559837baca20 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.541+0000 7fd5abc45640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca90f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd59c052820 tx=0x7fd59c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.542+0000 7fd493fff640 1 -- 172.21.15.63:0/1430349069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c073070 con 0x559837baca20 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.542+0000 7fd493fff640 1 -- 172.21.15.63:0/1430349069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59c05d570 con 0x559837baca20 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.542+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559837ca98c0 con 0x559837baca20 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.542+0000 7fd493fff640 1 -- 172.21.15.63:0/1430349069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59c057070 con 0x559837baca20 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.542+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559837cac570 con 0x559837baca20 2024-04-23T19:10:28.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.542+0000 7fd493fff640 1 -- 172.21.15.63:0/1430349069 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd59c076070 con 0x559837baca20 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd493fff640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd494036e60 0x7fd494039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd493fff640 1 -- 172.21.15.63:0/1430349069 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd59c08e590 con 0x559837baca20 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd5ab444640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd494036e60 0x7fd494039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837bad380 0x559837cb0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cb0900 con 0x559837bad380 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd5ac446640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837bad380 0x559837cb0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd5ab444640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd494036e60 0x7fd494039320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0000f60 tx=0x7fd5a0001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.543+0000 7fd5ac446640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837bad380 0x559837cb0330 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.544+0000 7fd5ac446640 1 -- 172.21.15.63:0/1430349069 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837bad380 2024-04-23T19:10:28.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.544+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cad900 con 0x559837bad380 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.544+0000 7fd5ac446640 1 -- 172.21.15.63:0/1430349069 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837bad380 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.544+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cb2740 con 0x559837bad380 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.544+0000 7fd5ac446640 1 -- 172.21.15.63:0/1430349069 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837bad380 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.544+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cb1a80 con 0x559837bad380 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.545+0000 7fd5ac446640 1 -- 172.21.15.63:0/1430349069 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837bad380 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.545+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559837cb3af0 0x559837cb5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.545+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cb6460 con 0x559837cb3af0 2024-04-23T19:10:28.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.545+0000 7fd5abc45640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559837cb3af0 0x559837cb5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.545+0000 7fd5abc45640 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559837cb3af0 0x559837cb5ee0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.545+0000 7fd5abc45640 1 -- 172.21.15.63:0/1430349069 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd59c075030 con 0x559837cb3af0 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ae146800 1 Processor -- start 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ae146800 1 -- start start 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ae146800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cc4830 0x559837cc4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ae146800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559837a9fe30 con 0x559837cc4830 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ac446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cc4830 0x559837cc4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ac446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cc4830 0x559837cc4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53860/0 (socket says 172.21.15.63:53860) 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ac446640 1 -- 172.21.15.63:0/3483004092 learned_addr learned my addr 172.21.15.63:0/3483004092 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.546+0000 7fd5ac446640 1 -- 172.21.15.63:0/3483004092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559837cc5190 con 0x559837cc4830 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.547+0000 7fd5ac446640 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cc4830 0x559837cc4c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x559837cb0d00 tx=0x7fd5a40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.547+0000 7fd4917fa640 1 -- 172.21.15.63:0/3483004092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a4013440 con 0x559837cc4830 2024-04-23T19:10:28.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.547+0000 7fd4917fa640 1 -- 172.21.15.63:0/3483004092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a40135e0 con 0x559837cc4830 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.547+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559837cc5c00 con 0x559837cc4830 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.547+0000 7fd4917fa640 1 -- 172.21.15.63:0/3483004092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a40138c0 con 0x559837cc4830 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.547+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559837cc6100 con 0x559837cc4830 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.547+0000 7fd4917fa640 1 -- 172.21.15.63:0/3483004092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd5a4011040 con 0x559837cc4830 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd4917fa640 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd484036e60 0x7fd484039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd4917fa640 1 -- 172.21.15.63:0/3483004092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd5a401d070 con 0x559837cc4830 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5abc45640 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd484036e60 0x7fd484039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5ae146800 1 Processor -- start 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5ae146800 1 -- start start 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5abc45640 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd484036e60 0x7fd484039320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd59c006f60 tx=0x7fd59c01d010 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5ae146800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5ae146800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559837ca1e70 con 0x559837cd1b30 2024-04-23T19:10:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5ac446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5ac446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53874/0 (socket says 172.21.15.63:53874) 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.548+0000 7fd5ac446640 1 -- 172.21.15.63:0/4027526525 learned_addr learned my addr 172.21.15.63:0/4027526525 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.549+0000 7fd5ac446640 1 -- 172.21.15.63:0/4027526525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559837cd2520 con 0x559837cd1b30 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.549+0000 7fd5ac446640 1 --2- 172.21.15.63:0/4027526525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd1f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd5a404cd50 tx=0x7fd5a404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf2e26fb2189b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.549+0000 7fd48affd640 1 -- 172.21.15.63:0/4027526525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a4015560 con 0x559837cd1b30 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.549+0000 7fd48affd640 1 -- 172.21.15.63:0/4027526525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a402a350 con 0x559837cd1b30 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.549+0000 7fd48affd640 1 -- 172.21.15.63:0/4027526525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a4012ca0 con 0x559837cd1b30 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.549+0000 7fd5ae146800 1 -- 172.21.15.63:0/4027526525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 msgr2=0x559837cd1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.549+0000 7fd5ae146800 1 --2- 172.21.15.63:0/4027526525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd1f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd5a404cd50 tx=0x7fd5a404d000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 -- 172.21.15.63:0/4027526525 shutdown_connections 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 --2- 172.21.15.63:0/4027526525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd1f50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 -- 172.21.15.63:0/4027526525 >> 172.21.15.63:0/4027526525 conn(0x559837ccde90 msgr2=0x559837ccbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 -- 172.21.15.63:0/4027526525 shutdown_connections 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 -- 172.21.15.63:0/4027526525 wait complete. 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 Processor -- start 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 -- start start 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ae146800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559837cd2c90 con 0x559837cd1b30 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ac446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ac446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53886/0 (socket says 172.21.15.63:53886) 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 learned_addr learned my addr 172.21.15.63:0/654663783 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.550+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559837cd9600 con 0x559837cd1b30 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.551+0000 7fd5ac446640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd90c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd5a404cd50 tx=0x7fd5a404cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.551+0000 7fd4897fa640 1 -- 172.21.15.63:0/654663783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a4015950 con 0x559837cd1b30 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.551+0000 7fd4897fa640 1 -- 172.21.15.63:0/654663783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a402a8e0 con 0x559837cd1b30 2024-04-23T19:10:28.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.551+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559837cd9890 con 0x559837cd1b30 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.551+0000 7fd4897fa640 1 -- 172.21.15.63:0/654663783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a4013c20 con 0x559837cd1b30 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.551+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559837cdc540 con 0x559837cd1b30 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.551+0000 7fd4897fa640 1 -- 172.21.15.63:0/654663783 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd5a404eb20 con 0x559837cd1b30 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd4897fa640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd474036e60 0x7fd474039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd4897fa640 1 -- 172.21.15.63:0/654663783 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd5a4032070 con 0x559837cd1b30 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd5abc45640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd474036e60 0x7fd474039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd5abc45640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd474036e60 0x7fd474039320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x559837cb2710 tx=0x7fd59c08a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837ce4c10 0x559837ce70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837ce7820 con 0x559837ce4c10 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.552+0000 7fd5ab444640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837ce4c10 0x559837ce70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ab444640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837ce4c10 0x559837ce70d0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 20 realm 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837ce4060 con 0x559837ce4c10 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837ce8310 con 0x559837ce4c10 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.553+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837ce8be0 con 0x559837ce4c10 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ae146800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837ce8f80 con 0x559837ce4c10 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ae146800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ae146800 20 searching for the correct realm 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559837ceae60 0x559837ced290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ac446640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559837ceae60 0x559837ced290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.554+0000 7fd5ac446640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559837ceae60 0x559837ced290 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.555+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.555+0000 7fd5ac446640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5a4001540 0x7fd5a402b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.555+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd5a4001540 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.555+0000 7fd5abc45640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5a4001540 0x7fd5a402b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.555+0000 7fd5abc45640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5a4001540 0x7fd5a402b3a0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.556+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.556+0000 7fd5abc45640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd59c09d040 0x7fd59c08f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.556+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.556+0000 7fd5ab444640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd59c09d040 0x7fd59c08f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.556+0000 7fd5ab444640 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd59c09d040 0x7fd59c08f980 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.556+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.556+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.557+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd5a4001540 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.558+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd5a4001540 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.559+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.560+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd5a4001540 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.561+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.562+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.562+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.562+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.562+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.563+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.563+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd5a4001540 2024-04-23T19:10:28.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.563+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.563+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ceae60 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.563+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.563+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd59c09d040 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x559837ce4c10 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559837ced960 con 0x7fd5a4001540 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.564+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837ce9900 con 0x559837ce4c10 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cea1a0 con 0x559837ce4c10 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cedde0 con 0x7fd59c09d040 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 zone default found 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 4 Realm: () 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 zonegroup default 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.565+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837ce9e00 con 0x7fd59c09d040 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cefcf0 con 0x559837ceae60 2024-04-23T19:10:28.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cf0100 con 0x559837ceae60 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 20 started sync module instance, tier type = 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.566+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cef190 con 0x7fd5a4001540 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.567+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.567+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94112259700096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837ceedf0 con 0x7fd5a4001540 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.567+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1030 con 0x559837ce4c10 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.568+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94112259700096] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.569+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.569+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94112259705728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1450 con 0x559837ce4c10 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.569+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1e10 con 0x559837ceae60 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.570+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.570+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94112259709280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf2270 con 0x559837ceae60 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.570+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf2d90 con 0x7fd5a4001540 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.570+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94112259705728] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.571+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94112259709280] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.571+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.571+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94112259713120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1d80 con 0x7fd5a4001540 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.571+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf37a0 con 0x559837ce4c10 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.572+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.572+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94112259716304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf3ec0 con 0x559837ce4c10 2024-04-23T19:10:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.572+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf49e0 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.572+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94112259713120] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.573+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.573+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94112259720368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf49a0 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.573+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf5430 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.573+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94112259716304] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.574+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94112259720368] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.574+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.574+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94112259706784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf3930 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf58e0 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94112259724336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf5d00 con 0x7fd59c09d040 2024-04-23T19:10:28.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 20 add_watcher() i=0 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 20 add_watcher() i=1 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 20 add_watcher() i=2 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 20 add_watcher() i=3 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 20 add_watcher() i=4 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.575+0000 7fd5ae146800 20 add_watcher() i=5 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.576+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94112259706784] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.576+0000 7fd5ae146800 20 add_watcher() i=6 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.576+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94112259724336] v30'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.576+0000 7fd5ae146800 20 add_watcher() i=7 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.576+0000 7fd5ae146800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.576+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837cf5d00 con 0x559837ce4c10 2024-04-23T19:10:28.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.577+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.577+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94112259725392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf5c70 con 0x559837ce4c10 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94112259725392] v30'84 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd461fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 5 note: GC not initialized 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd460fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 20 Filter name: none 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd3da7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.578+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837d2c240 con 0x559837ce4c10 2024-04-23T19:10:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.580+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~582 out=582b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+705 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.580+0000 7fd5ae146800 20 rados_obj.operate() r=0 bl.length=582 2024-04-23T19:10:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.580+0000 7fd5ae146800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.580+0000 7fd5ae146800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.580+0000 7fd5ae146800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.580+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837d32550 con 0x559837ce4c10 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.581+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v30'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.581+0000 7fd5ae146800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.581+0000 7fd5ae146800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.581+0000 7fd5ae146800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.581+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94112259973056 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559837d2e090 con 0x559837ce4c10 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94112259716304 notify 128849018889 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7fd5a0018040 con 0x559837ce4c10 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd48affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018889 cookie 94112259716304 notifier 4408 bl.length()=732 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(64 notify.4 [notify cookie 94112259973056 out=8b] v0'0 uv75 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd48affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd48affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd48affd640 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fd470003ea0 con 0x559837ce4c10 2024-04-23T19:10:28.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.582+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94112259973056 notify 128849018889 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fd5a0009050 con 0x559837ce4c10 2024-04-23T19:10:28.594 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-23T19:10:28.195295Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:28.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.583+0000 7fd461fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.583+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94112259725392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837d30ea0 con 0x559837ce4c10 2024-04-23T19:10:28.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.584+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94112259725392] v30'85 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.584+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94112259700096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf5850 con 0x7fd5a4001540 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.585+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94112259700096] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.585+0000 7fd5ae146800 20 remove_watcher() i=0 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.585+0000 7fd5ae146800 2 removed watcher, disabling cache 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.585+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94112259705728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cef580 con 0x559837ce4c10 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.586+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94112259705728] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.586+0000 7fd5ae146800 20 remove_watcher() i=1 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.586+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94112259709280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cef580 con 0x559837ceae60 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.587+0000 7fd5ac446640 1 -- 172.21.15.63:0/654663783 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94112259709280] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a400a4a0 con 0x559837ceae60 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.587+0000 7fd5ae146800 20 remove_watcher() i=2 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.587+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94112259713120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1960 con 0x7fd5a4001540 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.588+0000 7fd5abc45640 1 -- 172.21.15.63:0/654663783 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94112259713120] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd59c075030 con 0x7fd5a4001540 2024-04-23T19:10:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.588+0000 7fd5ae146800 20 remove_watcher() i=3 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.588+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94112259716304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cef580 con 0x559837ce4c10 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.589+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94112259716304] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x559837ce4c10 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.589+0000 7fd5ae146800 20 remove_watcher() i=4 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.589+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94112259720368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1960 con 0x7fd59c09d040 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.590+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94112259720368] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.590+0000 7fd5ae146800 20 remove_watcher() i=5 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.590+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94112259706784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1960 con 0x7fd59c09d040 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.591+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94112259706784] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.591+0000 7fd5ae146800 20 remove_watcher() i=6 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.591+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94112259724336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559837cf1960 con 0x7fd59c09d040 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94112259724336] v30'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd5a000f610 con 0x7fd59c09d040 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 20 remove_watcher() i=7 2024-04-23T19:10:28.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd59c09d040 msgr2=0x7fd59c08f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd59c09d040 0x7fd59c08f980 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837ce4c10 msgr2=0x559837ce70d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837ce4c10 0x559837ce70d0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5a4001540 msgr2=0x7fd5a402b3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5a4001540 0x7fd5a402b3a0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559837ceae60 msgr2=0x559837ced290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559837ceae60 0x559837ced290 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd474036e60 msgr2=0x7fd474039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd474036e60 0x7fd474039320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x559837cb2710 tx=0x7fd59c08a000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 msgr2=0x559837cd90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ab444640 1 -- 172.21.15.63:0/654663783 reap_dead start 2024-04-23T19:10:28.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd90c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd5a404cd50 tx=0x7fd5a404cce0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 shutdown_connections 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 --2- 172.21.15.63:0/654663783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cd1b30 0x559837cd90c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 >> 172.21.15.63:0/654663783 conn(0x559837ccde90 msgr2=0x559837ccb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 shutdown_connections 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.592+0000 7fd5ae146800 1 -- 172.21.15.63:0/654663783 wait complete. 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd484036e60 msgr2=0x7fd484039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd484036e60 0x7fd484039320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd59c006f60 tx=0x7fd59c01d010 comp rx=0 tx=0).stop 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cc4830 msgr2=0x559837cc4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cc4830 0x559837cc4c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x559837cb0d00 tx=0x7fd5a40178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 shutdown_connections 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd484036e60 0x7fd484039320 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/3483004092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837cc4830 0x559837cc4c50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 >> 172.21.15.63:0/3483004092 conn(0x559837cb88d0 msgr2=0x559837cb8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 shutdown_connections 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/3483004092 wait complete. 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559837cb3af0 msgr2=0x559837cb5ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559837cb3af0 0x559837cb5ee0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837bad380 msgr2=0x559837cb0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837bad380 0x559837cb0330 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd494036e60 msgr2=0x7fd494039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd494036e60 0x7fd494039320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0000f60 tx=0x7fd5a0001370 comp rx=0 tx=0).stop 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 msgr2=0x559837ca90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca90f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd59c052820 tx=0x7fd59c003020 comp rx=0 tx=0).stop 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 shutdown_connections 2024-04-23T19:10:28.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559837cb3af0 0x559837cb5ee0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559837bad380 0x559837cb0330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd494036e60 0x7fd494039320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 --2- 172.21.15.63:0/1430349069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559837baca20 0x559837ca90f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 >> 172.21.15.63:0/1430349069 conn(0x559837ba8600 msgr2=0x559837c9b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 shutdown_connections 2024-04-23T19:10:28.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.593+0000 7fd5ae146800 1 -- 172.21.15.63:0/1430349069 wait complete. 2024-04-23T19:10:28.606 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-23T19:10:28.195295Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:28.606 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-23T19:10:28.606 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-23T19:10:28.606 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-04-23T19:10:28.654 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.654 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.654+0000 7f3e9e72d800 1 Processor -- start 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.654+0000 7f3e9e72d800 1 -- start start 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e9e72d800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55d452b926d0 0x55d452b94ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e9e72d800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d452a407f0 con 0x55d452b8ebc0 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e9e72d800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55d452a2fe20 con 0x55d452b926d0 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e99b7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55d452b926d0 0x55d452b94ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e9a37f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55d452b8ebc0 0x55d452b90fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38670/0 (socket says 172.21.15.63:38670) 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e9a37f640 1 -- 172.21.15.63:0/2484980819 learned_addr learned my addr 172.21.15.63:0/2484980819 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e99b7e640 1 -- 172.21.15.63:0/2484980819 >> v1:172.21.15.16:6789/0 conn(0x55d452b8ebc0 legacy=0x55d452b90fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.655+0000 7f3e99b7e640 1 -- 172.21.15.63:0/2484980819 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452a4eb50 con 0x55d452b926d0 2024-04-23T19:10:28.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.657+0000 7f3e99b7e640 1 --2- 172.21.15.63:0/2484980819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b926d0 0x55d452b94ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000a470 tx=0x7f3e9001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf99709e4732795 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.657+0000 7f3e9937d640 1 -- 172.21.15.63:0/2484980819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e90057070 con 0x55d452b926d0 2024-04-23T19:10:28.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.657+0000 7f3e9937d640 1 -- 172.21.15.63:0/2484980819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e90052570 con 0x55d452b926d0 2024-04-23T19:10:28.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.657+0000 7f3e9937d640 1 -- 172.21.15.63:0/2484980819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e90052870 con 0x55d452b926d0 2024-04-23T19:10:28.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.658+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2484980819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b926d0 msgr2=0x55d452b94ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.658+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2484980819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b926d0 0x55d452b94ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000a470 tx=0x7f3e9001ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:28.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.658+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2484980819 shutdown_connections 2024-04-23T19:10:28.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.658+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2484980819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b926d0 0x55d452b94ac0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.658+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2484980819 >> 172.21.15.63:0/2484980819 conn(0x55d452b88650 msgr2=0x55d452b8aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.658+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2484980819 shutdown_connections 2024-04-23T19:10:28.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.658+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2484980819 wait complete. 2024-04-23T19:10:28.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.670+0000 7f3e9e72d800 1 Processor -- start 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.670+0000 7f3e9e72d800 1 -- start start 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3e9e72d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452b8e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3e9e72d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d452ad9710 con 0x55d452b8ebc0 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3e9a37f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452b8e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3e9a37f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452b8e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53906/0 (socket says 172.21.15.63:53906) 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3e9a37f640 1 -- 172.21.15.63:0/787162843 learned_addr learned my addr 172.21.15.63:0/787162843 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3e9a37f640 1 -- 172.21.15.63:0/787162843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452b8cac0 con 0x55d452b8ebc0 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/787162843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452b8e500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c001d50 tx=0x7f3e8c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73c997f7a136680 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3d88608640 1 -- 172.21.15.63:0/787162843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c013030 con 0x55d452b8ebc0 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3d88608640 1 -- 172.21.15.63:0/787162843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8c00e470 con 0x55d452b8ebc0 2024-04-23T19:10:28.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.671+0000 7f3d88608640 1 -- 172.21.15.63:0/787162843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c00e810 con 0x55d452b8ebc0 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 -- 172.21.15.63:0/787162843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 msgr2=0x55d452b8e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/787162843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452b8e500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c001d50 tx=0x7f3e8c00b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 -- 172.21.15.63:0/787162843 shutdown_connections 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/787162843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452b8e500 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 -- 172.21.15.63:0/787162843 >> 172.21.15.63:0/787162843 conn(0x55d452b88650 msgr2=0x55d452c7ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 -- 172.21.15.63:0/787162843 shutdown_connections 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 -- 172.21.15.63:0/787162843 wait complete. 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.672+0000 7f3e9e72d800 1 Processor -- start 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9e72d800 1 -- start start 2024-04-23T19:10:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9e72d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452c890d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9e72d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d452b95300 con 0x55d452b8ebc0 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9a37f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452c890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9a37f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452c890d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53920/0 (socket says 172.21.15.63:53920) 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9a37f640 1 -- 172.21.15.63:0/2777372410 learned_addr learned my addr 172.21.15.63:0/2777372410 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9a37f640 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452c89610 con 0x55d452b8ebc0 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452c890d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55d452b8db30 tx=0x7f3e8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.673+0000 7f3d86e05640 1 -- 172.21.15.63:0/2777372410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c027070 con 0x55d452b8ebc0 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.674+0000 7f3d86e05640 1 -- 172.21.15.63:0/2777372410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8c019900 con 0x55d452b8ebc0 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.674+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d452c898a0 con 0x55d452b8ebc0 2024-04-23T19:10:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.674+0000 7f3d86e05640 1 -- 172.21.15.63:0/2777372410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e8c019c40 con 0x55d452b8ebc0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.674+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d452c8c550 con 0x55d452b8ebc0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.674+0000 7f3d86e05640 1 -- 172.21.15.63:0/2777372410 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3e8c02a070 con 0x55d452b8ebc0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.674+0000 7f3d86e05640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d80036e60 0x7f3d80039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3d86e05640 1 -- 172.21.15.63:0/2777372410 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f3e8c01f070 con 0x55d452b8ebc0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e99b7e640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d80036e60 0x7f3d80039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452b926d0 0x55d452c90310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452c908a0 con 0x55d452b926d0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452b926d0 0x55d452c90310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e99b7e640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d80036e60 0x7f3d80039320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e90054000 tx=0x7f3e9004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452b926d0 0x55d452c90310 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e9ab80640 1 -- 172.21.15.63:0/2777372410 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452b926d0 2024-04-23T19:10:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.675+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452c8d8e0 con 0x55d452b926d0 2024-04-23T19:10:28.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.676+0000 7f3e9ab80640 1 -- 172.21.15.63:0/2777372410 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452b926d0 2024-04-23T19:10:28.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.676+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452c926e0 con 0x55d452b926d0 2024-04-23T19:10:28.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.676+0000 7f3e9ab80640 1 -- 172.21.15.63:0/2777372410 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452b926d0 2024-04-23T19:10:28.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.676+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452c91a20 con 0x55d452b926d0 2024-04-23T19:10:28.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.676+0000 7f3e9ab80640 1 -- 172.21.15.63:0/2777372410 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452b926d0 2024-04-23T19:10:28.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.676+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d452c93a90 0x55d452c95e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.676+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452c96400 con 0x55d452c93a90 2024-04-23T19:10:28.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.677+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d452c93a90 0x55d452c95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.677+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d452c93a90 0x55d452c95e80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9a37f640 1 -- 172.21.15.63:0/2777372410 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3e8c029030 con 0x55d452c93a90 2024-04-23T19:10:28.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9e72d800 1 Processor -- start 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9e72d800 1 -- start start 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9e72d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452ca4800 0x55d452ca4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9e72d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d452a7fe30 con 0x55d452ca4800 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9ab80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452ca4800 0x55d452ca4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9ab80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452ca4800 0x55d452ca4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53928/0 (socket says 172.21.15.63:53928) 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.678+0000 7f3e9ab80640 1 -- 172.21.15.63:0/2582210103 learned_addr learned my addr 172.21.15.63:0/2582210103 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.679+0000 7f3e9ab80640 1 -- 172.21.15.63:0/2582210103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452ca5160 con 0x55d452ca4800 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.679+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452ca4800 0x55d452ca4c20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55d452c90ca0 tx=0x7f3e940178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.679+0000 7f3d7ffff640 1 -- 172.21.15.63:0/2582210103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94013440 con 0x55d452ca4800 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.679+0000 7f3d7ffff640 1 -- 172.21.15.63:0/2582210103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e940135e0 con 0x55d452ca4800 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.679+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d452ca5bd0 con 0x55d452ca4800 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.679+0000 7f3d7ffff640 1 -- 172.21.15.63:0/2582210103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e940138c0 con 0x55d452ca4800 2024-04-23T19:10:28.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.679+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d452ca60d0 con 0x55d452ca4800 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3d7ffff640 1 -- 172.21.15.63:0/2582210103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3e94011040 con 0x55d452ca4800 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3d7ffff640 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d74036e60 0x7f3d74039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3d7ffff640 1 -- 172.21.15.63:0/2582210103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f3e9401d070 con 0x55d452ca4800 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d74036e60 0x7f3d74039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3e9e72d800 1 Processor -- start 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3e9e72d800 1 -- start start 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d74036e60 0x7f3d74039320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55d452c967e0 tx=0x7f3e8c00ef30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3e9e72d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.680+0000 7f3e9e72d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d452b8ce40 con 0x55d452cb1b90 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3e9ab80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3e9ab80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53942/0 (socket says 172.21.15.63:53942) 2024-04-23T19:10:28.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3e9ab80640 1 -- 172.21.15.63:0/1257929292 learned_addr learned my addr 172.21.15.63:0/1257929292 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3e9ab80640 1 -- 172.21.15.63:0/1257929292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452cb2580 con 0x55d452cb1b90 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/1257929292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb1fb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e9404cd50 tx=0x7f3e9404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a4b734895d7faa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3d7dffb640 1 -- 172.21.15.63:0/1257929292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94015560 con 0x55d452cb1b90 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3d7dffb640 1 -- 172.21.15.63:0/1257929292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9402a350 con 0x55d452cb1b90 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3d7dffb640 1 -- 172.21.15.63:0/1257929292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94012ca0 con 0x55d452cb1b90 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3e9e72d800 1 -- 172.21.15.63:0/1257929292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 msgr2=0x55d452cb1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.681+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/1257929292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb1fb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e9404cd50 tx=0x7f3e9404d000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 -- 172.21.15.63:0/1257929292 shutdown_connections 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/1257929292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb1fb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 -- 172.21.15.63:0/1257929292 >> 172.21.15.63:0/1257929292 conn(0x55d452cade60 msgr2=0x55d452cabec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 -- 172.21.15.63:0/1257929292 shutdown_connections 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 -- 172.21.15.63:0/1257929292 wait complete. 2024-04-23T19:10:28.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 Processor -- start 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 -- start start 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9e72d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d452cb2d10 con 0x55d452cb1b90 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9ab80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9ab80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53946/0 (socket says 172.21.15.63:53946) 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.682+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 learned_addr learned my addr 172.21.15.63:0/3058396416 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.683+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d452cb9580 con 0x55d452cb1b90 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.683+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb9040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e9404d040 tx=0x7f3e9404cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.683+0000 7f3d6bfff640 1 -- 172.21.15.63:0/3058396416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e9402a8e0 con 0x55d452cb1b90 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.683+0000 7f3d6bfff640 1 -- 172.21.15.63:0/3058396416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e94013af0 con 0x55d452cb1b90 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.683+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d452cb9810 con 0x55d452cb1b90 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.683+0000 7f3d6bfff640 1 -- 172.21.15.63:0/3058396416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94013dd0 con 0x55d452cb1b90 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.683+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d452cbc4c0 con 0x55d452cb1b90 2024-04-23T19:10:28.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.684+0000 7f3d6bfff640 1 -- 172.21.15.63:0/3058396416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3e9402b3a0 con 0x55d452cb1b90 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.684+0000 7f3d6bfff640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d64036e10 0x7f3d640392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.684+0000 7f3d6bfff640 1 -- 172.21.15.63:0/3058396416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f3e94032070 con 0x55d452cb1b90 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.684+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d64036e10 0x7f3d640392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.684+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d64036e10 0x7f3d640392d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c017a60 tx=0x7f3e8c042000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.684+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.684+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452cc4b90 0x55d452cc7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cc77a0 con 0x55d452cc4b90 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e99b7e640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452cc4b90 0x55d452cc7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e99b7e640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452cc4b90 0x55d452cc7050 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 20 realm 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cc7b70 con 0x55d452cc4b90 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.685+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cc8440 con 0x55d452cc4b90 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cc87e0 con 0x55d452cc4b90 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cc9030 con 0x55d452cc4b90 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 20 searching for the correct realm 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d452ccada0 0x55d452ccd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.686+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d452ccada0 0x55d452ccd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.687+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d452ccada0 0x55d452ccd190 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.687+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.687+0000 7f3e9ab80640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3e9402c430 0x7f3e94045460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.687+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e9402c430 2024-04-23T19:10:28.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.687+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3e9402c430 0x7f3e94045460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.687+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3e9402c430 0x7f3e94045460 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.688+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.688+0000 7f3e9a37f640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3e8c02c060 0x7f3e8c04b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.688+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.688+0000 7f3e99b7e640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3e8c02c060 0x7f3e8c04b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.688+0000 7f3e99b7e640 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3e8c02c060 0x7f3e8c04b5d0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.689+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e9402c430 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.690+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e9402c430 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.691+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.692+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.693+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e9402c430 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.694+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.694+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.694+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.694+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.694+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.694+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e9402c430 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452ccada0 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.695+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e8c02c060 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x55d452cc4b90 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccd720 con 0x7f3e9402c430 2024-04-23T19:10:28.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9e72d800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9e72d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9e72d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9e72d800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9e72d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9e72d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.696+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cc99b0 con 0x55d452cc4b90 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e9e72d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cca250 con 0x55d452cc4b90 2024-04-23T19:10:28.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e9e72d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.697+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cca250 con 0x7f3e8c02c060 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 zone default found 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 4 Realm: () 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 zonegroup default 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccdba0 con 0x7f3e8c02c060 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cc9fd0 con 0x55d452ccada0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452ccfb80 con 0x55d452ccada0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 started sync module instance, tier type = 2024-04-23T19:10:28.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.698+0000 7f3e9e72d800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.699+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cceec0 con 0x7f3e9402c430 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.700+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.700+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94370410590896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd0490 con 0x7f3e9402c430 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.700+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd0ce0 con 0x55d452cc4b90 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.701+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94370410590896] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.701+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.701+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94370410594272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd0ce0 con 0x55d452cc4b90 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.701+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd17e0 con 0x55d452ccada0 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.702+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94370410594272] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.702+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.702+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94370410599120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd0ce0 con 0x55d452ccada0 2024-04-23T19:10:28.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.702+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd2270 con 0x7f3e9402c430 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.703+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94370410599120] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.703+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.703+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94370410596400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd0c50 con 0x7f3e9402c430 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.703+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd3130 con 0x55d452cc4b90 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.704+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.704+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94370410606176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd38b0 con 0x55d452cc4b90 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.704+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd43d0 con 0x7f3e8c02c060 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.704+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94370410596400] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.705+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.705+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94370410606176] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.705+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94370410610336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4390 con 0x7f3e8c02c060 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.705+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd3340 con 0x7f3e8c02c060 2024-04-23T19:10:28.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.706+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94370410610336] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.706+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.706+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94370410604128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4930 con 0x7f3e8c02c060 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.706+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd52f0 con 0x7f3e8c02c060 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94370410604128] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94370410607360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd52f0 con 0x7f3e8c02c060 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 20 add_watcher() i=0 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 20 add_watcher() i=1 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 20 add_watcher() i=2 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 20 add_watcher() i=3 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 20 add_watcher() i=4 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 20 add_watcher() i=5 2024-04-23T19:10:28.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.707+0000 7f3e9e72d800 20 add_watcher() i=6 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.708+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94370410607360] v30'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.708+0000 7f3e9e72d800 20 add_watcher() i=7 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.708+0000 7f3e9e72d800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.708+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd85e0 con 0x55d452cc4b90 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.709+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.709+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94370410614704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd5880 con 0x55d452cc4b90 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94370410614704] v30'86 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3d50fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 5 note: GC not initialized 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3ccb7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:28.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 20 Filter name: none 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3cc97fa640 20 reqs_thread_entry: start 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.710+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452d0ba90 con 0x55d452cc4b90 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.711+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.711+0000 7f3e9e72d800 20 rados_obj.operate() r=0 bl.length=556 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.711+0000 7f3e9e72d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.711+0000 7f3e9e72d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.711+0000 7f3e9e72d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.711+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452d11f30 con 0x55d452cc4b90 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v30'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3e9e72d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3e9e72d800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3e9e72d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94370410863008 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452d10ac0 con 0x55d452cc4b90 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94370410606176 notify 128849018890 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55d452a4eb50 con 0x55d452cc4b90 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3d7dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018890 cookie 94370410606176 notifier 4415 bl.length()=567 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3d7dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3d7dffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3d7dffb640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f3d60003ea0 con 0x55d452cc4b90 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.712+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(64 notify.4 [notify cookie 94370410863008 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.713+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.713+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94370410863008 notify 128849018890 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f3e9005dd10 con 0x55d452cc4b90 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.713+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452d119a0 con 0x7f3e9402c430 2024-04-23T19:10:28.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 foo:foo2 [delete] v30'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3e9e72d800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94370410863008 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d452d0d7c0 con 0x7f3e9402c430 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== watch-notify(notify (1) cookie 94370410590896 notify 128849018885 ret 0) ==== 201+0+0 (crc 0 0 0) 0x55d452c8c550 con 0x7f3e9402c430 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3d7d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018885 cookie 94370410590896 notifier 4415 bl.length()=159 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3d7d7fa640 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f3cd00032c0 con 0x7f3e9402c430 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [notify cookie 94370410863008 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.714+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.715+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== watch-notify(notify_complete (2) cookie 94370410863008 notify 128849018885 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d452c898a0 con 0x7f3e9402c430 2024-04-23T19:10:28.728 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:28.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.715+0000 7f3d50fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.715+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94370410614704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452d10ed0 con 0x55d452cc4b90 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.716+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94370410614704] v30'87 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.716+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94370410590896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4fb0 con 0x7f3e9402c430 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.717+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94370410590896] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.717+0000 7f3e9e72d800 20 remove_watcher() i=0 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.717+0000 7f3e9e72d800 2 removed watcher, disabling cache 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.717+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94370410594272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4fb0 con 0x55d452cc4b90 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.718+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94370410594272] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.719+0000 7f3e9e72d800 20 remove_watcher() i=1 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.719+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94370410599120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4fb0 con 0x55d452ccada0 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.719+0000 7f3e9ab80640 1 -- 172.21.15.63:0/3058396416 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94370410599120] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9400a4a0 con 0x55d452ccada0 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.719+0000 7f3e9e72d800 20 remove_watcher() i=2 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.719+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94370410596400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4fb0 con 0x7f3e9402c430 2024-04-23T19:10:28.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.720+0000 7f3e9a37f640 1 -- 172.21.15.63:0/3058396416 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94370410596400] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e8c03e070 con 0x7f3e9402c430 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.720+0000 7f3e9e72d800 20 remove_watcher() i=3 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.720+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94370410606176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4fb0 con 0x55d452cc4b90 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.721+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94370410606176] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x55d452cc4b90 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.721+0000 7f3e9e72d800 20 remove_watcher() i=4 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.721+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94370410610336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd2e60 con 0x7f3e8c02c060 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.722+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94370410610336] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.722+0000 7f3e9e72d800 20 remove_watcher() i=5 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.722+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94370410604128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd4fb0 con 0x7f3e8c02c060 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.723+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94370410604128] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.723+0000 7f3e9e72d800 20 remove_watcher() i=6 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.723+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94370410607360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d452cd3ea0 con 0x7f3e8c02c060 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94370410607360] v30'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3e9005a040 con 0x7f3e8c02c060 2024-04-23T19:10:28.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 20 remove_watcher() i=7 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3e8c02c060 msgr2=0x7f3e8c04b5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3e8c02c060 0x7f3e8c04b5d0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452cc4b90 msgr2=0x55d452cc7050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452cc4b90 0x55d452cc7050 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3e9402c430 msgr2=0x7f3e94045460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3e9402c430 0x7f3e94045460 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d452ccada0 msgr2=0x55d452ccd190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d452ccada0 0x55d452ccd190 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d64036e10 msgr2=0x7f3d640392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d64036e10 0x7f3d640392d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c017a60 tx=0x7f3e8c042000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 msgr2=0x55d452cb9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/3058396416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452cb1b90 0x55d452cb9040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e9404d040 tx=0x7f3e9404cce0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.724+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 reap_dead start 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e99b7e640 1 -- 172.21.15.63:0/3058396416 reap_dead start 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 shutdown_connections 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 >> 172.21.15.63:0/3058396416 conn(0x55d452cade60 msgr2=0x55d452cab4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 shutdown_connections 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/3058396416 wait complete. 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d74036e60 msgr2=0x7f3d74039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d74036e60 0x7f3d74039320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55d452c967e0 tx=0x7f3e8c00ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452ca4800 msgr2=0x55d452ca4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452ca4800 0x55d452ca4c20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55d452c90ca0 tx=0x7f3e940178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 shutdown_connections 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d74036e60 0x7f3d74039320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2582210103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452ca4800 0x55d452ca4c20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 >> 172.21.15.63:0/2582210103 conn(0x55d452c988a0 msgr2=0x55d452c98ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 shutdown_connections 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2582210103 wait complete. 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d452c93a90 msgr2=0x55d452c95e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d452c93a90 0x55d452c95e80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452b926d0 msgr2=0x55d452c90310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452b926d0 0x55d452c90310 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d80036e60 msgr2=0x7f3d80039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d80036e60 0x7f3d80039320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e90054000 tx=0x7f3e9004e000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 msgr2=0x55d452c890d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452c890d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55d452b8db30 tx=0x7f3e8c000bc0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 shutdown_connections 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d452c93a90 0x55d452c95e80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d452b926d0 0x55d452c90310 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3d80036e60 0x7f3d80039320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 --2- 172.21.15.63:0/2777372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d452b8ebc0 0x55d452c890d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.725+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 >> 172.21.15.63:0/2777372410 conn(0x55d452b88650 msgr2=0x55d452b8d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.726+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 shutdown_connections 2024-04-23T19:10:28.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.726+0000 7f3e9e72d800 1 -- 172.21.15.63:0/2777372410 wait complete. 2024-04-23T19:10:28.740 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:28.740 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-23T19:10:28.740 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-23T19:10:28.740 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-23T19:10:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.784 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff90ad21800 1 Processor -- start 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff90ad21800 1 -- start start 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff90ad21800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557843d7da20 0x557843d7de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff90ad21800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557843c317f0 con 0x557843d7e380 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff90ad21800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x557843c20e20 con 0x557843d7da20 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff906a59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557843d7da20 0x557843d7de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff906258640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x557843d7e380 0x557843d7e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38674/0 (socket says 172.21.15.63:38674) 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.785+0000 7ff906258640 1 -- 172.21.15.63:0/3522792941 learned_addr learned my addr 172.21.15.63:0/3522792941 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.786+0000 7ff905a57640 1 -- 172.21.15.63:0/3522792941 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736935303 0 0) 0x557843c317f0 con 0x557843d7e380 2024-04-23T19:10:28.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.786+0000 7ff905a57640 1 -- 172.21.15.63:0/3522792941 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8f0001e10 con 0x557843d7e380 2024-04-23T19:10:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.786+0000 7ff906a59640 1 -- 172.21.15.63:0/3522792941 >> v1:172.21.15.16:6789/0 conn(0x557843d7e380 legacy=0x557843d7e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.786+0000 7ff906a59640 1 -- 172.21.15.63:0/3522792941 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557843c3fb50 con 0x557843d7da20 2024-04-23T19:10:28.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.787+0000 7ff906a59640 1 --2- 172.21.15.63:0/3522792941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843d7de40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800a470 tx=0x7ff8f801cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec052ec1e0708e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff905a57640 1 -- 172.21.15.63:0/3522792941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f8057070 con 0x557843d7da20 2024-04-23T19:10:28.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff905a57640 1 -- 172.21.15.63:0/3522792941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f804acd0 con 0x557843d7da20 2024-04-23T19:10:28.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff905a57640 1 -- 172.21.15.63:0/3522792941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f8052440 con 0x557843d7da20 2024-04-23T19:10:28.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff90ad21800 1 -- 172.21.15.63:0/3522792941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 msgr2=0x557843d7de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3522792941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843d7de40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800a470 tx=0x7ff8f801cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff90ad21800 1 -- 172.21.15.63:0/3522792941 shutdown_connections 2024-04-23T19:10:28.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3522792941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843d7de40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff90ad21800 1 -- 172.21.15.63:0/3522792941 >> 172.21.15.63:0/3522792941 conn(0x557843d79600 msgr2=0x557843d7ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff90ad21800 1 -- 172.21.15.63:0/3522792941 shutdown_connections 2024-04-23T19:10:28.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.788+0000 7ff90ad21800 1 -- 172.21.15.63:0/3522792941 wait complete. 2024-04-23T19:10:28.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.801+0000 7ff90ad21800 1 Processor -- start 2024-04-23T19:10:28.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.801+0000 7ff90ad21800 1 -- start start 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.801+0000 7ff90ad21800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e72460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.801+0000 7ff90ad21800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557843cca6f0 con 0x557843d7da20 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff906a59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e72460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff906a59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e72460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53954/0 (socket says 172.21.15.63:53954) 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff906a59640 1 -- 172.21.15.63:0/3054344909 learned_addr learned my addr 172.21.15.63:0/3054344909 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff906a59640 1 -- 172.21.15.63:0/3054344909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557843e72a30 con 0x557843d7da20 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff906a59640 1 --2- 172.21.15.63:0/3054344909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e72460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8000c00 tx=0x7ff8f804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9978bc8da5659404 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff7f4e09640 1 -- 172.21.15.63:0/3054344909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f8057040 con 0x557843d7da20 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff7f4e09640 1 -- 172.21.15.63:0/3054344909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f8063070 con 0x557843d7da20 2024-04-23T19:10:28.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.802+0000 7ff7f4e09640 1 -- 172.21.15.63:0/3054344909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f8052760 con 0x557843d7da20 2024-04-23T19:10:28.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 -- 172.21.15.63:0/3054344909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 msgr2=0x557843e72460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3054344909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e72460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8000c00 tx=0x7ff8f804c000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 -- 172.21.15.63:0/3054344909 shutdown_connections 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3054344909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e72460 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 -- 172.21.15.63:0/3054344909 >> 172.21.15.63:0/3054344909 conn(0x557843d79600 msgr2=0x557843e6dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 -- 172.21.15.63:0/3054344909 shutdown_connections 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 -- 172.21.15.63:0/3054344909 wait complete. 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 Processor -- start 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.803+0000 7ff90ad21800 1 -- start start 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff90ad21800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e79fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff90ad21800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557843d7f1e0 con 0x557843d7da20 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff906a59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e79fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff906a59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e79fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53964/0 (socket says 172.21.15.63:53964) 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff906a59640 1 -- 172.21.15.63:0/1558266990 learned_addr learned my addr 172.21.15.63:0/1558266990 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff906a59640 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557843e7a520 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff906a59640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e79fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff8f801baa0 tx=0x7ff8f8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff7f3606640 1 -- 172.21.15.63:0/1558266990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f806d070 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff7f3606640 1 -- 172.21.15.63:0/1558266990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f8052c70 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff7f3606640 1 -- 172.21.15.63:0/1558266990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f8057050 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557843e7a7b0 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.804+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557843e7d410 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.805+0000 7ff7f3606640 1 -- 172.21.15.63:0/1558266990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff8f8070070 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.805+0000 7ff7f3606640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7ec036eb0 0x7ff7ec039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.805+0000 7ff7f3606640 1 -- 172.21.15.63:0/1558266990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff8f808e1b0 con 0x557843d7da20 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.805+0000 7ff906258640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7ec036eb0 0x7ff7ec039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.805+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843d7e380 0x557843e811d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.805+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843e817a0 con 0x557843d7e380 2024-04-23T19:10:28.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.806+0000 7ff90725a640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843d7e380 0x557843e811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.806+0000 7ff906258640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7ec036eb0 0x7ff7ec039370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc000f60 tx=0x7ff8fc0010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.806+0000 7ff90725a640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843d7e380 0x557843e811d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.806+0000 7ff90725a640 1 -- 172.21.15.63:0/1558266990 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff90000a4a0 con 0x557843d7e380 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.806+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843e7e780 con 0x557843d7e380 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.806+0000 7ff90725a640 1 -- 172.21.15.63:0/1558266990 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff90000a4a0 con 0x557843d7e380 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.806+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843e83610 con 0x557843d7e380 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.807+0000 7ff90725a640 1 -- 172.21.15.63:0/1558266990 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff90000a4a0 con 0x557843d7e380 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.807+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843e82950 con 0x557843d7e380 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.807+0000 7ff90725a640 1 -- 172.21.15.63:0/1558266990 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff90000a4a0 con 0x557843d7e380 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.807+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557843e849c0 0x557843e86db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.807+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843e872f0 con 0x557843e849c0 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.807+0000 7ff906a59640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557843e849c0 0x557843e86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.807+0000 7ff906a59640 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557843e849c0 0x557843e86db0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.808+0000 7ff906a59640 1 -- 172.21.15.63:0/1558266990 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff8f8079070 con 0x557843e849c0 2024-04-23T19:10:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.808+0000 7ff90ad21800 1 Processor -- start 2024-04-23T19:10:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.808+0000 7ff90ad21800 1 -- start start 2024-04-23T19:10:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.808+0000 7ff90ad21800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843e956f0 0x557843e95b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.808+0000 7ff90ad21800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557843c70e30 con 0x557843e956f0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff90725a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843e956f0 0x557843e95b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff90725a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843e956f0 0x557843e95b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53970/0 (socket says 172.21.15.63:53970) 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff90725a640 1 -- 172.21.15.63:0/3770116634 learned_addr learned my addr 172.21.15.63:0/3770116634 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff90725a640 1 -- 172.21.15.63:0/3770116634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557843e96050 con 0x557843e956f0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff90725a640 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843e956f0 0x557843e95b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x557843e81ba0 tx=0x7ff9000178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff7f0e01640 1 -- 172.21.15.63:0/3770116634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff900013440 con 0x557843e956f0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff7f0e01640 1 -- 172.21.15.63:0/3770116634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9000135e0 con 0x557843e956f0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557843e96ac0 con 0x557843e956f0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff7f0e01640 1 -- 172.21.15.63:0/3770116634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9000138c0 con 0x557843e956f0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.809+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557843e96fc0 con 0x557843e956f0 2024-04-23T19:10:28.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.810+0000 7ff7f0e01640 1 -- 172.21.15.63:0/3770116634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff900011040 con 0x557843e956f0 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.810+0000 7ff7f0e01640 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7e0036e60 0x7ff7e0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.810+0000 7ff7f0e01640 1 -- 172.21.15.63:0/3770116634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff90001d070 con 0x557843e956f0 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.810+0000 7ff906a59640 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7e0036e60 0x7ff7e0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.810+0000 7ff90ad21800 1 Processor -- start 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90ad21800 1 -- start start 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff906a59640 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7e0036e60 0x7ff7e0039320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x557843e82cf0 tx=0x7ff8f808c550 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90ad21800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90ad21800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557843e72cf0 con 0x557843ea2aa0 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90725a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90725a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53982/0 (socket says 172.21.15.63:53982) 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90725a640 1 -- 172.21.15.63:0/1459447489 learned_addr learned my addr 172.21.15.63:0/1459447489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90725a640 1 -- 172.21.15.63:0/1459447489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557843ea3490 con 0x557843ea2aa0 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff90725a640 1 --2- 172.21.15.63:0/1459447489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea2ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff9000013e0 tx=0x7ff900012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2528152643026d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff7ea7fc640 1 -- 172.21.15.63:0/1459447489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff900043070 con 0x557843ea2aa0 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff7ea7fc640 1 -- 172.21.15.63:0/1459447489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90002ac50 con 0x557843ea2aa0 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.811+0000 7ff7ea7fc640 1 -- 172.21.15.63:0/1459447489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9000014f0 con 0x557843ea2aa0 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 -- 172.21.15.63:0/1459447489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 msgr2=0x557843ea2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1459447489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea2ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff9000013e0 tx=0x7ff900012c90 comp rx=0 tx=0).stop 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 -- 172.21.15.63:0/1459447489 shutdown_connections 2024-04-23T19:10:28.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1459447489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea2ec0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 -- 172.21.15.63:0/1459447489 >> 172.21.15.63:0/1459447489 conn(0x557843e9ed70 msgr2=0x557843e9cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 -- 172.21.15.63:0/1459447489 shutdown_connections 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 -- 172.21.15.63:0/1459447489 wait complete. 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 Processor -- start 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 -- start start 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90ad21800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557843ea3bb0 con 0x557843ea2aa0 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90725a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90725a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:53992/0 (socket says 172.21.15.63:53992) 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.812+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 learned_addr learned my addr 172.21.15.63:0/1902157804 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.813+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557843eaa470 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.813+0000 7ff90725a640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea9f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff900001da0 tx=0x7ff90004cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.813+0000 7ff7e8ff9640 1 -- 172.21.15.63:0/1902157804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff900043040 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.813+0000 7ff7e8ff9640 1 -- 172.21.15.63:0/1902157804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff900030030 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.813+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557843eaa760 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.813+0000 7ff7e8ff9640 1 -- 172.21.15.63:0/1902157804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9000123a0 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.813+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557843ead3b0 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff7e8ff9640 1 -- 172.21.15.63:0/1902157804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff900012540 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff7e8ff9640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7d0036eb0 0x7ff7d0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff7e8ff9640 1 -- 172.21.15.63:0/1902157804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff90000e040 con 0x557843ea2aa0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff906a59640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7d0036eb0 0x7ff7d0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843eb5c90 0x557843eb8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843eb88a0 con 0x557843eb5c90 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff906258640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843eb5c90 0x557843eb8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.814+0000 7ff906a59640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7d0036eb0 0x7ff7d0039370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8f805c910 tx=0x7ff8f808f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff906258640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843eb5c90 0x557843eb8150 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8fc009050 con 0x557843eb5c90 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 20 realm 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843eb50e0 con 0x557843eb5c90 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8fc009050 con 0x557843eb5c90 2024-04-23T19:10:28.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843eb9380 con 0x557843eb5c90 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8fc009050 con 0x557843eb5c90 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.815+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843eb9c50 con 0x557843eb5c90 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff8fc009050 con 0x557843eb5c90 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90ad21800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843eb9ff0 con 0x557843eb5c90 2024-04-23T19:10:28.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff8fc009050 con 0x557843eb5c90 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90ad21800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90ad21800 20 searching for the correct realm 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557843ebbed0 0x557843ebe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90725a640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557843ebbed0 0x557843ebe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.816+0000 7ff90725a640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557843ebbed0 0x557843ebe2c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.817+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.817+0000 7ff90725a640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff90003b260 0x7ff90004e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.817+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff90003b260 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.817+0000 7ff906a59640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff90003b260 0x7ff90004e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.817+0000 7ff906a59640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff90003b260 0x7ff90004e110 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.818+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.818+0000 7ff906a59640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff8f808a070 0x7ff8f808c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.818+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.818+0000 7ff906258640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff8f808a070 0x7ff8f808c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.818+0000 7ff906258640 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff8f808a070 0x7ff8f808c700 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.818+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc009050 con 0x7ff8f808a070 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.818+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc009050 con 0x557843eb5c90 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.819+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff90003b260 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.820+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff90003b260 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.821+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.822+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.822+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.822+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.822+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.822+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.822+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.823+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff90003b260 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.824+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.824+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.824+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.824+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.824+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.824+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff90003b260 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843ebbed0 2024-04-23T19:10:28.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.825+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff8f808a070 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x557843eb5c90 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557843ebe990 con 0x7ff90003b260 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.826+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843eba970 con 0x557843eb5c90 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843eba5b0 con 0x557843eb5c90 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843ebb2d0 con 0x7ff8f808a070 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 zone default found 2024-04-23T19:10:28.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 4 Realm: () 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 zonegroup default 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.827+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843ebee10 con 0x7ff8f808a070 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843ec0ca0 con 0x557843ebbed0 2024-04-23T19:10:28.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843ec1040 con 0x557843ebbed0 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 20 started sync module instance, tier type = 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.828+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec00e0 con 0x7ff90003b260 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.829+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.829+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93975023977728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ebfd40 con 0x7ff90003b260 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.829+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec1eb0 con 0x557843eb5c90 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.830+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93975023977728] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.830+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.831+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93975023983104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec22d0 con 0x557843eb5c90 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.831+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec2c90 con 0x557843ebbed0 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.831+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93975023983104] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.832+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.832+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93975023986656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec3030 con 0x557843ebbed0 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.832+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec3a80 con 0x7ff90003b260 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.833+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93975023986656] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.833+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.833+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93975023990128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec3e70 con 0x7ff90003b260 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.833+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec48b0 con 0x557843eb5c90 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.834+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.834+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93975023993728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec4870 con 0x557843eb5c90 2024-04-23T19:10:28.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.834+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec5870 con 0x7ff8f808a070 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.834+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93975023990128] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.835+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93975023993728] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.835+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.835+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93975023997856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec4a50 con 0x7ff8f808a070 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.835+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec4a50 con 0x7ff8f808a070 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.836+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93975023997856] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.836+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.836+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93975023984160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec5f60 con 0x7ff8f808a070 2024-04-23T19:10:28.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.836+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec67b0 con 0x7ff8f808a070 2024-04-23T19:10:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.837+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93975024001344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec6b30 con 0x7ff8f808a070 2024-04-23T19:10:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 20 add_watcher() i=0 2024-04-23T19:10:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 20 add_watcher() i=1 2024-04-23T19:10:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 20 add_watcher() i=2 2024-04-23T19:10:28.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 20 add_watcher() i=3 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93975023984160] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 20 add_watcher() i=4 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 20 add_watcher() i=5 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.838+0000 7ff90ad21800 20 add_watcher() i=6 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.840+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93975024001344] v30'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.840+0000 7ff90ad21800 20 add_watcher() i=7 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.840+0000 7ff90ad21800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.840+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843ec5ae0 con 0x557843eb5c90 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.841+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.841+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93975024002528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec5ae0 con 0x557843eb5c90 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93975024002528] v30'88 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff7bd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff90ad21800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff90ad21800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff90ad21800 5 note: GC not initialized 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff737fff640 20 reqs_thread_entry: start 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff90ad21800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff90ad21800 20 Filter name: none 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff735ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.842+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843efcb70 con 0x557843eb5c90 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff90ad21800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff90ad21800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843efcf50 con 0x557843eb5c90 2024-04-23T19:10:28.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff90ad21800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557843efcf50 con 0x7ff90003b260 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.843+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.844+0000 7ff7bd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.844+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93975024002528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843efcf50 con 0x557843eb5c90 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.845+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93975024002528] v30'89 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.845+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93975023977728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843efcf50 con 0x7ff90003b260 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.846+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93975023977728] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.846+0000 7ff90ad21800 20 remove_watcher() i=0 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.846+0000 7ff90ad21800 2 removed watcher, disabling cache 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.846+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93975023983104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843efcf50 con 0x557843eb5c90 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.847+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93975023983104] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.847+0000 7ff90ad21800 20 remove_watcher() i=1 2024-04-23T19:10:28.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.847+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93975023986656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec1a00 con 0x557843ebbed0 2024-04-23T19:10:28.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.848+0000 7ff90725a640 1 -- 172.21.15.63:0/1902157804 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93975023986656] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff900057040 con 0x557843ebbed0 2024-04-23T19:10:28.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.848+0000 7ff90ad21800 20 remove_watcher() i=2 2024-04-23T19:10:28.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.848+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93975023990128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec27e0 con 0x7ff90003b260 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.849+0000 7ff906a59640 1 -- 172.21.15.63:0/1902157804 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93975023990128] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8f8079070 con 0x7ff90003b260 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.849+0000 7ff90ad21800 20 remove_watcher() i=3 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.849+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93975023993728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec1a00 con 0x557843eb5c90 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.850+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93975023993728] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x557843eb5c90 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.850+0000 7ff90ad21800 20 remove_watcher() i=4 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.850+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93975023997856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec27e0 con 0x7ff8f808a070 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.851+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93975023997856] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.851+0000 7ff90ad21800 20 remove_watcher() i=5 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.851+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93975023984160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec1a00 con 0x7ff8f808a070 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.852+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93975023984160] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.852+0000 7ff90ad21800 20 remove_watcher() i=6 2024-04-23T19:10:28.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.852+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93975024001344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557843ec27e0 con 0x7ff8f808a070 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.853+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93975024001344] v30'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8fc02a040 con 0x7ff8f808a070 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.853+0000 7ff90ad21800 20 remove_watcher() i=7 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff8f808a070 msgr2=0x7ff8f808c700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff8f808a070 0x7ff8f808c700 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843eb5c90 msgr2=0x557843eb8150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843eb5c90 0x557843eb8150 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff90003b260 msgr2=0x7ff90004e110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff90003b260 0x7ff90004e110 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557843ebbed0 msgr2=0x557843ebe2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557843ebbed0 0x557843ebe2c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7d0036eb0 msgr2=0x7ff7d0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7d0036eb0 0x7ff7d0039370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8f805c910 tx=0x7ff8f808f000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 msgr2=0x557843ea9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff906258640 1 -- 172.21.15.63:0/1902157804 reap_dead start 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea9f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff900001da0 tx=0x7ff90004cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 shutdown_connections 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1902157804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843ea2aa0 0x557843ea9f30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 >> 172.21.15.63:0/1902157804 conn(0x557843e9ed70 msgr2=0x557843e9c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 shutdown_connections 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/1902157804 wait complete. 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7e0036e60 msgr2=0x7ff7e0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7e0036e60 0x7ff7e0039320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x557843e82cf0 tx=0x7ff8f808c550 comp rx=0 tx=0).stop 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843e956f0 msgr2=0x557843e95b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843e956f0 0x557843e95b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x557843e81ba0 tx=0x7ff9000178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 shutdown_connections 2024-04-23T19:10:28.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7e0036e60 0x7ff7e0039320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 --2- 172.21.15.63:0/3770116634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843e956f0 0x557843e95b10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 >> 172.21.15.63:0/3770116634 conn(0x557843e89790 msgr2=0x557843e89b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 shutdown_connections 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.854+0000 7ff90ad21800 1 -- 172.21.15.63:0/3770116634 wait complete. 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557843e849c0 msgr2=0x557843e86db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557843e849c0 0x557843e86db0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843d7e380 msgr2=0x557843e811d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843d7e380 0x557843e811d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7ec036eb0 msgr2=0x7ff7ec039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7ec036eb0 0x7ff7ec039370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc000f60 tx=0x7ff8fc0010a0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 msgr2=0x557843e79fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e79fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff8f801baa0 tx=0x7ff8f8002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 shutdown_connections 2024-04-23T19:10:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557843e849c0 0x557843e86db0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557843d7e380 0x557843e811d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff7ec036eb0 0x7ff7ec039370 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 --2- 172.21.15.63:0/1558266990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557843d7da20 0x557843e79fe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 >> 172.21.15.63:0/1558266990 conn(0x557843d79600 msgr2=0x557843e6c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 shutdown_connections 2024-04-23T19:10:28.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.855+0000 7ff90ad21800 1 -- 172.21.15.63:0/1558266990 wait complete. 2024-04-23T19:10:28.871 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-04-23T19:10:28.871 DEBUG:tasks.util.rgw: json result: [] 2024-04-23T19:10:28.871 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-23T19:10:28.871 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-23T19:10:28.872 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-23T19:10:28.916 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:28.916 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.916+0000 7f8ebf327800 1 Processor -- start 2024-04-23T19:10:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.916+0000 7f8ebf327800 1 -- start start 2024-04-23T19:10:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.916+0000 7f8ebf327800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5588767314d0 0x558876739fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.916+0000 7f8ebf327800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5588765e37f0 con 0x55887672f990 2024-04-23T19:10:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.916+0000 7f8ebf327800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5588765d2e20 con 0x5588767314d0 2024-04-23T19:10:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.917+0000 7f8eba85f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5588767314d0 0x558876739fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.917+0000 7f8ebb060640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55887672f990 0x55887672fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38684/0 (socket says 172.21.15.63:38684) 2024-04-23T19:10:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.917+0000 7f8ebb060640 1 -- 172.21.15.63:0/1222066965 learned_addr learned my addr 172.21.15.63:0/1222066965 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.917+0000 7f8eba05e640 1 -- 172.21.15.63:0/1222066965 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630294852 0 0) 0x5588765e37f0 con 0x55887672f990 2024-04-23T19:10:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.917+0000 7f8eba05e640 1 -- 172.21.15.63:0/1222066965 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea8001e10 con 0x55887672f990 2024-04-23T19:10:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.917+0000 7f8eba85f640 1 -- 172.21.15.63:0/1222066965 >> v1:172.21.15.16:6789/0 conn(0x55887672f990 legacy=0x55887672fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.917+0000 7f8eba85f640 1 -- 172.21.15.63:0/1222066965 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588765f1b50 con 0x5588767314d0 2024-04-23T19:10:28.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8eba85f640 1 --2- 172.21.15.63:0/1222066965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588767314d0 0x558876739fd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400a470 tx=0x7f8ea401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1eaef0aa63f42d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8eba05e640 1 -- 172.21.15.63:0/1222066965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ea4057070 con 0x5588767314d0 2024-04-23T19:10:28.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8eba05e640 1 -- 172.21.15.63:0/1222066965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea4052570 con 0x5588767314d0 2024-04-23T19:10:28.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8eba05e640 1 -- 172.21.15.63:0/1222066965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ea4052870 con 0x5588767314d0 2024-04-23T19:10:28.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8ebf327800 1 -- 172.21.15.63:0/1222066965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588767314d0 msgr2=0x558876739fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1222066965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588767314d0 0x558876739fd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400a470 tx=0x7f8ea401cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8ebf327800 1 -- 172.21.15.63:0/1222066965 shutdown_connections 2024-04-23T19:10:28.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1222066965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588767314d0 0x558876739fd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.919+0000 7f8ebf327800 1 -- 172.21.15.63:0/1222066965 >> 172.21.15.63:0/1222066965 conn(0x55887672b630 msgr2=0x55887672da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.920+0000 7f8ebf327800 1 -- 172.21.15.63:0/1222066965 shutdown_connections 2024-04-23T19:10:28.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.920+0000 7f8ebf327800 1 -- 172.21.15.63:0/1222066965 wait complete. 2024-04-23T19:10:28.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.932+0000 7f8ebf327800 1 Processor -- start 2024-04-23T19:10:28.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.932+0000 7f8ebf327800 1 -- start start 2024-04-23T19:10:28.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.932+0000 7f8ebf327800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x558876824600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.932+0000 7f8ebf327800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55887667c6f0 con 0x55887672f990 2024-04-23T19:10:28.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.933+0000 7f8ebb060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x558876824600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.933+0000 7f8ebb060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x558876824600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54014/0 (socket says 172.21.15.63:54014) 2024-04-23T19:10:28.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.933+0000 7f8ebb060640 1 -- 172.21.15.63:0/1215294140 learned_addr learned my addr 172.21.15.63:0/1215294140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.933+0000 7f8ebb060640 1 -- 172.21.15.63:0/1215294140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558876824bd0 con 0x55887672f990 2024-04-23T19:10:28.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.933+0000 7f8ebb060640 1 --2- 172.21.15.63:0/1215294140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x558876824600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001650 tx=0x7f8eac001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c10986dbb042cc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.933+0000 7f8da960a640 1 -- 172.21.15.63:0/1215294140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eac004520 con 0x55887672f990 2024-04-23T19:10:28.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8da960a640 1 -- 172.21.15.63:0/1215294140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac0046c0 con 0x55887672f990 2024-04-23T19:10:28.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8da960a640 1 -- 172.21.15.63:0/1215294140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eac0049a0 con 0x55887672f990 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8ebf327800 1 -- 172.21.15.63:0/1215294140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 msgr2=0x558876824600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1215294140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x558876824600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001650 tx=0x7f8eac001b10 comp rx=0 tx=0).stop 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8ebf327800 1 -- 172.21.15.63:0/1215294140 shutdown_connections 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1215294140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x558876824600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8ebf327800 1 -- 172.21.15.63:0/1215294140 >> 172.21.15.63:0/1215294140 conn(0x55887672b630 msgr2=0x55887681ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8ebf327800 1 -- 172.21.15.63:0/1215294140 shutdown_connections 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.934+0000 7f8ebf327800 1 -- 172.21.15.63:0/1215294140 wait complete. 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebf327800 1 Processor -- start 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebf327800 1 -- start start 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebf327800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x55887682c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebf327800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55887673c7f0 con 0x55887672f990 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebb060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x55887682c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebb060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x55887682c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54022/0 (socket says 172.21.15.63:54022) 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebb060640 1 -- 172.21.15.63:0/3517495125 learned_addr learned my addr 172.21.15.63:0/3517495125 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.935+0000 7f8ebb060640 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887682c660 con 0x55887672f990 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.936+0000 7f8ebb060640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x55887682c120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8eac000f80 tx=0x7f8eac001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.936+0000 7f8da7e07640 1 -- 172.21.15.63:0/3517495125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eac022020 con 0x55887672f990 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.936+0000 7f8da7e07640 1 -- 172.21.15.63:0/3517495125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac0148b0 con 0x55887672f990 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.936+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55887682c8f0 con 0x55887672f990 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.936+0000 7f8da7e07640 1 -- 172.21.15.63:0/3517495125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eac014bd0 con 0x55887672f990 2024-04-23T19:10:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.936+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887682f5a0 con 0x55887672f990 2024-04-23T19:10:28.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8da7e07640 1 -- 172.21.15.63:0/3517495125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8eac025020 con 0x55887672f990 2024-04-23T19:10:28.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8da7e07640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8da0036eb0 0x7f8da0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8da7e07640 1 -- 172.21.15.63:0/3517495125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f8eac018030 con 0x55887672f990 2024-04-23T19:10:28.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8eba85f640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8da0036eb0 0x7f8da0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588767314d0 0x558876833360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876833930 con 0x5588767314d0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8ebb861640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588767314d0 0x558876833360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8eba85f640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8da0036eb0 0x7f8da0039370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4054000 tx=0x7f8ea404c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.937+0000 7f8ebb861640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588767314d0 0x558876833360 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.938+0000 7f8ebb861640 1 -- 172.21.15.63:0/3517495125 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8eb40083e0 con 0x5588767314d0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.938+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876830910 con 0x5588767314d0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.938+0000 7f8ebb861640 1 -- 172.21.15.63:0/3517495125 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8eb40083e0 con 0x5588767314d0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.938+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876835760 con 0x5588767314d0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.938+0000 7f8ebb861640 1 -- 172.21.15.63:0/3517495125 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8eb40083e0 con 0x5588767314d0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.938+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876834aa0 con 0x5588767314d0 2024-04-23T19:10:28.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.939+0000 7f8ebb861640 1 -- 172.21.15.63:0/3517495125 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8eb40083e0 con 0x5588767314d0 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.939+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558876836b10 0x558876838f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.939+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876839480 con 0x558876836b10 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.939+0000 7f8ebb060640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558876836b10 0x558876838f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.939+0000 7f8ebb060640 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558876836b10 0x558876838f00 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.939+0000 7f8ebb060640 1 -- 172.21.15.63:0/3517495125 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8eac01b070 con 0x558876836b10 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebf327800 1 Processor -- start 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebf327800 1 -- start start 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebf327800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588768478c0 0x558876847ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebf327800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558876622e30 con 0x5588768478c0 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebb861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588768478c0 0x558876847ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebb861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588768478c0 0x558876847ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54038/0 (socket says 172.21.15.63:54038) 2024-04-23T19:10:28.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebb861640 1 -- 172.21.15.63:0/4248297563 learned_addr learned my addr 172.21.15.63:0/4248297563 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.940+0000 7f8ebb861640 1 -- 172.21.15.63:0/4248297563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558876848220 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.941+0000 7f8ebb861640 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588768478c0 0x558876847ce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55887682cff0 tx=0x7f8eb400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.941+0000 7f8da5602640 1 -- 172.21.15.63:0/4248297563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb401c070 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.941+0000 7f8da5602640 1 -- 172.21.15.63:0/4248297563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4018410 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.941+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558876848c90 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.941+0000 7f8da5602640 1 -- 172.21.15.63:0/4248297563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4018730 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.941+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558876849190 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8da5602640 1 -- 172.21.15.63:0/4248297563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8eb401f070 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8da5602640 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d94036e60 0x7f8d94039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8da5602640 1 -- 172.21.15.63:0/4248297563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f8eb404e8d0 con 0x5588768478c0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8ebb060640 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d94036e60 0x7f8d94039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8ebf327800 1 Processor -- start 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8ebf327800 1 -- start start 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8ebb060640 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d94036e60 0x7f8d94039320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8eac04b470 tx=0x7f8eac02afc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8ebf327800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x558876855020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.942+0000 7f8ebf327800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558876824f50 con 0x558876854c00 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8ebb861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x558876855020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8ebb861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x558876855020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54044/0 (socket says 172.21.15.63:54044) 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8ebb861640 1 -- 172.21.15.63:0/3921620592 learned_addr learned my addr 172.21.15.63:0/3921620592 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8ebb861640 1 -- 172.21.15.63:0/3921620592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588768555f0 con 0x558876854c00 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8ebb861640 1 --2- 172.21.15.63:0/3921620592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x558876855020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8eb404cce0 tx=0x7f8eb4014a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac0558ec76155d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8d9effd640 1 -- 172.21.15.63:0/3921620592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4016db0 con 0x558876854c00 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8d9effd640 1 -- 172.21.15.63:0/3921620592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4014b70 con 0x558876854c00 2024-04-23T19:10:28.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8d9effd640 1 -- 172.21.15.63:0/3921620592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4014e70 con 0x558876854c00 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8ebf327800 1 -- 172.21.15.63:0/3921620592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 msgr2=0x558876855020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.943+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3921620592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x558876855020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8eb404cce0 tx=0x7f8eb4014a20 comp rx=0 tx=0).stop 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 -- 172.21.15.63:0/3921620592 shutdown_connections 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3921620592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x558876855020 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 -- 172.21.15.63:0/3921620592 >> 172.21.15.63:0/3921620592 conn(0x558876850f10 msgr2=0x55887684efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 -- 172.21.15.63:0/3921620592 shutdown_connections 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 -- 172.21.15.63:0/3921620592 wait complete. 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 Processor -- start 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 -- start start 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x55887685c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebf327800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558876855dc0 con 0x558876854c00 2024-04-23T19:10:28.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebb861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x55887685c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebb861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x55887685c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54052/0 (socket says 172.21.15.63:54052) 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 learned_addr learned my addr 172.21.15.63:0/1980872537 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.944+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887685c680 con 0x558876854c00 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.945+0000 7f8ebb861640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x55887685c140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8eb401e000 tx=0x7f8eb404cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.945+0000 7f8d9d7fa640 1 -- 172.21.15.63:0/1980872537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb4018da0 con 0x558876854c00 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.945+0000 7f8d9d7fa640 1 -- 172.21.15.63:0/1980872537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb404e110 con 0x558876854c00 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.945+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55887685c910 con 0x558876854c00 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.945+0000 7f8d9d7fa640 1 -- 172.21.15.63:0/1980872537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eb404e3f0 con 0x558876854c00 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.945+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887685f5c0 con 0x558876854c00 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.945+0000 7f8d9d7fa640 1 -- 172.21.15.63:0/1980872537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8eb4029350 con 0x558876854c00 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8d9d7fa640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d84036e60 0x7f8d84039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8d9d7fa640 1 -- 172.21.15.63:0/1980872537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f8eb4036c30 con 0x558876854c00 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8ebb060640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d84036e60 0x7f8d84039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558876867be0 0x55887686a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686a7f0 con 0x558876867be0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8eba85f640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558876867be0 0x55887686a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8ebb060640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d84036e60 0x7f8d84039320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x558876839880 tx=0x7f8eac032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.946+0000 7f8eba85f640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558876867be0 0x55887686a0a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 realm 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686abc0 con 0x558876867be0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686b490 con 0x558876867be0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686b830 con 0x558876867be0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.947+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686bc50 con 0x558876867be0 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8ebf327800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8ebf327800 20 searching for the correct realm 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55887686dc50 0x5588768700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8ebb861640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55887686dc50 0x5588768700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8ebb861640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55887686dc50 0x5588768700a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.948+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8ebb861640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8eb4021070 0x7f8eb400c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eb4021070 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8ebb060640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8eb4021070 0x7f8eb400c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8ebb060640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8eb4021070 0x7f8eb400c290 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8ebb060640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8eac027020 0x7f8eac01e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.949+0000 7f8eba85f640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8eac027020 0x7f8eac01e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.950+0000 7f8eba85f640 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8eac027020 0x7f8eac01e3f0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.950+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.950+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.950+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.950+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.950+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.950+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.951+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.951+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.951+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.951+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eb4021070 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.951+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.951+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.952+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eb4021070 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.953+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.954+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eb4021070 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.955+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.956+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.956+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.956+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.956+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eb4021070 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x55887686dc50 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eac027020 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.957+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x558876867be0 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558876870770 con 0x7f8eb4021070 2024-04-23T19:10:28.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686c6f0 con 0x558876867be0 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686cf90 con 0x558876867be0 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.958+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55887686c330 con 0x7f8eac027020 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 zone default found 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 4 Realm: () 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 zonegroup default 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876870c40 con 0x7f8eac027020 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876872ad0 con 0x55887686dc50 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876872e70 con 0x55887686dc50 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.959+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.960+0000 7f8ebf327800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.960+0000 7f8ebf327800 20 started sync module instance, tier type = 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.960+0000 7f8ebf327800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.960+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876871f10 con 0x7f8eb4021070 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.961+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.961+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94044592481024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876872720 con 0x7f8eb4021070 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.961+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876873bd0 con 0x558876867be0 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.962+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.962+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94044592481024] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.962+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94044592486080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876874470 con 0x558876867be0 2024-04-23T19:10:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.962+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876873250 con 0x55887686dc50 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.963+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94044592486080] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.963+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.963+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94044592488384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588768743e0 con 0x55887686dc50 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.963+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876875510 con 0x7f8eb4021070 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.964+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.964+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94044592488384] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.964+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94044592492576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876875db0 con 0x7f8eb4021070 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.964+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876874a00 con 0x558876867be0 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.965+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94044592492576] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.965+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.965+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94044592494848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876876820 con 0x558876867be0 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.965+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588768771e0 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.966+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.966+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94044592500016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876877600 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.966+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876878040 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.966+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94044592494848] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.967+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94044592500016] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.967+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.967+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94044592503568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876877150 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.967+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588768788e0 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94044592505904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876877150 con 0x7f8eac027020 2024-04-23T19:10:28.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 20 add_watcher() i=0 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 20 add_watcher() i=1 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 20 add_watcher() i=2 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 20 add_watcher() i=3 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 20 add_watcher() i=4 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 20 add_watcher() i=5 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94044592503568] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.968+0000 7f8ebf327800 20 add_watcher() i=6 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.969+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94044592505904] v30'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.969+0000 7f8ebf327800 20 add_watcher() i=7 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.969+0000 7f8ebf327800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.969+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558876876370 con 0x558876867be0 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.969+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.970+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94044592506960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876876370 con 0x558876867be0 2024-04-23T19:10:28.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.970+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94044592506960] v30'90 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8d71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8ebf327800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8ebf327800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8ebf327800 5 note: GC not initialized 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8d70fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8ebf327800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8ebf327800 20 Filter name: none 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8cea7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588768ae830 con 0x558876867be0 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.971+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.972+0000 7f8ebf327800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.972+0000 7f8ebf327800 20 rados->read ofs=0 len=0 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.972+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588768aec10 con 0x558876867be0 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.972+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.972+0000 7f8ebf327800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.972+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5588768aec10 con 0x7f8eb4021070 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.972+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.973+0000 7f8d71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.973+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94044592506960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876878dc0 con 0x558876867be0 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.974+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94044592506960] v30'91 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.974+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94044592481024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876878850 con 0x7f8eb4021070 2024-04-23T19:10:28.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.975+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94044592481024] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.975+0000 7f8ebf327800 20 remove_watcher() i=0 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.975+0000 7f8ebf327800 2 removed watcher, disabling cache 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.975+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94044592486080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876878850 con 0x558876867be0 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.976+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94044592486080] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.976+0000 7f8ebf327800 20 remove_watcher() i=1 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.976+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94044592488384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5588768736c0 con 0x55887686dc50 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.977+0000 7f8ebb861640 1 -- 172.21.15.63:0/1980872537 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94044592488384] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eb4028070 con 0x55887686dc50 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.977+0000 7f8ebf327800 20 remove_watcher() i=2 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.977+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94044592492576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876878850 con 0x7f8eb4021070 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.978+0000 7f8ebb060640 1 -- 172.21.15.63:0/1980872537 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94044592492576] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8eac042040 con 0x7f8eb4021070 2024-04-23T19:10:28.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.978+0000 7f8ebf327800 20 remove_watcher() i=3 2024-04-23T19:10:28.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.978+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94044592494848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876873fc0 con 0x558876867be0 2024-04-23T19:10:28.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.979+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94044592494848] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x558876867be0 2024-04-23T19:10:28.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.979+0000 7f8ebf327800 20 remove_watcher() i=4 2024-04-23T19:10:28.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.979+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94044592500016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876873fc0 con 0x7f8eac027020 2024-04-23T19:10:28.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.980+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94044592500016] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.980+0000 7f8ebf327800 20 remove_watcher() i=5 2024-04-23T19:10:28.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.980+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94044592503568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876873fc0 con 0x7f8eac027020 2024-04-23T19:10:28.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.981+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94044592503568] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.981+0000 7f8ebf327800 20 remove_watcher() i=6 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.981+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94044592505904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558876873fc0 con 0x7f8eac027020 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94044592505904] v30'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8ea405a040 con 0x7f8eac027020 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 20 remove_watcher() i=7 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8eac027020 msgr2=0x7f8eac01e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8eac027020 0x7f8eac01e3f0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558876867be0 msgr2=0x55887686a0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558876867be0 0x55887686a0a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8eb4021070 msgr2=0x7f8eb400c290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8eb4021070 0x7f8eb400c290 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55887686dc50 msgr2=0x5588768700a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55887686dc50 0x5588768700a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d84036e60 msgr2=0x7f8d84039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d84036e60 0x7f8d84039320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x558876839880 tx=0x7f8eac032000 comp rx=0 tx=0).stop 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 msgr2=0x55887685c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 --2- 172.21.15.63:0/1980872537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558876854c00 0x55887685c140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8eb401e000 tx=0x7f8eb404cc70 comp rx=0 tx=0).stop 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 reap_dead start 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8eba85f640 1 -- 172.21.15.63:0/1980872537 reap_dead start 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 shutdown_connections 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 >> 172.21.15.63:0/1980872537 conn(0x558876850f10 msgr2=0x55887684e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 shutdown_connections 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.982+0000 7f8ebf327800 1 -- 172.21.15.63:0/1980872537 wait complete. 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d94036e60 msgr2=0x7f8d94039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d94036e60 0x7f8d94039320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8eac04b470 tx=0x7f8eac02afc0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588768478c0 msgr2=0x558876847ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588768478c0 0x558876847ce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55887682cff0 tx=0x7f8eb400ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 shutdown_connections 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8d94036e60 0x7f8d94039320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/4248297563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588768478c0 0x558876847ce0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 >> 172.21.15.63:0/4248297563 conn(0x55887683b960 msgr2=0x55887683bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 shutdown_connections 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/4248297563 wait complete. 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558876836b10 msgr2=0x558876838f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558876836b10 0x558876838f00 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588767314d0 msgr2=0x558876833360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588767314d0 0x558876833360 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:28.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8da0036eb0 msgr2=0x7f8da0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8da0036eb0 0x7f8da0039370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4054000 tx=0x7f8ea404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 msgr2=0x55887682c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x55887682c120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8eac000f80 tx=0x7f8eac001250 comp rx=0 tx=0).stop 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 shutdown_connections 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558876836b10 0x558876838f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5588767314d0 0x558876833360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8da0036eb0 0x7f8da0039370 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 --2- 172.21.15.63:0/3517495125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55887672f990 0x55887682c120 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 >> 172.21.15.63:0/3517495125 conn(0x55887672b630 msgr2=0x55887681e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 shutdown_connections 2024-04-23T19:10:29.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:28.983+0000 7f8ebf327800 1 -- 172.21.15.63:0/3517495125 wait complete. 2024-04-23T19:10:29.001 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-04-23T19:10:29.001 DEBUG:tasks.util.rgw: json result: [] 2024-04-23T19:10:29.018 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.019 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-23T19:10:29.019 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-23T19:10:29.019 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-23T19:10:29.065 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:29.065 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:29.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.065+0000 7f4fa292e800 1 Processor -- start 2024-04-23T19:10:29.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4fa292e800 1 -- start start 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4fa292e800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e653c82a40 0x55e653c82e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4fa292e800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e653b367f0 con 0x55e653c833a0 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4fa292e800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55e653b25e20 con 0x55e653c82a40 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4f9e667640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e653c82a40 0x55e653c82e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4f9de66640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55e653c833a0 0x55e653c83850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38688/0 (socket says 172.21.15.63:38688) 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4f9de66640 1 -- 172.21.15.63:0/238494524 learned_addr learned my addr 172.21.15.63:0/238494524 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4f9e667640 1 -- 172.21.15.63:0/238494524 >> v1:172.21.15.16:6789/0 conn(0x55e653c833a0 legacy=0x55e653c83850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.066+0000 7f4f9e667640 1 -- 172.21.15.63:0/238494524 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e653b44b50 con 0x55e653c82a40 2024-04-23T19:10:29.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.068+0000 7f4f9e667640 1 --2- 172.21.15.63:0/238494524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653c82e60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000a470 tx=0x7f4f9001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30bed85bc01b41e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.068+0000 7f4f9d665640 1 -- 172.21.15.63:0/238494524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90057070 con 0x55e653c82a40 2024-04-23T19:10:29.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.068+0000 7f4f9d665640 1 -- 172.21.15.63:0/238494524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9004adc0 con 0x55e653c82a40 2024-04-23T19:10:29.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.068+0000 7f4f9d665640 1 -- 172.21.15.63:0/238494524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90052590 con 0x55e653c82a40 2024-04-23T19:10:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.069+0000 7f4fa292e800 1 -- 172.21.15.63:0/238494524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 msgr2=0x55e653c82e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.069+0000 7f4fa292e800 1 --2- 172.21.15.63:0/238494524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653c82e60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000a470 tx=0x7f4f9001ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.069+0000 7f4fa292e800 1 -- 172.21.15.63:0/238494524 shutdown_connections 2024-04-23T19:10:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.069+0000 7f4fa292e800 1 --2- 172.21.15.63:0/238494524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653c82e60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.069+0000 7f4fa292e800 1 -- 172.21.15.63:0/238494524 >> 172.21.15.63:0/238494524 conn(0x55e653c7e600 msgr2=0x55e653c80a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.069+0000 7f4fa292e800 1 -- 172.21.15.63:0/238494524 shutdown_connections 2024-04-23T19:10:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.069+0000 7f4fa292e800 1 -- 172.21.15.63:0/238494524 wait complete. 2024-04-23T19:10:29.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4fa292e800 1 Processor -- start 2024-04-23T19:10:29.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4fa292e800 1 -- start start 2024-04-23T19:10:29.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4fa292e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d77660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4fa292e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e653bcf6f0 con 0x55e653c82a40 2024-04-23T19:10:29.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4f9e667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d77660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4f9e667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d77660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54064/0 (socket says 172.21.15.63:54064) 2024-04-23T19:10:29.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4f9e667640 1 -- 172.21.15.63:0/1058294687 learned_addr learned my addr 172.21.15.63:0/1058294687 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.082+0000 7f4f9e667640 1 -- 172.21.15.63:0/1058294687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e653d77c30 con 0x55e653c82a40 2024-04-23T19:10:29.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.083+0000 7f4f9e667640 1 --2- 172.21.15.63:0/1058294687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d77660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f90055040 tx=0x7f4f9004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c092b72abe2707ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.083+0000 7f4e8c608640 1 -- 172.21.15.63:0/1058294687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90057040 con 0x55e653c82a40 2024-04-23T19:10:29.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.083+0000 7f4e8c608640 1 -- 172.21.15.63:0/1058294687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f90063070 con 0x55e653c82a40 2024-04-23T19:10:29.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.083+0000 7f4e8c608640 1 -- 172.21.15.63:0/1058294687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9005d790 con 0x55e653c82a40 2024-04-23T19:10:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.083+0000 7f4fa292e800 1 -- 172.21.15.63:0/1058294687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 msgr2=0x55e653d77660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.083+0000 7f4fa292e800 1 --2- 172.21.15.63:0/1058294687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d77660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f90055040 tx=0x7f4f9004c000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 -- 172.21.15.63:0/1058294687 shutdown_connections 2024-04-23T19:10:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 --2- 172.21.15.63:0/1058294687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d77660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 -- 172.21.15.63:0/1058294687 >> 172.21.15.63:0/1058294687 conn(0x55e653c7e600 msgr2=0x55e653d72f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 -- 172.21.15.63:0/1058294687 shutdown_connections 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 -- 172.21.15.63:0/1058294687 wait complete. 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 Processor -- start 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 -- start start 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d7f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4fa292e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e653c84200 con 0x55e653c82a40 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4f9e667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d7f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4f9e667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d7f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54068/0 (socket says 172.21.15.63:54068) 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.084+0000 7f4f9e667640 1 -- 172.21.15.63:0/3085324547 learned_addr learned my addr 172.21.15.63:0/3085324547 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.085+0000 7f4f9e667640 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e653d7f640 con 0x55e653c82a40 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.085+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d7f100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f90060040 tx=0x7f4f90002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.085+0000 7f4e8ae05640 1 -- 172.21.15.63:0/3085324547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90057050 con 0x55e653c82a40 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.085+0000 7f4e8ae05640 1 -- 172.21.15.63:0/3085324547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9005dcd0 con 0x55e653c82a40 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.085+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e653d7f8d0 con 0x55e653c82a40 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.085+0000 7f4e8ae05640 1 -- 172.21.15.63:0/3085324547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f90052440 con 0x55e653c82a40 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.085+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e653d82580 con 0x55e653c82a40 2024-04-23T19:10:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.086+0000 7f4e8ae05640 1 -- 172.21.15.63:0/3085324547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4f90052600 con 0x55e653c82a40 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.086+0000 7f4e8ae05640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e84036eb0 0x7f4e84039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.086+0000 7f4e8ae05640 1 -- 172.21.15.63:0/3085324547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4f9008e350 con 0x55e653c82a40 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.086+0000 7f4f9de66640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e84036eb0 0x7f4e84039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.086+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653c833a0 0x55e653d86340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.086+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653d868c0 con 0x55e653c833a0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.086+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653c833a0 0x55e653d86340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.087+0000 7f4f9de66640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e84036eb0 0x7f4e84039370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000f60 tx=0x7f4f880012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.087+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653c833a0 0x55e653d86340 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.087+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3085324547 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f9800a4a0 con 0x55e653c833a0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.087+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653d83910 con 0x55e653c833a0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.087+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3085324547 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4f9800a4a0 con 0x55e653c833a0 2024-04-23T19:10:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.087+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653d886c0 con 0x55e653c833a0 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.088+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3085324547 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4f9800a4a0 con 0x55e653c833a0 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.088+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653d87a00 con 0x55e653c833a0 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.088+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3085324547 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4f9800a4a0 con 0x55e653c833a0 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.088+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e653d89a70 0x55e653d8be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.088+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653d8c3e0 con 0x55e653d89a70 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.088+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e653d89a70 0x55e653d8be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.088+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e653d89a70 0x55e653d8be60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.089+0000 7f4f9e667640 1 -- 172.21.15.63:0/3085324547 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4f90078070 con 0x55e653d89a70 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.089+0000 7f4fa292e800 1 Processor -- start 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.089+0000 7f4fa292e800 1 -- start start 2024-04-23T19:10:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.089+0000 7f4fa292e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653d9a800 0x55e653d9ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.089+0000 7f4fa292e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e653b75e30 con 0x55e653d9a800 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.089+0000 7f4f9ee68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653d9a800 0x55e653d9ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4f9ee68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653d9a800 0x55e653d9ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54084/0 (socket says 172.21.15.63:54084) 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4f9ee68640 1 -- 172.21.15.63:0/2411501049 learned_addr learned my addr 172.21.15.63:0/2411501049 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4f9ee68640 1 -- 172.21.15.63:0/2411501049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e653d9b160 con 0x55e653d9a800 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653d9a800 0x55e653d9ac20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55e653d80500 tx=0x7f4f980178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4e83fff640 1 -- 172.21.15.63:0/2411501049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98013440 con 0x55e653d9a800 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4e83fff640 1 -- 172.21.15.63:0/2411501049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f980135e0 con 0x55e653d9a800 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e653d9bbd0 con 0x55e653d9a800 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4e83fff640 1 -- 172.21.15.63:0/2411501049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f980138c0 con 0x55e653d9a800 2024-04-23T19:10:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.090+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e653d9c0d0 con 0x55e653d9a800 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.091+0000 7f4e83fff640 1 -- 172.21.15.63:0/2411501049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4f98011040 con 0x55e653d9a800 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.091+0000 7f4e83fff640 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e78036eb0 0x7f4e78039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.091+0000 7f4e83fff640 1 -- 172.21.15.63:0/2411501049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4f9801d070 con 0x55e653d9a800 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.091+0000 7f4f9e667640 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e78036eb0 0x7f4e78039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.091+0000 7f4fa292e800 1 Processor -- start 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4fa292e800 1 -- start start 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4f9e667640 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e78036eb0 0x7f4e78039370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f9005efd0 tx=0x7f4f9008c630 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4fa292e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653da7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4fa292e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e653d77ef0 con 0x55e653da7be0 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4f9ee68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653da7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4f9ee68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653da7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54092/0 (socket says 172.21.15.63:54092) 2024-04-23T19:10:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4f9ee68640 1 -- 172.21.15.63:0/1796347833 learned_addr learned my addr 172.21.15.63:0/1796347833 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4f9ee68640 1 -- 172.21.15.63:0/1796347833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e653da85b0 con 0x55e653da7be0 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/1796347833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653da7fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4f9804cca0 tx=0x7f4f980127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11feb8c80c3015ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4e81ffb640 1 -- 172.21.15.63:0/1796347833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98042020 con 0x55e653da7be0 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4e81ffb640 1 -- 172.21.15.63:0/1796347833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f98012cc0 con 0x55e653da7be0 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.092+0000 7f4e81ffb640 1 -- 172.21.15.63:0/1796347833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98027040 con 0x55e653da7be0 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 -- 172.21.15.63:0/1796347833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 msgr2=0x55e653da7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 --2- 172.21.15.63:0/1796347833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653da7fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4f9804cca0 tx=0x7f4f980127a0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 -- 172.21.15.63:0/1796347833 shutdown_connections 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 --2- 172.21.15.63:0/1796347833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653da7fe0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 -- 172.21.15.63:0/1796347833 >> 172.21.15.63:0/1796347833 conn(0x55e653da3e60 msgr2=0x55e653da1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 -- 172.21.15.63:0/1796347833 shutdown_connections 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 -- 172.21.15.63:0/1796347833 wait complete. 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 Processor -- start 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 -- start start 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653daf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.093+0000 7f4fa292e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e653da8d30 con 0x55e653da7be0 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4f9ee68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653daf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4f9ee68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653daf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54108/0 (socket says 172.21.15.63:54108) 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 learned_addr learned my addr 172.21.15.63:0/3997538188 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e653daf600 con 0x55e653da7be0 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653daf0c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4f980127e0 tx=0x7f4f9804cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4e6ffff640 1 -- 172.21.15.63:0/3997538188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98027040 con 0x55e653da7be0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4e6ffff640 1 -- 172.21.15.63:0/3997538188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f98021c60 con 0x55e653da7be0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e653daf890 con 0x55e653da7be0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4e6ffff640 1 -- 172.21.15.63:0/3997538188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9802a350 con 0x55e653da7be0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.094+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e653db2540 con 0x55e653da7be0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4e6ffff640 1 -- 172.21.15.63:0/3997538188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4f98045020 con 0x55e653da7be0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4e6ffff640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e68036f00 0x7f4e680393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4e6ffff640 1 -- 172.21.15.63:0/3997538188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4f98033030 con 0x55e653da7be0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e68036f00 0x7f4e680393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653dbab10 0x55e653dbcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dbd720 con 0x55e653dbab10 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.095+0000 7f4f9de66640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653dbab10 0x55e653dbcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e68036f00 0x7f4e680393c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000a470 tx=0x7f4f90083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4f9de66640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653dbab10 0x55e653dbcfd0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f88009050 con 0x55e653dbab10 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 20 realm 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653db9fb0 con 0x55e653dbab10 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f88009050 con 0x55e653dbab10 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.096+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dbe200 con 0x55e653dbab10 2024-04-23T19:10:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f88009050 con 0x55e653dbab10 2024-04-23T19:10:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dbead0 con 0x55e653dbab10 2024-04-23T19:10:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4f88009050 con 0x55e653dbab10 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dbee70 con 0x55e653dbab10 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4f88009050 con 0x55e653dbab10 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 20 searching for the correct realm 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e653dc0d50 0x55e653dc3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e653dc0d50 0x55e653dc3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.097+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e653dc0d50 0x55e653dc3140 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.098+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.098+0000 7f4f9ee68640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f980123a0 0x7f4f9802b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.098+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f980123a0 2024-04-23T19:10:29.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.098+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f980123a0 0x7f4f9802b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.098+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f980123a0 0x7f4f9802b3e0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.099+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.099+0000 7f4f9e667640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f9009b040 0x7f4f9005e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.099+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.099+0000 7f4f9de66640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f9009b040 0x7f4f9005e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.099+0000 7f4f9de66640 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f9009b040 0x7f4f9005e900 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.099+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f88009050 con 0x7f4f9009b040 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.099+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f88009050 con 0x55e653dbab10 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.100+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f980123a0 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.101+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.101+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.101+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.101+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.101+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.101+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f980123a0 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.102+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.103+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.103+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.103+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.103+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.103+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.103+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.103+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f980123a0 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.104+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.105+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.105+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.105+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.105+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.105+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.105+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.105+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.106+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.106+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f980123a0 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.106+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.106+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dc0d50 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.106+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.106+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f9009b040 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x55e653dbab10 2024-04-23T19:10:29.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc37f0 con 0x7f4f980123a0 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dbf820 con 0x55e653dbab10 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.107+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc00c0 con 0x55e653dbab10 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dbf460 con 0x7f4f9009b040 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 20 zone default found 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 4 Realm: () 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 20 zonegroup default 2024-04-23T19:10:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc3c50 con 0x7f4f9009b040 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.108+0000 7f4fa292e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc5ae0 con 0x55e653dc0d50 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc5e80 con 0x55e653dc0d50 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 20 started sync module instance, tier type = 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.109+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc4f20 con 0x7f4f980123a0 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.110+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.110+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94447737787200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc66d0 con 0x7f4f980123a0 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.110+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc6f20 con 0x55e653dbab10 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.111+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94447737787200] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.111+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.111+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94447737791008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc6f20 con 0x55e653dbab10 2024-04-23T19:10:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.111+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc7a10 con 0x55e653dc0d50 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.112+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94447737791008] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.113+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.113+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94447737793136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc6e90 con 0x55e653dc0d50 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.113+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc88d0 con 0x7f4f980123a0 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.114+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94447737793136] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.114+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.114+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94447737799712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc8cc0 con 0x7f4f980123a0 2024-04-23T19:10:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.114+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc96e0 con 0x55e653dbab10 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.115+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94447737799712] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.115+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.115+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94447737803216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc9ad0 con 0x55e653dbab10 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.115+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dca4d0 con 0x7f4f9009b040 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.116+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.116+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94447737806816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dca400 con 0x7f4f9009b040 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.116+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dcb420 con 0x7f4f9009b040 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.116+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94447737803216] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.117+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94447737806816] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.117+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.117+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94447737809088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc7e00 con 0x7f4f9009b040 2024-04-23T19:10:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.117+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dcb8d0 con 0x7f4f9009b040 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94447737811936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dcbcf0 con 0x7f4f9009b040 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 20 add_watcher() i=0 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 20 add_watcher() i=1 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 20 add_watcher() i=2 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 20 add_watcher() i=3 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 20 add_watcher() i=4 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 20 add_watcher() i=5 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94447737809088] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.118+0000 7f4fa292e800 20 add_watcher() i=6 2024-04-23T19:10:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.119+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94447737811936] v30'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.119+0000 7f4fa292e800 20 add_watcher() i=7 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.119+0000 7f4fa292e800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.119+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653dcaf10 con 0x55e653dbab10 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.119+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.119+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94447737812992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dcaf10 con 0x55e653dbab10 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.120+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94447737812992] v30'92 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.120+0000 7f4e54fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.120+0000 7f4fa292e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.120+0000 7f4fa292e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.120+0000 7f4fa292e800 5 note: GC not initialized 2024-04-23T19:10:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4dcf7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4fa292e800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4fa292e800 20 Filter name: none 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4dcd7fa640 20 reqs_thread_entry: start 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653e01d90 con 0x55e653dbab10 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4fa292e800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4fa292e800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.121+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653e02170 con 0x55e653dbab10 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.122+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.122+0000 7f4fa292e800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.122+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55e653e03180 con 0x7f4f980123a0 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.122+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.123+0000 7f4e54fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.123+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94447737812992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653e02170 con 0x55e653dbab10 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.124+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94447737812992] v30'93 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.124+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94447737787200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dcb800 con 0x7f4f980123a0 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.125+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94447737787200] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.125+0000 7f4fa292e800 20 remove_watcher() i=0 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.125+0000 7f4fa292e800 2 removed watcher, disabling cache 2024-04-23T19:10:29.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.125+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94447737791008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc5340 con 0x55e653dbab10 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.126+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94447737791008] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.126+0000 7f4fa292e800 20 remove_watcher() i=1 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.126+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94447737793136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc6220 con 0x55e653dc0d50 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.127+0000 7f4f9ee68640 1 -- 172.21.15.63:0/3997538188 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94447737793136] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f98029070 con 0x55e653dc0d50 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.127+0000 7f4fa292e800 20 remove_watcher() i=2 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.127+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94447737799712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc6220 con 0x7f4f980123a0 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.128+0000 7f4f9e667640 1 -- 172.21.15.63:0/3997538188 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94447737799712] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9007e050 con 0x7f4f980123a0 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.128+0000 7f4fa292e800 20 remove_watcher() i=3 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.128+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94447737803216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc8420 con 0x55e653dbab10 2024-04-23T19:10:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.129+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94447737803216] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x55e653dbab10 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.129+0000 7f4fa292e800 20 remove_watcher() i=4 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.129+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94447737806816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc6220 con 0x7f4f9009b040 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.130+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94447737806816] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.130+0000 7f4fa292e800 20 remove_watcher() i=5 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.130+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94447737809088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc91d0 con 0x7f4f9009b040 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.131+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94447737809088] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.131+0000 7f4fa292e800 20 remove_watcher() i=6 2024-04-23T19:10:29.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.131+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94447737811936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55e653dc9fe0 con 0x7f4f9009b040 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94447737811936] v30'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f8802a040 con 0x7f4f9009b040 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 20 remove_watcher() i=7 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f9009b040 msgr2=0x7f4f9005e900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f9009b040 0x7f4f9005e900 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653dbab10 msgr2=0x55e653dbcfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653dbab10 0x55e653dbcfd0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f980123a0 msgr2=0x7f4f9802b3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f980123a0 0x7f4f9802b3e0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e653dc0d50 msgr2=0x55e653dc3140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e653dc0d50 0x55e653dc3140 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e68036f00 msgr2=0x7f4e680393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e68036f00 0x7f4e680393c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000a470 tx=0x7f4f90083000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 msgr2=0x55e653daf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3997538188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653da7be0 0x55e653daf0c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4f980127e0 tx=0x7f4f9804cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 reap_dead start 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4f9de66640 1 -- 172.21.15.63:0/3997538188 reap_dead start 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 shutdown_connections 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 >> 172.21.15.63:0/3997538188 conn(0x55e653da3e60 msgr2=0x55e653da14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 shutdown_connections 2024-04-23T19:10:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.132+0000 7f4fa292e800 1 -- 172.21.15.63:0/3997538188 wait complete. 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e78036eb0 msgr2=0x7f4e78039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e78036eb0 0x7f4e78039370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f9005efd0 tx=0x7f4f9008c630 comp rx=0 tx=0).stop 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653d9a800 msgr2=0x55e653d9ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653d9a800 0x55e653d9ac20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55e653d80500 tx=0x7f4f980178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 shutdown_connections 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e78036eb0 0x7f4e78039370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/2411501049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653d9a800 0x55e653d9ac20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 >> 172.21.15.63:0/2411501049 conn(0x55e653d8e8a0 msgr2=0x55e653d8eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 shutdown_connections 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/2411501049 wait complete. 2024-04-23T19:10:29.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e653d89a70 msgr2=0x55e653d8be60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e653d89a70 0x55e653d8be60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653c833a0 msgr2=0x55e653d86340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653c833a0 0x55e653d86340 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e84036eb0 msgr2=0x7f4e84039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e84036eb0 0x7f4e84039370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000f60 tx=0x7f4f880012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 msgr2=0x55e653d7f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d7f100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f90060040 tx=0x7f4f90002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 shutdown_connections 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e653d89a70 0x55e653d8be60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e653c833a0 0x55e653d86340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e84036eb0 0x7f4e84039370 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 --2- 172.21.15.63:0/3085324547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e653c82a40 0x55e653d7f100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 >> 172.21.15.63:0/3085324547 conn(0x55e653c7e600 msgr2=0x55e653d71780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 shutdown_connections 2024-04-23T19:10:29.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.133+0000 7f4fa292e800 1 -- 172.21.15.63:0/3085324547 wait complete. 2024-04-23T19:10:29.161 INFO:teuthology.orchestra.run.smithi063.stdout:["myfoo"] 2024-04-23T19:10:29.161 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-23T19:10:29.165 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.165 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-23T19:10:29.166 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-23T19:10:29.166 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-23T19:10:29.208 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:29.208 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:29.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.208+0000 7f557781f800 1 Processor -- start 2024-04-23T19:10:29.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.208+0000 7f557781f800 1 -- start start 2024-04-23T19:10:29.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f557781f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559bfa98c000 0x559bfa98c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f557781f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559bfa83f7f0 con 0x559bfa98d260 2024-04-23T19:10:29.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f557781f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x559bfa82ee20 con 0x559bfa98c000 2024-04-23T19:10:29.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f557531e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559bfa98c000 0x559bfa98c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f5574b1d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x559bfa98d260 0x559bfa98d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38704/0 (socket says 172.21.15.63:38704) 2024-04-23T19:10:29.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f5574b1d640 1 -- 172.21.15.63:0/2680261116 learned_addr learned my addr 172.21.15.63:0/2680261116 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:29.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f557531e640 1 -- 172.21.15.63:0/2680261116 >> v1:172.21.15.16:6789/0 conn(0x559bfa98d260 legacy=0x559bfa98d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.209+0000 7f557531e640 1 -- 172.21.15.63:0/2680261116 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfa84db50 con 0x559bfa98c000 2024-04-23T19:10:29.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.211+0000 7f557531e640 1 --2- 172.21.15.63:0/2680261116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa98c420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f556800a470 tx=0x7f556801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3d3269188066f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.211+0000 7f5567fff640 1 -- 172.21.15.63:0/2680261116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5568057070 con 0x559bfa98c000 2024-04-23T19:10:29.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.211+0000 7f5567fff640 1 -- 172.21.15.63:0/2680261116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5568052570 con 0x559bfa98c000 2024-04-23T19:10:29.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.211+0000 7f5567fff640 1 -- 172.21.15.63:0/2680261116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5568052870 con 0x559bfa98c000 2024-04-23T19:10:29.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.212+0000 7f557781f800 1 -- 172.21.15.63:0/2680261116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 msgr2=0x559bfa98c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.212+0000 7f557781f800 1 --2- 172.21.15.63:0/2680261116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa98c420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f556800a470 tx=0x7f556801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:29.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.212+0000 7f557781f800 1 -- 172.21.15.63:0/2680261116 shutdown_connections 2024-04-23T19:10:29.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.212+0000 7f557781f800 1 --2- 172.21.15.63:0/2680261116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa98c420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.212+0000 7f557781f800 1 -- 172.21.15.63:0/2680261116 >> 172.21.15.63:0/2680261116 conn(0x559bfa987620 msgr2=0x559bfa989a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.212+0000 7f557781f800 1 -- 172.21.15.63:0/2680261116 shutdown_connections 2024-04-23T19:10:29.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.212+0000 7f557781f800 1 -- 172.21.15.63:0/2680261116 wait complete. 2024-04-23T19:10:29.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.224+0000 7f557781f800 1 Processor -- start 2024-04-23T19:10:29.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.225+0000 7f557781f800 1 -- start start 2024-04-23T19:10:29.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.225+0000 7f557781f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfaa7d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.225+0000 7f557781f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bfa8d86e0 con 0x559bfa98c000 2024-04-23T19:10:29.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.225+0000 7f557531e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfaa7d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.225+0000 7f557531e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfaa7d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54118/0 (socket says 172.21.15.63:54118) 2024-04-23T19:10:29.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.225+0000 7f557531e640 1 -- 172.21.15.63:0/1788342942 learned_addr learned my addr 172.21.15.63:0/1788342942 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.225+0000 7f557531e640 1 -- 172.21.15.63:0/1788342942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfaa7dcd0 con 0x559bfa98c000 2024-04-23T19:10:29.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.226+0000 7f557531e640 1 --2- 172.21.15.63:0/1788342942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfaa7d700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5568055040 tx=0x7f556804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3efae1a2963af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.226+0000 7f5463606640 1 -- 172.21.15.63:0/1788342942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5568057040 con 0x559bfa98c000 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.226+0000 7f5463606640 1 -- 172.21.15.63:0/1788342942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5568052990 con 0x559bfa98c000 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.226+0000 7f5463606640 1 -- 172.21.15.63:0/1788342942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5568052c70 con 0x559bfa98c000 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.226+0000 7f557781f800 1 -- 172.21.15.63:0/1788342942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 msgr2=0x559bfaa7d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.226+0000 7f557781f800 1 --2- 172.21.15.63:0/1788342942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfaa7d700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5568055040 tx=0x7f556804c000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 -- 172.21.15.63:0/1788342942 shutdown_connections 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 --2- 172.21.15.63:0/1788342942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfaa7d700 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 -- 172.21.15.63:0/1788342942 >> 172.21.15.63:0/1788342942 conn(0x559bfa987620 msgr2=0x559bfaa79020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 -- 172.21.15.63:0/1788342942 shutdown_connections 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 -- 172.21.15.63:0/1788342942 wait complete. 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 Processor -- start 2024-04-23T19:10:29.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 -- start start 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa900110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557781f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bfa9956d0 con 0x559bfa98c000 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557531e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa900110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557531e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa900110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54132/0 (socket says 172.21.15.63:54132) 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.227+0000 7f557531e640 1 -- 172.21.15.63:0/3378990522 learned_addr learned my addr 172.21.15.63:0/3378990522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.228+0000 7f557531e640 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfaa88f00 con 0x559bfa98c000 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.228+0000 7f557531e640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa900110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f556800a470 tx=0x7f5568003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.228+0000 7f5461e03640 1 -- 172.21.15.63:0/3378990522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5568073070 con 0x559bfa98c000 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.228+0000 7f5461e03640 1 -- 172.21.15.63:0/3378990522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556805d440 con 0x559bfa98c000 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.228+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559bfaa890d0 con 0x559bfa98c000 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.228+0000 7f5461e03640 1 -- 172.21.15.63:0/3378990522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5568057070 con 0x559bfa98c000 2024-04-23T19:10:29.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.228+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bfaa895f0 con 0x559bfa98c000 2024-04-23T19:10:29.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.229+0000 7f5461e03640 1 -- 172.21.15.63:0/3378990522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5568076070 con 0x559bfa98c000 2024-04-23T19:10:29.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.229+0000 7f5461e03640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f545c036e60 0x7f545c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.229+0000 7f5461e03640 1 -- 172.21.15.63:0/3378990522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f556808e470 con 0x559bfa98c000 2024-04-23T19:10:29.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.229+0000 7f5574b1d640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f545c036e60 0x7f545c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.229+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfa98d260 0x559bfaa8d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.229+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaa8daa0 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.229+0000 7f5575b1f640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfa98d260 0x559bfaa8d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.230+0000 7f5574b1d640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f545c036e60 0x7f545c039320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f556c000f60 tx=0x7f556c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.230+0000 7f5575b1f640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfa98d260 0x559bfaa8d4d0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.230+0000 7f5575b1f640 1 -- 172.21.15.63:0/3378990522 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f557000a4a0 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.230+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaa8aaa0 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.230+0000 7f5575b1f640 1 -- 172.21.15.63:0/3378990522 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f557000a4a0 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.230+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaa8f910 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.231+0000 7f5575b1f640 1 -- 172.21.15.63:0/3378990522 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f557000a4a0 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.231+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaa8ec50 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.231+0000 7f5575b1f640 1 -- 172.21.15.63:0/3378990522 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f557000a4a0 con 0x559bfa98d260 2024-04-23T19:10:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.231+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559bfaa90cc0 0x559bfaa930b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.231+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaa93630 con 0x559bfaa90cc0 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.231+0000 7f557531e640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559bfaa90cc0 0x559bfaa930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.231+0000 7f557531e640 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559bfaa90cc0 0x559bfaa930b0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f557531e640 1 -- 172.21.15.63:0/3378990522 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5568075030 con 0x559bfaa90cc0 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f557781f800 1 Processor -- start 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f557781f800 1 -- start start 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f557781f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaa1a50 0x559bfaaa1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f557781f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bfa87ee30 con 0x559bfaaa1a50 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f5575b1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaa1a50 0x559bfaaa1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f5575b1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaa1a50 0x559bfaaa1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54148/0 (socket says 172.21.15.63:54148) 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.232+0000 7f5575b1f640 1 -- 172.21.15.63:0/2249171772 learned_addr learned my addr 172.21.15.63:0/2249171772 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.233+0000 7f5575b1f640 1 -- 172.21.15.63:0/2249171772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfaaa23b0 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.233+0000 7f5575b1f640 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaa1a50 0x559bfaaa1e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x559bfaa8de80 tx=0x7f55700178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.233+0000 7f545affd640 1 -- 172.21.15.63:0/2249171772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5570013440 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.233+0000 7f545affd640 1 -- 172.21.15.63:0/2249171772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55700135e0 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.233+0000 7f545affd640 1 -- 172.21.15.63:0/2249171772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55700138c0 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.233+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559bfaaa2e20 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.233+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bfaaa3320 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f545affd640 1 -- 172.21.15.63:0/2249171772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5570011040 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f545affd640 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f544c036e60 0x7f544c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f545affd640 1 -- 172.21.15.63:0/2249171772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f557001d070 con 0x559bfaaa1a50 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f557531e640 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f544c036e60 0x7f544c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f557781f800 1 Processor -- start 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f557781f800 1 -- start start 2024-04-23T19:10:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f557531e640 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f544c036e60 0x7f544c039320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5568006b30 tx=0x7f55680067e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.234+0000 7f557781f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaaaf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f557781f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bfaa7df90 con 0x559bfaaaed50 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5575b1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaaaf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5575b1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaaaf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54160/0 (socket says 172.21.15.63:54160) 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5575b1f640 1 -- 172.21.15.63:0/2797751126 learned_addr learned my addr 172.21.15.63:0/2797751126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5575b1f640 1 -- 172.21.15.63:0/2797751126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfaaaf740 con 0x559bfaaaed50 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5575b1f640 1 --2- 172.21.15.63:0/2797751126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaaaf170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5570001470 tx=0x7f5570012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e5f4a782adc753 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5458ff9640 1 -- 172.21.15.63:0/2797751126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5570043070 con 0x559bfaaaed50 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5458ff9640 1 -- 172.21.15.63:0/2797751126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5570015430 con 0x559bfaaaed50 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.235+0000 7f5458ff9640 1 -- 172.21.15.63:0/2797751126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557002ac50 con 0x559bfaaaed50 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 -- 172.21.15.63:0/2797751126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 msgr2=0x559bfaaaf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 --2- 172.21.15.63:0/2797751126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaaaf170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5570001470 tx=0x7f5570012a30 comp rx=0 tx=0).stop 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 -- 172.21.15.63:0/2797751126 shutdown_connections 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 --2- 172.21.15.63:0/2797751126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaaaf170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 -- 172.21.15.63:0/2797751126 >> 172.21.15.63:0/2797751126 conn(0x559bfaaab0b0 msgr2=0x559bfaaa9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 -- 172.21.15.63:0/2797751126 shutdown_connections 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 -- 172.21.15.63:0/2797751126 wait complete. 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 Processor -- start 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 -- start start 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaab6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f557781f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bfaaafeb0 con 0x559bfaaaed50 2024-04-23T19:10:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f5575b1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaab6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f5575b1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaab6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54174/0 (socket says 172.21.15.63:54174) 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.236+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 learned_addr learned my addr 172.21.15.63:0/1012616634 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.237+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfaab6860 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.237+0000 7f5575b1f640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaab6320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f557004cb80 tx=0x7f557004cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.237+0000 7f544affd640 1 -- 172.21.15.63:0/1012616634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5570043030 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.237+0000 7f544affd640 1 -- 172.21.15.63:0/1012616634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55700017f0 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.237+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559bfaab6af0 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.237+0000 7f544affd640 1 -- 172.21.15.63:0/1012616634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55700124d0 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.237+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bfaab97a0 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f544affd640 1 -- 172.21.15.63:0/1012616634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f557004e090 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f544affd640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5440036e10 0x7f54400392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f544affd640 1 -- 172.21.15.63:0/1012616634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f5570044070 con 0x559bfaaaed50 2024-04-23T19:10:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f557531e640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5440036e10 0x7f54400392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfaac1fa0 0x559bfaac4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac4bb0 con 0x559bfaac1fa0 2024-04-23T19:10:29.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f557531e640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5440036e10 0x7f54400392d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5568006470 tx=0x7f556808a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.238+0000 7f5574b1d640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfaac1fa0 0x559bfaac4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f5574b1d640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfaac1fa0 0x559bfaac4460 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f556c009050 con 0x559bfaac1fa0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 20 realm 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac13f0 con 0x559bfaac1fa0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f556c009050 con 0x559bfaac1fa0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac5690 con 0x559bfaac1fa0 2024-04-23T19:10:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f556c009050 con 0x559bfaac1fa0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.239+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac5f60 con 0x559bfaac1fa0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f556c009050 con 0x559bfaac1fa0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f557781f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac6300 con 0x559bfaac1fa0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f556c009050 con 0x559bfaac1fa0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f557781f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f557781f800 20 searching for the correct realm 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559bfaac81e0 0x559bfaaca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f5575b1f640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559bfaac81e0 0x559bfaaca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.240+0000 7f5575b1f640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559bfaac81e0 0x559bfaaca5d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.241+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.241+0000 7f5575b1f640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f557002b5c0 0x7f557003b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.241+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f557002b5c0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.241+0000 7f557531e640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f557002b5c0 0x7f557003b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.241+0000 7f557531e640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f557002b5c0 0x7f557003b800 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.241+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.241+0000 7f557531e640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f556809c040 0x7f556808e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.242+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.242+0000 7f5574b1d640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f556809c040 0x7f556808e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.242+0000 7f5574b1d640 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f556809c040 0x7f556808e7d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.242+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c009050 con 0x7f556809c040 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.242+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c009050 con 0x559bfaac1fa0 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.243+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f557002b5c0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.244+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.244+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.244+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.244+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.244+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.244+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f557002b5c0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.245+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.246+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.247+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f557002b5c0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.248+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.249+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.249+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f557002b5c0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.249+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.249+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac81e0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.249+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.249+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f556809c040 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x559bfaac1fa0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacac80 con 0x7f557002b5c0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.250+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac6cb0 con 0x559bfaac1fa0 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac68f0 con 0x7f556809c040 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 zone default found 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 4 Realm: () 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 zonegroup default 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.251+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacb0e0 con 0x7f556809c040 2024-04-23T19:10:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaaccf70 con 0x559bfaac81e0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacd310 con 0x559bfaac81e0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 20 started sync module instance, tier type = 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.252+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacc400 con 0x7f557002b5c0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.253+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.253+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94128413919264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacdbb0 con 0x7f557002b5c0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.254+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaace400 con 0x559bfaac1fa0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.254+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94128413919264] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.255+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.255+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94128413925200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaace820 con 0x559bfaac1fa0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.255+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacf070 con 0x559bfaac81e0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.256+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94128413925200] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.256+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.256+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94128413926736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacf460 con 0x559bfaac81e0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.256+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacfe80 con 0x7f557002b5c0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.257+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94128413926736] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.257+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.257+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94128413931888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad0270 con 0x7f557002b5c0 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.257+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad0c90 con 0x559bfaac1fa0 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.258+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94128413931888] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.258+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.258+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94128413935488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad1080 con 0x559bfaac1fa0 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.258+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad1a80 con 0x7f556809c040 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.259+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94128413935488] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.259+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.259+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94128413927856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad1a40 con 0x7f556809c040 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.259+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad2490 con 0x7f556809c040 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.260+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94128413927856] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.260+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.260+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94128413939088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad2670 con 0x7f556809c040 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.261+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad2ec0 con 0x7f556809c040 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94128413939088] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94128413942688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad2cf0 con 0x7f556809c040 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=0 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=1 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=2 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=3 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=4 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=5 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=6 2024-04-23T19:10:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94128413942688] v30'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 20 add_watcher() i=7 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.262+0000 7f557781f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.263+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad6290 con 0x559bfaac1fa0 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.263+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.263+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94128413943744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad3780 con 0x559bfaac1fa0 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.264+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94128413943744] v30'94 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.264+0000 7f542bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.264+0000 7f557781f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.264+0000 7f557781f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:29.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.264+0000 7f557781f800 5 note: GC not initialized 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.264+0000 7f542afe5640 20 reqs_thread_entry: start 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.264+0000 7f557781f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.265+0000 7f557781f800 20 Filter name: none 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.265+0000 7f5428fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.265+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfab08f10 con 0x559bfaac81e0 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.265+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.265+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacf070 con 0x559bfaac81e0 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.265+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.265+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacf070 con 0x559bfaac1fa0 2024-04-23T19:10:29.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(64 [pgnls start_epoch 30 out=127b] v30'3 uv3 ondisk = 1) ==== 144+0+127 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v30'11 uv11 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f557002b5c0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.266+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f557002b5c0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v30'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.267+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(74 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(75 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.268+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.269+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(76 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.269+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.269+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(77 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.269+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.269+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(78 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.269+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(79 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f557002b5c0 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(80 [pgnls start_epoch 30 out=70b] v30'2 uv2 ondisk = 1) ==== 144+0+70 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(81 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(82 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(83 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.270+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f557002b5c0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.271+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(84 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.271+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.271+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(85 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.271+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.271+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(86 [pgnls start_epoch 30 out=49b] v30'1 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.271+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.272+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(87 [pgnls start_epoch 30 out=49b] v30'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.272+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.272+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(88 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.272+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f557002b5c0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.272+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(89 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.272+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f556809c040 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(90 [pgnls start_epoch 30 out=49b] v30'1 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac1fa0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(91 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x559bfaac81e0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(92 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559bfaac7550 con 0x7f557002b5c0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(93 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f557781f800 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.273+0000 7f557781f800 20 RGWRados::pool_iterate: got myfoo 2024-04-23T19:10:29.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.274+0000 7f542bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.274+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94128413943744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacf070 con 0x559bfaac1fa0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.275+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94128413943744] v30'95 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.275+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94128413919264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad27c0 con 0x7f557002b5c0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.276+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94128413919264] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.276+0000 7f557781f800 20 remove_watcher() i=0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.276+0000 7f557781f800 2 removed watcher, disabling cache 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.276+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94128413925200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad27c0 con 0x559bfaac1fa0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.277+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94128413925200] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.277+0000 7f557781f800 20 remove_watcher() i=1 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.277+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94128413926736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacdf50 con 0x559bfaac81e0 2024-04-23T19:10:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.278+0000 7f5575b1f640 1 -- 172.21.15.63:0/1012616634 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94128413926736] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5570029070 con 0x559bfaac81e0 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.278+0000 7f557781f800 20 remove_watcher() i=2 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.278+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94128413931888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad27c0 con 0x7f557002b5c0 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.279+0000 7f557531e640 1 -- 172.21.15.63:0/1012616634 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94128413931888] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5568075030 con 0x7f557002b5c0 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.279+0000 7f557781f800 20 remove_watcher() i=3 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.279+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94128413935488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacf970 con 0x559bfaac1fa0 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.280+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94128413935488] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x559bfaac1fa0 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.280+0000 7f557781f800 20 remove_watcher() i=4 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.280+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94128413927856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad27c0 con 0x7f556809c040 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.281+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94128413927856] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.281+0000 7f557781f800 20 remove_watcher() i=5 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.281+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94128413939088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaad0780 con 0x7f556809c040 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.282+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94128413939088] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.282+0000 7f557781f800 20 remove_watcher() i=6 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.282+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94128413942688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559bfaacf970 con 0x7f556809c040 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94128413942688] v30'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f556c02a040 con 0x7f556809c040 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 20 remove_watcher() i=7 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f556809c040 msgr2=0x7f556808e7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f556809c040 0x7f556808e7d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfaac1fa0 msgr2=0x559bfaac4460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfaac1fa0 0x559bfaac4460 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f557002b5c0 msgr2=0x7f557003b800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f557002b5c0 0x7f557003b800 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559bfaac81e0 msgr2=0x559bfaaca5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559bfaac81e0 0x559bfaaca5d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5440036e10 msgr2=0x7f54400392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5440036e10 0x7f54400392d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5568006470 tx=0x7f556808a000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 msgr2=0x559bfaab6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 --2- 172.21.15.63:0/1012616634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaaed50 0x559bfaab6320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f557004cb80 tx=0x7f557004cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 reap_dead start 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f5574b1d640 1 -- 172.21.15.63:0/1012616634 reap_dead start 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 shutdown_connections 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.283+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 >> 172.21.15.63:0/1012616634 conn(0x559bfaaab0b0 msgr2=0x559bfaaa8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 shutdown_connections 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/1012616634 wait complete. 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f544c036e60 msgr2=0x7f544c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f544c036e60 0x7f544c039320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5568006b30 tx=0x7f55680067e0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaa1a50 msgr2=0x559bfaaa1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaa1a50 0x559bfaaa1e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x559bfaa8de80 tx=0x7f55700178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 shutdown_connections 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f544c036e60 0x7f544c039320 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/2249171772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfaaa1a50 0x559bfaaa1e70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 >> 172.21.15.63:0/2249171772 conn(0x559bfaa95af0 msgr2=0x559bfaa95ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 shutdown_connections 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/2249171772 wait complete. 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559bfaa90cc0 msgr2=0x559bfaa930b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559bfaa90cc0 0x559bfaa930b0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfa98d260 msgr2=0x559bfaa8d4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfa98d260 0x559bfaa8d4d0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f545c036e60 msgr2=0x7f545c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f545c036e60 0x7f545c039320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f556c000f60 tx=0x7f556c0012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 msgr2=0x559bfa900110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa900110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f556800a470 tx=0x7f5568003020 comp rx=0 tx=0).stop 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 shutdown_connections 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559bfaa90cc0 0x559bfaa930b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559bfa98d260 0x559bfaa8d4d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f545c036e60 0x7f545c039320 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 --2- 172.21.15.63:0/3378990522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bfa98c000 0x559bfa900110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 >> 172.21.15.63:0/3378990522 conn(0x559bfa987620 msgr2=0x559bfaa77820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.284+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 shutdown_connections 2024-04-23T19:10:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.285+0000 7f557781f800 1 -- 172.21.15.63:0/3378990522 wait complete. 2024-04-23T19:10:29.304 INFO:teuthology.orchestra.run.smithi063.stdout:["myfoo"] 2024-04-23T19:10:29.304 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-23T19:10:29.319 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.331 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.342 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.387 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-04-23T19:10:29.414 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.440 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.468 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:29.468 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-23T19:10:29.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-23T19:10:29.468 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-23T19:10:29.513 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:29.514 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82cdb33800 1 Processor -- start 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82cdb33800 1 -- start start 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82cdb33800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55b7f312cc10 0x55b7f312f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82cdb33800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b7f2fde7f0 con 0x55b7f312f540 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82cdb33800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55b7f2fcde20 con 0x55b7f312cc10 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82c98f2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55b7f312cc10 0x55b7f312f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82c90f1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55b7f312f540 0x55b7f3131930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38720/0 (socket says 172.21.15.63:38720) 2024-04-23T19:10:29.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82c90f1640 1 -- 172.21.15.63:0/3135123116 learned_addr learned my addr 172.21.15.63:0/3135123116 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:29.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.514+0000 7f82c98f2640 1 -- 172.21.15.63:0/3135123116 >> v1:172.21.15.16:6789/0 conn(0x55b7f312f540 legacy=0x55b7f3131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.515+0000 7f82c98f2640 1 -- 172.21.15.63:0/3135123116 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f2fecb50 con 0x55b7f312cc10 2024-04-23T19:10:29.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.516+0000 7f82c98f2640 1 --2- 172.21.15.63:0/3135123116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312f000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00a470 tx=0x7f82bc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e83576687cd8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.516+0000 7f82c88f0640 1 -- 172.21.15.63:0/3135123116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc057070 con 0x55b7f312cc10 2024-04-23T19:10:29.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.516+0000 7f82c88f0640 1 -- 172.21.15.63:0/3135123116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc052570 con 0x55b7f312cc10 2024-04-23T19:10:29.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82c88f0640 1 -- 172.21.15.63:0/3135123116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc052870 con 0x55b7f312cc10 2024-04-23T19:10:29.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82cdb33800 1 -- 172.21.15.63:0/3135123116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 msgr2=0x55b7f312f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3135123116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312f000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00a470 tx=0x7f82bc01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:29.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82cdb33800 1 -- 172.21.15.63:0/3135123116 shutdown_connections 2024-04-23T19:10:29.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3135123116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312f000 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82cdb33800 1 -- 172.21.15.63:0/3135123116 >> 172.21.15.63:0/3135123116 conn(0x55b7f3126600 msgr2=0x55b7f3128a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82cdb33800 1 -- 172.21.15.63:0/3135123116 shutdown_connections 2024-04-23T19:10:29.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.517+0000 7f82cdb33800 1 -- 172.21.15.63:0/3135123116 wait complete. 2024-04-23T19:10:29.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.530+0000 7f82cdb33800 1 Processor -- start 2024-04-23T19:10:29.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.530+0000 7f82cdb33800 1 -- start start 2024-04-23T19:10:29.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.530+0000 7f82cdb33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.530+0000 7f82cdb33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7f30776f0 con 0x55b7f312cc10 2024-04-23T19:10:29.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.530+0000 7f82c98f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.530+0000 7f82c98f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54186/0 (socket says 172.21.15.63:54186) 2024-04-23T19:10:29.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.530+0000 7f82c98f2640 1 -- 172.21.15.63:0/1305244914 learned_addr learned my addr 172.21.15.63:0/1305244914 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.531+0000 7f82c98f2640 1 -- 172.21.15.63:0/1305244914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f312ab10 con 0x55b7f312cc10 2024-04-23T19:10:29.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.531+0000 7f82c98f2640 1 --2- 172.21.15.63:0/1305244914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312c550 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f82bc055040 tx=0x7f82bc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18e4d98a4db417e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.531+0000 7f81b7e07640 1 -- 172.21.15.63:0/1305244914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc057040 con 0x55b7f312cc10 2024-04-23T19:10:29.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.531+0000 7f81b7e07640 1 -- 172.21.15.63:0/1305244914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc052990 con 0x55b7f312cc10 2024-04-23T19:10:29.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.531+0000 7f81b7e07640 1 -- 172.21.15.63:0/1305244914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc052c70 con 0x55b7f312cc10 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 -- 172.21.15.63:0/1305244914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 msgr2=0x55b7f312c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 --2- 172.21.15.63:0/1305244914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312c550 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f82bc055040 tx=0x7f82bc04c000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 -- 172.21.15.63:0/1305244914 shutdown_connections 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 --2- 172.21.15.63:0/1305244914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f312c550 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 -- 172.21.15.63:0/1305244914 >> 172.21.15.63:0/1305244914 conn(0x55b7f3126600 msgr2=0x55b7f3218c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 -- 172.21.15.63:0/1305244914 shutdown_connections 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 -- 172.21.15.63:0/1305244914 wait complete. 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 Processor -- start 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 -- start start 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f3227020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82cdb33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7f3135360 con 0x55b7f312cc10 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82c98f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f3227020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82c98f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f3227020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54198/0 (socket says 172.21.15.63:54198) 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.532+0000 7f82c98f2640 1 -- 172.21.15.63:0/3118830462 learned_addr learned my addr 172.21.15.63:0/3118830462 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.533+0000 7f82c98f2640 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f3227560 con 0x55b7f312cc10 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.533+0000 7f82c98f2640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f3227020 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00a470 tx=0x7f82bc003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.533+0000 7f81b6604640 1 -- 172.21.15.63:0/3118830462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc073070 con 0x55b7f312cc10 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.533+0000 7f81b6604640 1 -- 172.21.15.63:0/3118830462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc05d650 con 0x55b7f312cc10 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.533+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7f32277f0 con 0x55b7f312cc10 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.533+0000 7f81b6604640 1 -- 172.21.15.63:0/3118830462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc057070 con 0x55b7f312cc10 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.533+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f322a4a0 con 0x55b7f312cc10 2024-04-23T19:10:29.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.534+0000 7f81b6604640 1 -- 172.21.15.63:0/3118830462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f82bc076070 con 0x55b7f312cc10 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.534+0000 7f81b6604640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81b0036eb0 0x7f81b0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.534+0000 7f81b6604640 1 -- 172.21.15.63:0/3118830462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f82bc08e550 con 0x55b7f312cc10 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.534+0000 7f82c90f1640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81b0036eb0 0x7f81b0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.534+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f312f540 0x55b7f322e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.534+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f322e7e0 con 0x55b7f312f540 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.534+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f312f540 0x55b7f322e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.535+0000 7f82c90f1640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81b0036eb0 0x7f81b0039370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82c0000f60 tx=0x7f82c00012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.535+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f312f540 0x55b7f322e260 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.535+0000 7f82ca0f3640 1 -- 172.21.15.63:0/3118830462 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f82c400a4a0 con 0x55b7f312f540 2024-04-23T19:10:29.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.535+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f322b830 con 0x55b7f312f540 2024-04-23T19:10:29.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.535+0000 7f82ca0f3640 1 -- 172.21.15.63:0/3118830462 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f82c400a4a0 con 0x55b7f312f540 2024-04-23T19:10:29.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.535+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32305e0 con 0x55b7f312f540 2024-04-23T19:10:29.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.536+0000 7f82ca0f3640 1 -- 172.21.15.63:0/3118830462 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f82c400a4a0 con 0x55b7f312f540 2024-04-23T19:10:29.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.536+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f322f920 con 0x55b7f312f540 2024-04-23T19:10:29.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.536+0000 7f82ca0f3640 1 -- 172.21.15.63:0/3118830462 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f82c400a4a0 con 0x55b7f312f540 2024-04-23T19:10:29.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.536+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7f3231990 0x55b7f3233d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.536+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3234300 con 0x55b7f3231990 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.536+0000 7f82c98f2640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7f3231990 0x55b7f3233d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.536+0000 7f82c98f2640 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7f3231990 0x55b7f3233d80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82c98f2640 1 -- 172.21.15.63:0/3118830462 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f82bc075030 con 0x55b7f3231990 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82cdb33800 1 Processor -- start 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82cdb33800 1 -- start start 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82cdb33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f3242720 0x55b7f3242b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82cdb33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7f301de30 con 0x55b7f3242720 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82ca0f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f3242720 0x55b7f3242b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82ca0f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f3242720 0x55b7f3242b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54206/0 (socket says 172.21.15.63:54206) 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.537+0000 7f82ca0f3640 1 -- 172.21.15.63:0/2241823615 learned_addr learned my addr 172.21.15.63:0/2241823615 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.538+0000 7f82ca0f3640 1 -- 172.21.15.63:0/2241823615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f3243080 con 0x55b7f3242720 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.538+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f3242720 0x55b7f3242b40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55b7f3228420 tx=0x7f82c40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.538+0000 7f81af7fe640 1 -- 172.21.15.63:0/2241823615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82c4013440 con 0x55b7f3242720 2024-04-23T19:10:29.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.538+0000 7f81af7fe640 1 -- 172.21.15.63:0/2241823615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c40135e0 con 0x55b7f3242720 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.538+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7f3243af0 con 0x55b7f3242720 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.538+0000 7f81af7fe640 1 -- 172.21.15.63:0/2241823615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82c40138c0 con 0x55b7f3242720 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.538+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f3243ff0 con 0x55b7f3242720 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.539+0000 7f81af7fe640 1 -- 172.21.15.63:0/2241823615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f82c4011040 con 0x55b7f3242720 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.539+0000 7f81af7fe640 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81a4036eb0 0x7f81a4039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.539+0000 7f81af7fe640 1 -- 172.21.15.63:0/2241823615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f82c401d070 con 0x55b7f3242720 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.539+0000 7f82c98f2640 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81a4036eb0 0x7f81a4039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.539+0000 7f82cdb33800 1 Processor -- start 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.539+0000 7f82c98f2640 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81a4036eb0 0x7f81a4039370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006a00 tx=0x7f82bc006ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.539+0000 7f82cdb33800 1 -- start start 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f82cdb33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f324fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f82cdb33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7f312add0 con 0x55b7f324fa00 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f82ca0f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f324fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f82ca0f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f324fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54214/0 (socket says 172.21.15.63:54214) 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f82ca0f3640 1 -- 172.21.15.63:0/1347056092 learned_addr learned my addr 172.21.15.63:0/1347056092 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f82ca0f3640 1 -- 172.21.15.63:0/1347056092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f32503f0 con 0x55b7f324fa00 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/1347056092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f324fe20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f82c404cca0 tx=0x7f82c40127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc40de9242775 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f81ad7fa640 1 -- 172.21.15.63:0/1347056092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82c4042020 con 0x55b7f324fa00 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f81ad7fa640 1 -- 172.21.15.63:0/1347056092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c4012cc0 con 0x55b7f324fa00 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.540+0000 7f81ad7fa640 1 -- 172.21.15.63:0/1347056092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82c4027040 con 0x55b7f324fa00 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 -- 172.21.15.63:0/1347056092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 msgr2=0x55b7f324fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 --2- 172.21.15.63:0/1347056092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f324fe20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f82c404cca0 tx=0x7f82c40127a0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 -- 172.21.15.63:0/1347056092 shutdown_connections 2024-04-23T19:10:29.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 --2- 172.21.15.63:0/1347056092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f324fe20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 -- 172.21.15.63:0/1347056092 >> 172.21.15.63:0/1347056092 conn(0x55b7f324bd60 msgr2=0x55b7f3249dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 -- 172.21.15.63:0/1347056092 shutdown_connections 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 -- 172.21.15.63:0/1347056092 wait complete. 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 Processor -- start 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 -- start start 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f3257000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82cdb33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7f3250b60 con 0x55b7f324fa00 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82ca0f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f3257000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.541+0000 7f82ca0f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f3257000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54220/0 (socket says 172.21.15.63:54220) 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 learned_addr learned my addr 172.21.15.63:0/212136547 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f3257540 con 0x55b7f324fa00 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f3257000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82c404bb90 tx=0x7f82c404cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f819b7fe640 1 -- 172.21.15.63:0/212136547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82c4027040 con 0x55b7f324fa00 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f819b7fe640 1 -- 172.21.15.63:0/212136547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c4021c60 con 0x55b7f324fa00 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f819b7fe640 1 -- 172.21.15.63:0/212136547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82c402a350 con 0x55b7f324fa00 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7f32577d0 con 0x55b7f324fa00 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.542+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f325a480 con 0x55b7f324fa00 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f819b7fe640 1 -- 172.21.15.63:0/212136547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f82c4045020 con 0x55b7f324fa00 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f819b7fe640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8194036f00 0x7f81940393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f819b7fe640 1 -- 172.21.15.63:0/212136547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f82c4033070 con 0x55b7f324fa00 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f82c98f2640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8194036f00 0x7f81940393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f3262ca0 0x55b7f3265160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32658b0 con 0x55b7f3262ca0 2024-04-23T19:10:29.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.543+0000 7f82c90f1640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f3262ca0 0x55b7f3265160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82c98f2640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8194036f00 0x7f81940393c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006e60 tx=0x7f82bc084000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82c90f1640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f3262ca0 0x55b7f3265160 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f82c0009050 con 0x55b7f3262ca0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 20 realm 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32620f0 con 0x55b7f3262ca0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f82c0009050 con 0x55b7f3262ca0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3266390 con 0x55b7f3262ca0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f82c0009050 con 0x55b7f3262ca0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.544+0000 7f82cdb33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3266c60 con 0x55b7f3262ca0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f82c0009050 con 0x55b7f3262ca0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3267000 con 0x55b7f3262ca0 2024-04-23T19:10:29.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f82c0009050 con 0x55b7f3262ca0 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 20 searching for the correct realm 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7f3268ee0 0x55b7f326b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.545+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7f3268ee0 0x55b7f326b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.546+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7f3268ee0 0x55b7f326b2d0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.546+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.546+0000 7f82ca0f3640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f82c405b040 0x7f82c4041550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.546+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82c405b040 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.546+0000 7f82c98f2640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f82c405b040 0x7f82c4041550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.546+0000 7f82c98f2640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f82c405b040 0x7f82c4041550 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.547+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.547+0000 7f82c98f2640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f82bc09a040 0x7f82bc05ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.547+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.547+0000 7f82c90f1640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f82bc09a040 0x7f82bc05ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.547+0000 7f82c90f1640 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f82bc09a040 0x7f82bc05ca90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c0009050 con 0x7f82bc09a040 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c0009050 con 0x55b7f3262ca0 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.548+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82c405b040 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.549+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82c405b040 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.550+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.551+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.552+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.552+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.552+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.552+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.552+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.552+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82c405b040 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.553+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.554+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.554+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.554+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.554+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82c405b040 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.554+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.554+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3268ee0 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82bc09a040 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x55b7f3262ca0 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326b980 con 0x7f82c405b040 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.555+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3267980 con 0x55b7f3262ca0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82cdb33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3268220 con 0x55b7f3262ca0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.556+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32675c0 con 0x7f82bc09a040 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 zone default found 2024-04-23T19:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 4 Realm: () 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 zonegroup default 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326bde0 con 0x7f82bc09a040 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326dc70 con 0x55b7f3268ee0 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326e010 con 0x55b7f3268ee0 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.557+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.558+0000 7f82cdb33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.558+0000 7f82cdb33800 20 started sync module instance, tier type = 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.558+0000 7f82cdb33800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.558+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326d100 con 0x7f82c405b040 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.559+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.559+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94248546784496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326e8b0 con 0x7f82c405b040 2024-04-23T19:10:29.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.559+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326f100 con 0x55b7f3262ca0 2024-04-23T19:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.560+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94248546784496] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.560+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.560+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94248546788352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326f1a0 con 0x55b7f3262ca0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.560+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f326fcc0 con 0x55b7f3268ee0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.561+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94248546788352] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.562+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.562+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94248546793360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32700b0 con 0x55b7f3268ee0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.562+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3270af0 con 0x7f82c405b040 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.563+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94248546793360] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.563+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.563+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94248546796992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3270ee0 con 0x7f82c405b040 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.563+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3271900 con 0x55b7f3262ca0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.564+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.564+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94248546800624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32721d0 con 0x55b7f3262ca0 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.564+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3272cc0 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.564+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94248546796992] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.565+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.565+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94248546800624] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.565+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94248546805648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3272c80 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.565+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3273220 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.566+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94248546805648] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.566+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.566+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94248546802976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3273220 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.566+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3273be0 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94248546802976] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94248546790480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3273be0 con 0x7f82bc09a040 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 20 add_watcher() i=0 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 20 add_watcher() i=1 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 20 add_watcher() i=2 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 20 add_watcher() i=3 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 20 add_watcher() i=4 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 20 add_watcher() i=5 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.567+0000 7f82cdb33800 20 add_watcher() i=6 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.568+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94248546790480] v30'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.568+0000 7f82cdb33800 20 add_watcher() i=7 2024-04-23T19:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.568+0000 7f82cdb33800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.568+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3276ef0 con 0x55b7f3262ca0 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.568+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.568+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94248546810016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3274170 con 0x55b7f3262ca0 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.569+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94248546810016] v30'96 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.569+0000 7f817b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f82cdb33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f82cdb33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f82cdb33800 5 note: GC not initialized 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f817a7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f82cdb33800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f82cdb33800 20 Filter name: none 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f80effff640 20 reqs_thread_entry: start 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.570+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32aa4d0 con 0x7f82c405b040 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32ab080 con 0x55b7f3262ca0 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82cdb33800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32ab0c0 con 0x7f82c405b040 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.571+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32ac830 con 0x55b7f3262ca0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 4.20 4:061ef416:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32abdc0 con 0x55b7f3262ca0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 4.3a 4:5ce4e6b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32b13a0 con 0x55b7f3268ee0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 4.11 4:88bce1a7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32b3c10 con 0x7f82bc09a040 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 4.3d 4:bd56d65b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32b6480 con 0x55b7f3268ee0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 4.7 4:e0c3751e:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32b8cf0 con 0x7f82bc09a040 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 4.25 4:a745f109:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32bb6d0 con 0x55b7f3268ee0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 4.22 4:4491d676:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32bb6d0 con 0x55b7f3262ca0 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 4.3e 4:7f79def2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32c0a50 con 0x7f82c405b040 2024-04-23T19:10:29.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:74 4.1d 4:b89cda5a:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32c38f0 con 0x55b7f3268ee0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 4.16 4:681bb7d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32ae3d0 con 0x55b7f3268ee0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(68 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:76 4.d 4:b3b67914:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55b7f32b1740 con 0x55b7f3268ee0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.572+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(69 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.573+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.573+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(73 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.573+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(74 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.573+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(72 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.573+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(75 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.573+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(76 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.574+0000 7f817b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.574+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94248546810016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x55b7f3262ca0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.575+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94248546810016] v30'97 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.575+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94248546784496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x7f82c405b040 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.576+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94248546784496] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.576+0000 7f82cdb33800 20 remove_watcher() i=0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.576+0000 7f82cdb33800 2 removed watcher, disabling cache 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.576+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94248546788352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x55b7f3262ca0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.577+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94248546788352] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.577+0000 7f82cdb33800 20 remove_watcher() i=1 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.577+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94248546793360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x55b7f3268ee0 2024-04-23T19:10:29.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.578+0000 7f82ca0f3640 1 -- 172.21.15.63:0/212136547 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94248546793360] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c4029070 con 0x55b7f3268ee0 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.578+0000 7f82cdb33800 20 remove_watcher() i=2 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.578+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94248546796992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x7f82c405b040 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.579+0000 7f82c98f2640 1 -- 172.21.15.63:0/212136547 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94248546796992] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82bc097070 con 0x7f82c405b040 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.579+0000 7f82cdb33800 20 remove_watcher() i=3 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.579+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94248546800624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x55b7f3262ca0 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.580+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94248546800624] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x55b7f3262ca0 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.580+0000 7f82cdb33800 20 remove_watcher() i=4 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.580+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94248546805648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x7f82bc09a040 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.581+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94248546805648] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.581+0000 7f82cdb33800 20 remove_watcher() i=5 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.581+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94248546802976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x7f82bc09a040 2024-04-23T19:10:29.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.582+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94248546802976] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.582+0000 7f82cdb33800 20 remove_watcher() i=6 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.582+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94248546790480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55b7f3292a30 con 0x7f82bc09a040 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94248546790480] v30'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82c002a040 con 0x7f82bc09a040 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 20 remove_watcher() i=7 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f82bc09a040 msgr2=0x7f82bc05ca90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f82bc09a040 0x7f82bc05ca90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f3262ca0 msgr2=0x55b7f3265160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f3262ca0 0x55b7f3265160 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f82c405b040 msgr2=0x7f82c4041550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f82c405b040 0x7f82c4041550 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7f3268ee0 msgr2=0x55b7f326b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7f3268ee0 0x55b7f326b2d0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8194036f00 msgr2=0x7f81940393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8194036f00 0x7f81940393c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006e60 tx=0x7f82bc084000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 msgr2=0x55b7f3257000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f3257000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82c404bb90 tx=0x7f82c404cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.583+0000 7f82c90f1640 1 -- 172.21.15.63:0/212136547 reap_dead start 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 shutdown_connections 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/212136547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f324fa00 0x55b7f3257000 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 >> 172.21.15.63:0/212136547 conn(0x55b7f324bd60 msgr2=0x55b7f32493c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 shutdown_connections 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/212136547 wait complete. 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81a4036eb0 msgr2=0x7f81a4039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81a4036eb0 0x7f81a4039370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006a00 tx=0x7f82bc006ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f3242720 msgr2=0x55b7f3242b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f3242720 0x55b7f3242b40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55b7f3228420 tx=0x7f82c40178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 shutdown_connections 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81a4036eb0 0x7f81a4039370 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/2241823615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f3242720 0x55b7f3242b40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 >> 172.21.15.63:0/2241823615 conn(0x55b7f32367c0 msgr2=0x55b7f3236bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 shutdown_connections 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/2241823615 wait complete. 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7f3231990 msgr2=0x55b7f3233d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7f3231990 0x55b7f3233d80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f312f540 msgr2=0x55b7f322e260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f312f540 0x55b7f322e260 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81b0036eb0 msgr2=0x7f81b0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81b0036eb0 0x7f81b0039370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82c0000f60 tx=0x7f82c00012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 msgr2=0x55b7f3227020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f3227020 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00a470 tx=0x7f82bc003020 comp rx=0 tx=0).stop 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 shutdown_connections 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.584+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7f3231990 0x55b7f3233d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.585+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7f312f540 0x55b7f322e260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.585+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f81b0036eb0 0x7f81b0039370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.585+0000 7f82cdb33800 1 --2- 172.21.15.63:0/3118830462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7f312cc10 0x55b7f3227020 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.585+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 >> 172.21.15.63:0/3118830462 conn(0x55b7f3126600 msgr2=0x55b7f312b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.585+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 shutdown_connections 2024-04-23T19:10:29.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.585+0000 7f82cdb33800 1 -- 172.21.15.63:0/3118830462 wait complete. 2024-04-23T19:10:29.603 INFO:teuthology.orchestra.run.smithi063.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"8815d3af-8342-460f-8220-31ec38570cc3","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","marker":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-23T19:10:29.012224Z","creation_time":"2024-04-23T19:10:29.008577Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-04-23T19:10:29.604 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:10:29.012224Z', 'creation_time': '2024-04-23T19:10:29.008577Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-23T19:10:29.604 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-23T19:10:29.604 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-23T19:10:29.604 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-23T19:10:29.648 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:29.648 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:29.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.648+0000 7fbcb080a800 1 Processor -- start 2024-04-23T19:10:29.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.648+0000 7fbcb080a800 1 -- start start 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.648+0000 7fbcb080a800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55be109f74d0 0x55be109fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.648+0000 7fbcb080a800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55be108a97f0 con 0x55be109f5990 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.648+0000 7fbcb080a800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55be10898e20 con 0x55be109f74d0 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.649+0000 7fbcadb08640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55be109f74d0 0x55be109fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.649+0000 7fbcae309640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55be109f5990 0x55be109f5db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38734/0 (socket says 172.21.15.63:38734) 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.649+0000 7fbcae309640 1 -- 172.21.15.63:0/2889727165 learned_addr learned my addr 172.21.15.63:0/2889727165 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.649+0000 7fbcad307640 1 -- 172.21.15.63:0/2889727165 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006521915 0 0) 0x55be108a97f0 con 0x55be109f5990 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.649+0000 7fbcad307640 1 -- 172.21.15.63:0/2889727165 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc98001e10 con 0x55be109f5990 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.649+0000 7fbcadb08640 1 -- 172.21.15.63:0/2889727165 >> v1:172.21.15.16:6789/0 conn(0x55be109f5990 legacy=0x55be109f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.649+0000 7fbcadb08640 1 -- 172.21.15.63:0/2889727165 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be108b7b50 con 0x55be109f74d0 2024-04-23T19:10:29.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.651+0000 7fbcadb08640 1 --2- 172.21.15.63:0/2889727165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f74d0 0x55be109fffd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbca400a470 tx=0x7fbca401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b523ae239f12a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.651+0000 7fbcad307640 1 -- 172.21.15.63:0/2889727165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca4057070 con 0x55be109f74d0 2024-04-23T19:10:29.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.651+0000 7fbcad307640 1 -- 172.21.15.63:0/2889727165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca4052570 con 0x55be109f74d0 2024-04-23T19:10:29.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.651+0000 7fbcad307640 1 -- 172.21.15.63:0/2889727165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca4052850 con 0x55be109f74d0 2024-04-23T19:10:29.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.651+0000 7fbcb080a800 1 -- 172.21.15.63:0/2889727165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f74d0 msgr2=0x55be109fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.651+0000 7fbcb080a800 1 --2- 172.21.15.63:0/2889727165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f74d0 0x55be109fffd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbca400a470 tx=0x7fbca401cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.652+0000 7fbcb080a800 1 -- 172.21.15.63:0/2889727165 shutdown_connections 2024-04-23T19:10:29.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.652+0000 7fbcb080a800 1 --2- 172.21.15.63:0/2889727165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f74d0 0x55be109fffd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.652+0000 7fbcb080a800 1 -- 172.21.15.63:0/2889727165 >> 172.21.15.63:0/2889727165 conn(0x55be109f1630 msgr2=0x55be109f3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.652+0000 7fbcb080a800 1 -- 172.21.15.63:0/2889727165 shutdown_connections 2024-04-23T19:10:29.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.652+0000 7fbcb080a800 1 -- 172.21.15.63:0/2889727165 wait complete. 2024-04-23T19:10:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.664+0000 7fbcb080a800 1 Processor -- start 2024-04-23T19:10:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.664+0000 7fbcb080a800 1 -- start start 2024-04-23T19:10:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbcb080a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10aea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbcb080a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be109426f0 con 0x55be109f5990 2024-04-23T19:10:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbcae309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10aea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbcae309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10aea590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54248/0 (socket says 172.21.15.63:54248) 2024-04-23T19:10:29.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbcae309640 1 -- 172.21.15.63:0/1527956278 learned_addr learned my addr 172.21.15.63:0/1527956278 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbcae309640 1 -- 172.21.15.63:0/1527956278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be10aeab60 con 0x55be109f5990 2024-04-23T19:10:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbcae309640 1 --2- 172.21.15.63:0/1527956278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10aea590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbca0001d60 tx=0x7fbca000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb504304b7869b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.665+0000 7fbb9c608640 1 -- 172.21.15.63:0/1527956278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca0012020 con 0x55be109f5990 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbb9c608640 1 -- 172.21.15.63:0/1527956278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca000e470 con 0x55be109f5990 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbb9c608640 1 -- 172.21.15.63:0/1527956278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca000e810 con 0x55be109f5990 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbcb080a800 1 -- 172.21.15.63:0/1527956278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 msgr2=0x55be10aea590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbcb080a800 1 --2- 172.21.15.63:0/1527956278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10aea590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbca0001d60 tx=0x7fbca000b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbcb080a800 1 -- 172.21.15.63:0/1527956278 shutdown_connections 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbcb080a800 1 --2- 172.21.15.63:0/1527956278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10aea590 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbcb080a800 1 -- 172.21.15.63:0/1527956278 >> 172.21.15.63:0/1527956278 conn(0x55be109f1630 msgr2=0x55be10ae5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbcb080a800 1 -- 172.21.15.63:0/1527956278 shutdown_connections 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.666+0000 7fbcb080a800 1 -- 172.21.15.63:0/1527956278 wait complete. 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcb080a800 1 Processor -- start 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcb080a800 1 -- start start 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcb080a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10af2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcb080a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be10a027f0 con 0x55be109f5990 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcae309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10af2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcae309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10af2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54260/0 (socket says 172.21.15.63:54260) 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcae309640 1 -- 172.21.15.63:0/487270731 learned_addr learned my addr 172.21.15.63:0/487270731 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.667+0000 7fbcae309640 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be10af2640 con 0x55be109f5990 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.668+0000 7fbcae309640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10af2100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbca0001d60 tx=0x7fbca0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.668+0000 7fbb9ae05640 1 -- 172.21.15.63:0/487270731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca0012030 con 0x55be109f5990 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.668+0000 7fbb9ae05640 1 -- 172.21.15.63:0/487270731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca001a7a0 con 0x55be109f5990 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.668+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55be10af28d0 con 0x55be109f5990 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.668+0000 7fbb9ae05640 1 -- 172.21.15.63:0/487270731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca001aaa0 con 0x55be109f5990 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.668+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be10af5580 con 0x55be109f5990 2024-04-23T19:10:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbb9ae05640 1 -- 172.21.15.63:0/487270731 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbca0015020 con 0x55be109f5990 2024-04-23T19:10:29.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbb9ae05640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb94036eb0 0x7fbb94039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbb9ae05640 1 -- 172.21.15.63:0/487270731 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fbca0020030 con 0x55be109f5990 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbcadb08640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb94036eb0 0x7fbb94039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be109f74d0 0x55be10af9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10af9880 con 0x55be109f74d0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be109f74d0 0x55be10af9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbcadb08640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb94036eb0 0x7fbb94039370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbca4054000 tx=0x7fbca404c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.669+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be109f74d0 0x55be10af9340 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.670+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/487270731 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbca800a4a0 con 0x55be109f74d0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.670+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10af6910 con 0x55be109f74d0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.670+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/487270731 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbca800a4a0 con 0x55be109f74d0 2024-04-23T19:10:29.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.670+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10afb6a0 con 0x55be109f74d0 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.670+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/487270731 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbca800a4a0 con 0x55be109f74d0 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.670+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10afa9e0 con 0x55be109f74d0 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.671+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/487270731 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbca800a4a0 con 0x55be109f74d0 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.671+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55be10afca50 0x55be10afee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.671+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10aff3c0 con 0x55be10afca50 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.671+0000 7fbcae309640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55be10afca50 0x55be10afee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.671+0000 7fbcae309640 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55be10afca50 0x55be10afee40 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.671+0000 7fbcae309640 1 -- 172.21.15.63:0/487270731 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbca000dcd0 con 0x55be10afca50 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcb080a800 1 Processor -- start 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcb080a800 1 -- start start 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcb080a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b0d7c0 0x55be10b0dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcb080a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be108e8e30 con 0x55be10b0d7c0 2024-04-23T19:10:29.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcaeb0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b0d7c0 0x55be10b0dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcaeb0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b0d7c0 0x55be10b0dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54272/0 (socket says 172.21.15.63:54272) 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/3637646194 learned_addr learned my addr 172.21.15.63:0/3637646194 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.672+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/3637646194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be10b0e120 con 0x55be10b0d7c0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.673+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b0d7c0 0x55be10b0dbe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55be10af3500 tx=0x7fbca80178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.673+0000 7fbb93fff640 1 -- 172.21.15.63:0/3637646194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca8013440 con 0x55be10b0d7c0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.673+0000 7fbb93fff640 1 -- 172.21.15.63:0/3637646194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca80135e0 con 0x55be10b0d7c0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.673+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55be10b0eb90 con 0x55be10b0d7c0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.673+0000 7fbb93fff640 1 -- 172.21.15.63:0/3637646194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca80138c0 con 0x55be10b0d7c0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.673+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be10b0f090 con 0x55be10b0d7c0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbb93fff640 1 -- 172.21.15.63:0/3637646194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbca8011040 con 0x55be10b0d7c0 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbb93fff640 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb88036e60 0x7fbb88039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbb93fff640 1 -- 172.21.15.63:0/3637646194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fbca801d070 con 0x55be10b0d7c0 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcae309640 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb88036e60 0x7fbb88039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcb080a800 1 Processor -- start 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcb080a800 1 -- start start 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcae309640 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb88036e60 0x7fbb88039320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbca000bf90 tx=0x7fbca000bf20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcb080a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b1af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcb080a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be10aeaee0 con 0x55be10b1ab70 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcaeb0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b1af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcaeb0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b1af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54288/0 (socket says 172.21.15.63:54288) 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.674+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/342886764 learned_addr learned my addr 172.21.15.63:0/342886764 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/342886764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be10b1b560 con 0x55be10b1ab70 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/342886764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b1af90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbca804ccb0 tx=0x7fbca8012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86568f372050e03d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbb91ffb640 1 -- 172.21.15.63:0/342886764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca8012df0 con 0x55be10b1ab70 2024-04-23T19:10:29.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbb91ffb640 1 -- 172.21.15.63:0/342886764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca8001310 con 0x55be10b1ab70 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbb91ffb640 1 -- 172.21.15.63:0/342886764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca8001630 con 0x55be10b1ab70 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcb080a800 1 -- 172.21.15.63:0/342886764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 msgr2=0x55be10b1af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcb080a800 1 --2- 172.21.15.63:0/342886764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b1af90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbca804ccb0 tx=0x7fbca8012910 comp rx=0 tx=0).stop 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcb080a800 1 -- 172.21.15.63:0/342886764 shutdown_connections 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcb080a800 1 --2- 172.21.15.63:0/342886764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b1af90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcb080a800 1 -- 172.21.15.63:0/342886764 >> 172.21.15.63:0/342886764 conn(0x55be10b16e40 msgr2=0x55be10b14ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcb080a800 1 -- 172.21.15.63:0/342886764 shutdown_connections 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.675+0000 7fbcb080a800 1 -- 172.21.15.63:0/342886764 wait complete. 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcb080a800 1 Processor -- start 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcb080a800 1 -- start start 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcb080a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b22070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcb080a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be10b1bcf0 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcaeb0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b22070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcaeb0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b22070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54300/0 (socket says 172.21.15.63:54300) 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 learned_addr learned my addr 172.21.15.63:0/547609171 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.676+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be10b225b0 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.677+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b22070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbca8015ac0 tx=0x7fbca804cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.677+0000 7fbb7ffff640 1 -- 172.21.15.63:0/547609171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca8021c60 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.677+0000 7fbb7ffff640 1 -- 172.21.15.63:0/547609171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca8021e00 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.677+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55be10b22840 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.677+0000 7fbb7ffff640 1 -- 172.21.15.63:0/547609171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca802a350 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.677+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be10b254f0 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.677+0000 7fbb7ffff640 1 -- 172.21.15.63:0/547609171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbca8040040 con 0x55be10b1ab70 2024-04-23T19:10:29.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbb7ffff640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb78036eb0 0x7fbb78039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbb7ffff640 1 -- 172.21.15.63:0/547609171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fbca8033030 con 0x55be10b1ab70 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbcae309640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb78036eb0 0x7fbb78039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be10b2db20 0x55be10b2ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b30730 con 0x55be10b2db20 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbcadb08640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be10b2db20 0x55be10b2ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbcae309640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb78036eb0 0x7fbb78039370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55be10aff7a0 tx=0x7fbca0049000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.678+0000 7fbcadb08640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be10b2db20 0x55be10b2ffe0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 realm 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b30b00 con 0x55be10b2db20 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b313d0 con 0x55be10b2db20 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b31770 con 0x55be10b2db20 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.679+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b31b90 con 0x55be10b2db20 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.680+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.680+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.680+0000 7fbcb080a800 20 searching for the correct realm 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.680+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55be10b33b90 0x55be10b35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.680+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b33b90 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.680+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55be10b33b90 0x55be10b35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.680+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55be10b33b90 0x55be10b35fe0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcaeb0a640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbca80123a0 0x7fbca8038650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x7fbca80123a0 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcae309640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbca80123a0 0x7fbca8038650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcae309640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbca80123a0 0x7fbca8038650 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcae309640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbca0017050 0x7fbca003f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.681+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x7fbca0017050 2024-04-23T19:10:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.682+0000 7fbcadb08640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbca0017050 0x7fbca003f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.682+0000 7fbcadb08640 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbca0017050 0x7fbca003f720 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.682+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.682+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b2db20 2024-04-23T19:10:29.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.682+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.683+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b2db20 2024-04-23T19:10:29.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.683+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.683+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x7fbca0017050 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.683+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.683+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x7fbca0017050 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.683+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.683+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x7fbca80123a0 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.684+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.684+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b33b90 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.684+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.684+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b2db20 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.684+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.684+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x7fbca0017050 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b2db20 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x7fbca80123a0 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b33b90 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.685+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55be10b366b0 con 0x55be10b33b90 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.686+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.686+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca0017050 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.686+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.686+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b33b90 2024-04-23T19:10:29.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.686+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.686+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b33b90 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.686+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.687+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b2db20 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.687+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.687+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b2db20 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.687+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.687+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca0017050 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.687+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.687+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b33b90 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca80123a0 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b2db20 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca0017050 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.688+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b33b90 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca0017050 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca80123a0 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b33b90 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.689+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca0017050 2024-04-23T19:10:29.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x55be10b2db20 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fbca8066040 con 0x7fbca80123a0 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.690+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b32660 con 0x55be10b2db20 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b32f00 con 0x55be10b2db20 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b322a0 con 0x7fbca0017050 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 zone default found 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 4 Realm: () 2024-04-23T19:10:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 zonegroup default 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.691+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b32f00 con 0x7fbca0017050 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b38560 con 0x55be10b33b90 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b38970 con 0x55be10b33b90 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 20 started sync module instance, tier type = 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.692+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b37a00 con 0x7fbca80123a0 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.693+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.693+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94274812345840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b39280 con 0x7fbca80123a0 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.693+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b39ad0 con 0x55be10b2db20 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.694+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.694+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94274812352032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3a0b0 con 0x55be10b2db20 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.694+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94274812345840] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.694+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3abd0 con 0x55be10b33b90 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.695+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.695+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94274812349984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3ab90 con 0x55be10b33b90 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.695+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3b610 con 0x7fbca80123a0 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.696+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94274812352032] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.696+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94274812349984] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.697+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.697+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94274812356256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b39b00 con 0x7fbca80123a0 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.697+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3c050 con 0x55be10b2db20 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.697+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.698+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94274812361904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3c700 con 0x55be10b2db20 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.698+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3d200 con 0x7fbca0017050 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.698+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94274812356256] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.699+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94274812361904] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.699+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.699+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94274812366064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3c150 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.699+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3d7e0 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.700+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94274812366064] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.700+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.700+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94274812358944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3d7e0 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.700+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3e1a0 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94274812358944] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94274812368144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3e1a0 con 0x7fbca0017050 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 20 add_watcher() i=0 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 20 add_watcher() i=1 2024-04-23T19:10:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 20 add_watcher() i=2 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 20 add_watcher() i=3 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 20 add_watcher() i=4 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 20 add_watcher() i=5 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.701+0000 7fbcb080a800 20 add_watcher() i=6 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.702+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94274812368144] v30'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.702+0000 7fbcb080a800 20 add_watcher() i=7 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.702+0000 7fbcb080a800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.702+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b414b0 con 0x55be10b2db20 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.703+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.703+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94274812370528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b3e730 con 0x55be10b2db20 2024-04-23T19:10:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94274812370528] v30'98 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbb64fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcb080a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcb080a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcb080a800 5 note: GC not initialized 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbadf7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcb080a800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcb080a800 20 Filter name: none 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbadd7fa640 20 reqs_thread_entry: start 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.704+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b744f0 con 0x55be10b2db20 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b748d0 con 0x55be10b2db20 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b758e0 con 0x7fbca80123a0 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.705+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b732f0 con 0x7fbca80123a0 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(65 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b75cd0 con 0x55be10b2db20 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:29.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 4.20 4:061ef416:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b74f70 con 0x55be10b2db20 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 4.3a 4:5ce4e6b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b7a730 con 0x55be10b33b90 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 4.11 4:88bce1a7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b7cfe0 con 0x7fbca0017050 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 4.3d 4:bd56d65b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b7f8b0 con 0x55be10b33b90 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 4.7 4:e0c3751e:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b7a730 con 0x7fbca0017050 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.706+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:72 4.25 4:a745f109:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b84990 con 0x55be10b33b90 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 4.22 4:4491d676:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b86e60 con 0x55be10b2db20 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 4.3e 4:7f79def2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b89bc0 con 0x7fbca80123a0 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 4.1d 4:b89cda5a:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b8ca80 con 0x55be10b33b90 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:76 4.16 4:681bb7d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b86e60 con 0x55be10b33b90 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(67 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 4.d 4:b3b67914:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55be10b86e60 con 0x55be10b33b90 2024-04-23T19:10:29.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(71 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(74 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(70 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(73 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(72 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(75 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(76 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.707+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(77 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.708+0000 7fbb64fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.708+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94274812370528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x55be10b2db20 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.709+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94274812370528] v30'99 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.709+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94274812345840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x7fbca80123a0 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.710+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94274812345840] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.710+0000 7fbcb080a800 20 remove_watcher() i=0 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.710+0000 7fbcb080a800 2 removed watcher, disabling cache 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.710+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94274812352032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x55be10b2db20 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.712+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94274812352032] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.712+0000 7fbcb080a800 20 remove_watcher() i=1 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.712+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94274812349984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x55be10b33b90 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.712+0000 7fbcaeb0a640 1 -- 172.21.15.63:0/547609171 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94274812349984] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca8029070 con 0x55be10b33b90 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.713+0000 7fbcb080a800 20 remove_watcher() i=2 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.713+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94274812356256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x7fbca80123a0 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.713+0000 7fbcae309640 1 -- 172.21.15.63:0/547609171 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94274812356256] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca000dcd0 con 0x7fbca80123a0 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.713+0000 7fbcb080a800 20 remove_watcher() i=3 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.713+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94274812361904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x55be10b2db20 2024-04-23T19:10:29.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.714+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94274812361904] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x55be10b2db20 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.714+0000 7fbcb080a800 20 remove_watcher() i=4 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.714+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94274812366064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x7fbca0017050 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.715+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94274812366064] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.715+0000 7fbcb080a800 20 remove_watcher() i=5 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.715+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94274812358944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x7fbca0017050 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.716+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94274812358944] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.716+0000 7fbcb080a800 20 remove_watcher() i=6 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.716+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94274812368144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55be10b5cff0 con 0x7fbca0017050 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94274812368144] v30'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbca405a040 con 0x7fbca0017050 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 20 remove_watcher() i=7 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbca0017050 msgr2=0x7fbca003f720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbca0017050 0x7fbca003f720 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be10b2db20 msgr2=0x55be10b2ffe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be10b2db20 0x55be10b2ffe0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbca80123a0 msgr2=0x7fbca8038650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbca80123a0 0x7fbca8038650 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55be10b33b90 msgr2=0x55be10b35fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.717+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55be10b33b90 0x55be10b35fe0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb78036eb0 msgr2=0x7fbb78039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb78036eb0 0x7fbb78039370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55be10aff7a0 tx=0x7fbca0049000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 msgr2=0x55be10b22070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b22070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbca8015ac0 tx=0x7fbca804cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcadb08640 1 -- 172.21.15.63:0/547609171 reap_dead start 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 shutdown_connections 2024-04-23T19:10:29.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/547609171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b1ab70 0x55be10b22070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 >> 172.21.15.63:0/547609171 conn(0x55be10b16e40 msgr2=0x55be10b144a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 shutdown_connections 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/547609171 wait complete. 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb88036e60 msgr2=0x7fbb88039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb88036e60 0x7fbb88039320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbca000bf90 tx=0x7fbca000bf20 comp rx=0 tx=0).stop 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b0d7c0 msgr2=0x55be10b0dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b0d7c0 0x55be10b0dbe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55be10af3500 tx=0x7fbca80178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 shutdown_connections 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb88036e60 0x7fbb88039320 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/3637646194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be10b0d7c0 0x55be10b0dbe0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 >> 172.21.15.63:0/3637646194 conn(0x55be10b01860 msgr2=0x55be10b01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 shutdown_connections 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/3637646194 wait complete. 2024-04-23T19:10:29.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55be10afca50 msgr2=0x55be10afee40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55be10afca50 0x55be10afee40 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be109f74d0 msgr2=0x55be10af9340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.718+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be109f74d0 0x55be10af9340 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb94036eb0 msgr2=0x7fbb94039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb94036eb0 0x7fbb94039370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbca4054000 tx=0x7fbca404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 msgr2=0x55be10af2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10af2100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbca0001d60 tx=0x7fbca0000bc0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 shutdown_connections 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55be10afca50 0x55be10afee40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55be109f74d0 0x55be10af9340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbb94036eb0 0x7fbb94039370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 --2- 172.21.15.63:0/487270731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be109f5990 0x55be10af2100 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 >> 172.21.15.63:0/487270731 conn(0x55be109f1630 msgr2=0x55be10ae46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 shutdown_connections 2024-04-23T19:10:29.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.719+0000 7fbcb080a800 1 -- 172.21.15.63:0/487270731 wait complete. 2024-04-23T19:10:29.735 INFO:teuthology.orchestra.run.smithi063.stdout:[{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"8815d3af-8342-460f-8220-31ec38570cc3","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","marker":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-23T19:10:29.012224Z","creation_time":"2024-04-23T19:10:29.008577Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}}] 2024-04-23T19:10:29.735 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:10:29.012224Z', 'creation_time': '2024-04-23T19:10:29.008577Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-23T19:10:29.741 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-23T19:10:29.741 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-23T19:10:29.741 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-23T19:10:29.741 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-23T19:10:29.821 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:29.821 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:29.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.821+0000 7f22d0e98800 1 Processor -- start 2024-04-23T19:10:29.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.821+0000 7f22d0e98800 1 -- start start 2024-04-23T19:10:29.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22d0e98800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55cb53695c40 0x55cb53698030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22d0e98800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55cb535477f0 con 0x55cb53698570 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22d0e98800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55cb53536e20 con 0x55cb53695c40 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22ce99e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55cb53695c40 0x55cb53698030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22ce19d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55cb53698570 0x55cb5369a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38742/0 (socket says 172.21.15.63:38742) 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22ce19d640 1 -- 172.21.15.63:0/460697999 learned_addr learned my addr 172.21.15.63:0/460697999 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22cd99c640 1 -- 172.21.15.63:0/460697999 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1186831151 0 0) 0x55cb535477f0 con 0x55cb53698570 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22cd99c640 1 -- 172.21.15.63:0/460697999 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22b8001e10 con 0x55cb53698570 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22ce99e640 1 -- 172.21.15.63:0/460697999 >> v1:172.21.15.16:6789/0 conn(0x55cb53698570 legacy=0x55cb5369a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.822+0000 7f22ce99e640 1 -- 172.21.15.63:0/460697999 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb53555b50 con 0x55cb53695c40 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.824+0000 7f22ce99e640 1 --2- 172.21.15.63:0/460697999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53698030 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22c000a470 tx=0x7f22c001cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11e2cb4a39c0c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.824+0000 7f22cd99c640 1 -- 172.21.15.63:0/460697999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c0057070 con 0x55cb53695c40 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.824+0000 7f22cd99c640 1 -- 172.21.15.63:0/460697999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c0052570 con 0x55cb53695c40 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.824+0000 7f22cd99c640 1 -- 172.21.15.63:0/460697999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c0052870 con 0x55cb53695c40 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.825+0000 7f22d0e98800 1 -- 172.21.15.63:0/460697999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 msgr2=0x55cb53698030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.825+0000 7f22d0e98800 1 --2- 172.21.15.63:0/460697999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53698030 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22c000a470 tx=0x7f22c001cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.825+0000 7f22d0e98800 1 -- 172.21.15.63:0/460697999 shutdown_connections 2024-04-23T19:10:29.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.825+0000 7f22d0e98800 1 --2- 172.21.15.63:0/460697999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53698030 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.825+0000 7f22d0e98800 1 -- 172.21.15.63:0/460697999 >> 172.21.15.63:0/460697999 conn(0x55cb5368f630 msgr2=0x55cb53691a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.825+0000 7f22d0e98800 1 -- 172.21.15.63:0/460697999 shutdown_connections 2024-04-23T19:10:29.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.825+0000 7f22d0e98800 1 -- 172.21.15.63:0/460697999 wait complete. 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22d0e98800 1 Processor -- start 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22d0e98800 1 -- start start 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22d0e98800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53695580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22d0e98800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb535e06f0 con 0x55cb53695c40 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22ce99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53695580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22ce99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53695580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54330/0 (socket says 172.21.15.63:54330) 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22ce99e640 1 -- 172.21.15.63:0/3130583313 learned_addr learned my addr 172.21.15.63:0/3130583313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22ce99e640 1 -- 172.21.15.63:0/3130583313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb53693b40 con 0x55cb53695c40 2024-04-23T19:10:29.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.838+0000 7f22ce99e640 1 --2- 172.21.15.63:0/3130583313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53695580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22c0055040 tx=0x7f22c004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c3230b875acef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f21bce09640 1 -- 172.21.15.63:0/3130583313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c0057040 con 0x55cb53695c40 2024-04-23T19:10:29.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f21bce09640 1 -- 172.21.15.63:0/3130583313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c0052950 con 0x55cb53695c40 2024-04-23T19:10:29.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f21bce09640 1 -- 172.21.15.63:0/3130583313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c0052c70 con 0x55cb53695c40 2024-04-23T19:10:29.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f22d0e98800 1 -- 172.21.15.63:0/3130583313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 msgr2=0x55cb53695580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f22d0e98800 1 --2- 172.21.15.63:0/3130583313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53695580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22c0055040 tx=0x7f22c004c000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f22d0e98800 1 -- 172.21.15.63:0/3130583313 shutdown_connections 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f22d0e98800 1 --2- 172.21.15.63:0/3130583313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb53695580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f22d0e98800 1 -- 172.21.15.63:0/3130583313 >> 172.21.15.63:0/3130583313 conn(0x55cb5368f630 msgr2=0x55cb53781a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f22d0e98800 1 -- 172.21.15.63:0/3130583313 shutdown_connections 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.839+0000 7f22d0e98800 1 -- 172.21.15.63:0/3130583313 wait complete. 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22d0e98800 1 Processor -- start 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22d0e98800 1 -- start start 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22d0e98800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb5378fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22d0e98800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb5369e390 con 0x55cb53695c40 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22ce99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb5378fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22ce99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb5378fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54334/0 (socket says 172.21.15.63:54334) 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22ce99e640 1 -- 172.21.15.63:0/633951463 learned_addr learned my addr 172.21.15.63:0/633951463 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.840+0000 7f22ce99e640 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb53790350 con 0x55cb53695c40 2024-04-23T19:10:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.841+0000 7f22ce99e640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb5378fe10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f22c0052540 tx=0x7f22c0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.841+0000 7f21bb606640 1 -- 172.21.15.63:0/633951463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c0073070 con 0x55cb53695c40 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.841+0000 7f21bb606640 1 -- 172.21.15.63:0/633951463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c005d780 con 0x55cb53695c40 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.841+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb537905e0 con 0x55cb53695c40 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.841+0000 7f21bb606640 1 -- 172.21.15.63:0/633951463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c0057070 con 0x55cb53695c40 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.841+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb53793290 con 0x55cb53695c40 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f21bb606640 1 -- 172.21.15.63:0/633951463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f22c0076070 con 0x55cb53695c40 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f21bb606640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21b4036eb0 0x7f21b4039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f21bb606640 1 -- 172.21.15.63:0/633951463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f22c008e580 con 0x55cb53695c40 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f22ce19d640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21b4036eb0 0x7f21b4039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb53698570 0x55cb53797050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537975d0 con 0x55cb53698570 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f22cf19f640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb53698570 0x55cb53797050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f22ce19d640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21b4036eb0 0x7f21b4039370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22c4000f60 tx=0x7f22c40010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.842+0000 7f22cf19f640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb53698570 0x55cb53797050 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.843+0000 7f22cf19f640 1 -- 172.21.15.63:0/633951463 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f22c800a4a0 con 0x55cb53698570 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.843+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53794600 con 0x55cb53698570 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.843+0000 7f22cf19f640 1 -- 172.21.15.63:0/633951463 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f22c800a4a0 con 0x55cb53698570 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.843+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537993d0 con 0x55cb53698570 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.843+0000 7f22cf19f640 1 -- 172.21.15.63:0/633951463 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f22c800a4a0 con 0x55cb53698570 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.843+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53798710 con 0x55cb53698570 2024-04-23T19:10:29.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.844+0000 7f22cf19f640 1 -- 172.21.15.63:0/633951463 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f22c800a4a0 con 0x55cb53698570 2024-04-23T19:10:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.844+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cb5379a780 0x55cb5379cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.844+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb5379d0f0 con 0x55cb5379a780 2024-04-23T19:10:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.844+0000 7f22ce99e640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cb5379a780 0x55cb5379cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.844+0000 7f22ce99e640 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cb5379a780 0x55cb5379cb70 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22ce99e640 1 -- 172.21.15.63:0/633951463 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f22c0075030 con 0x55cb5379a780 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22d0e98800 1 Processor -- start 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22d0e98800 1 -- start start 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22d0e98800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537ab510 0x55cb537ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22d0e98800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb53586e30 con 0x55cb537ab510 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22cf19f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537ab510 0x55cb537ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22cf19f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537ab510 0x55cb537ab930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54338/0 (socket says 172.21.15.63:54338) 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.845+0000 7f22cf19f640 1 -- 172.21.15.63:0/4176677552 learned_addr learned my addr 172.21.15.63:0/4176677552 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.846+0000 7f22cf19f640 1 -- 172.21.15.63:0/4176677552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb537abe70 con 0x55cb537ab510 2024-04-23T19:10:29.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.846+0000 7f22cf19f640 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537ab510 0x55cb537ab930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55cb53791210 tx=0x7f22c80178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.846+0000 7f21b8e01640 1 -- 172.21.15.63:0/4176677552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c8013440 con 0x55cb537ab510 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.846+0000 7f21b8e01640 1 -- 172.21.15.63:0/4176677552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c80135e0 con 0x55cb537ab510 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.846+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb537ac8e0 con 0x55cb537ab510 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.846+0000 7f21b8e01640 1 -- 172.21.15.63:0/4176677552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c80138c0 con 0x55cb537ab510 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.846+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb537acde0 con 0x55cb537ab510 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.847+0000 7f21b8e01640 1 -- 172.21.15.63:0/4176677552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f22c8011040 con 0x55cb537ab510 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.847+0000 7f21b8e01640 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21a8036e60 0x7f21a8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.847+0000 7f21b8e01640 1 -- 172.21.15.63:0/4176677552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f22c801d070 con 0x55cb537ab510 2024-04-23T19:10:29.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.847+0000 7f22ce99e640 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21a8036e60 0x7f21a8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.847+0000 7f22d0e98800 1 Processor -- start 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.847+0000 7f22ce99e640 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21a8036e60 0x7f21a8039320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22c005b490 tx=0x7f22c0000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.847+0000 7f22d0e98800 1 -- start start 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f22d0e98800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f22d0e98800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb53693e00 con 0x55cb537b88c0 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f22cf19f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f22cf19f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537b8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54350/0 (socket says 172.21.15.63:54350) 2024-04-23T19:10:29.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f22cf19f640 1 -- 172.21.15.63:0/2302240142 learned_addr learned my addr 172.21.15.63:0/2302240142 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f22cf19f640 1 -- 172.21.15.63:0/2302240142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb537b92b0 con 0x55cb537b88c0 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f22cf19f640 1 --2- 172.21.15.63:0/2302240142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537b8ce0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f22c804ccb0 tx=0x7f22c8012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f5c102c4d06833 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f21b27fc640 1 -- 172.21.15.63:0/2302240142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c8012df0 con 0x55cb537b88c0 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f21b27fc640 1 -- 172.21.15.63:0/2302240142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c8001310 con 0x55cb537b88c0 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.848+0000 7f21b27fc640 1 -- 172.21.15.63:0/2302240142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c8001630 con 0x55cb537b88c0 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 -- 172.21.15.63:0/2302240142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 msgr2=0x55cb537b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 --2- 172.21.15.63:0/2302240142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537b8ce0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f22c804ccb0 tx=0x7f22c8012910 comp rx=0 tx=0).stop 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 -- 172.21.15.63:0/2302240142 shutdown_connections 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 --2- 172.21.15.63:0/2302240142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537b8ce0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 -- 172.21.15.63:0/2302240142 >> 172.21.15.63:0/2302240142 conn(0x55cb537b4b90 msgr2=0x55cb537b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 -- 172.21.15.63:0/2302240142 shutdown_connections 2024-04-23T19:10:29.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 -- 172.21.15.63:0/2302240142 wait complete. 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 Processor -- start 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 -- start start 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22d0e98800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb537b9a70 con 0x55cb537b88c0 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22cf19f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22cf19f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537bfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54362/0 (socket says 172.21.15.63:54362) 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.849+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 learned_addr learned my addr 172.21.15.63:0/4149301810 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.850+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb537c02e0 con 0x55cb537b88c0 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.850+0000 7f22cf19f640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537bfda0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f22c8015ac0 tx=0x7f22c804cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.850+0000 7f21b0ff9640 1 -- 172.21.15.63:0/4149301810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c802ae30 con 0x55cb537b88c0 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.850+0000 7f21b0ff9640 1 -- 172.21.15.63:0/4149301810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c8021c60 con 0x55cb537b88c0 2024-04-23T19:10:29.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.850+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb537c0570 con 0x55cb537b88c0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.850+0000 7f21b0ff9640 1 -- 172.21.15.63:0/4149301810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22c802a350 con 0x55cb537b88c0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.850+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb537c3220 con 0x55cb537b88c0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f21b0ff9640 1 -- 172.21.15.63:0/4149301810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f22c8040040 con 0x55cb537b88c0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f21b0ff9640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2198036eb0 0x7f2198039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f21b0ff9640 1 -- 172.21.15.63:0/4149301810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f22c8032070 con 0x55cb537b88c0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f22ce99e640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2198036eb0 0x7f2198039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb537cba60 0x55cb537cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537ce670 con 0x55cb537cba60 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f22ce99e640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2198036eb0 0x7f2198039370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55cb5379d4d0 tx=0x7f22c0083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.851+0000 7f22ce19d640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb537cba60 0x55cb537cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22ce19d640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb537cba60 0x55cb537cdf20 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f22c4009050 con 0x55cb537cba60 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 20 realm 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537caeb0 con 0x55cb537cba60 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f22c4009050 con 0x55cb537cba60 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537cf150 con 0x55cb537cba60 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f22c4009050 con 0x55cb537cba60 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.852+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537cfa20 con 0x55cb537cba60 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f22c4009050 con 0x55cb537cba60 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537cfdc0 con 0x55cb537cba60 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f22c4009050 con 0x55cb537cba60 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22d0e98800 20 searching for the correct realm 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cb537d1ca0 0x55cb537d4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x55cb537d1ca0 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22cf19f640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cb537d1ca0 0x55cb537d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.853+0000 7f22cf19f640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cb537d1ca0 0x55cb537d4090 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22cf19f640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f22c80123a0 0x7f22c80468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x7f22c80123a0 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22ce99e640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f22c80123a0 0x7f22c80468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22ce99e640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f22c80123a0 0x7f22c80468c0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22ce99e640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f22c0078060 0x7f22c008caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.854+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x7f22c0078060 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.855+0000 7f22ce19d640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f22c0078060 0x7f22c008caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.855+0000 7f22ce19d640 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f22c0078060 0x7f22c008caf0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.855+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c4009050 con 0x7f22c0078060 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.855+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x55cb537cba60 2024-04-23T19:10:29.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c4009050 con 0x55cb537cba60 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x55cb537cba60 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x7f22c0078060 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x7f22c0078060 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.856+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x7f22c80123a0 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x55cb537d1ca0 2024-04-23T19:10:29.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x55cb537cba60 2024-04-23T19:10:29.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x7f22c0078060 2024-04-23T19:10:29.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.857+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x55cb537cba60 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4760 con 0x7f22c80123a0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537d1ca0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537d1ca0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.858+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c0078060 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537d1ca0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537d1ca0 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537cba60 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.859+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537cba60 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.860+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.860+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c0078060 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.860+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.860+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537d1ca0 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.860+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.860+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c80123a0 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537cba60 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c0078060 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537d1ca0 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.861+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c0078060 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c80123a0 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537d1ca0 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c0078060 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.862+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x55cb537cba60 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f22c00a9060 con 0x7f22c80123a0 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:29.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d0740 con 0x55cb537cba60 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d0380 con 0x55cb537cba60 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.863+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d10a0 con 0x7f22c0078060 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 20 zone default found 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 4 Realm: () 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 20 zonegroup default 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d4be0 con 0x7f22c0078060 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d6a70 con 0x55cb537d1ca0 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.864+0000 7f22d0e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.865+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.865+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d6e10 con 0x55cb537d1ca0 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.865+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.865+0000 7f22d0e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.865+0000 7f22d0e98800 20 started sync module instance, tier type = 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.865+0000 7f22d0e98800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.865+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d5f00 con 0x7f22c80123a0 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.866+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.866+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94331767448304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d7660 con 0x7f22c80123a0 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.866+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d7eb0 con 0x55cb537cba60 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.867+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94331767448304] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.867+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.867+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94331767452080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d7f10 con 0x55cb537cba60 2024-04-23T19:10:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.867+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d8a00 con 0x55cb537d1ca0 2024-04-23T19:10:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.868+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94331767452080] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.868+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.868+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94331767456976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d89c0 con 0x55cb537d1ca0 2024-04-23T19:10:29.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.868+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d9490 con 0x7f22c80123a0 2024-04-23T19:10:29.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.869+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94331767456976] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.869+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.869+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94331767454208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537d9880 con 0x7f22c80123a0 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.869+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537da2d0 con 0x55cb537cba60 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.870+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.870+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94331767463360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537dab70 con 0x55cb537cba60 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.871+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537db610 con 0x7f22c0078060 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.871+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94331767454208] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.871+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94331767463360] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.871+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.871+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94331767468288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537da5d0 con 0x7f22c0078060 2024-04-23T19:10:29.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.871+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537dbee0 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.872+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94331767468288] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.873+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.873+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94331767459728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537dab90 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.873+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537dc410 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94331767459728] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94331767471840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537dc3d0 con 0x7f22c0078060 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 20 add_watcher() i=0 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 20 add_watcher() i=1 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 20 add_watcher() i=2 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 20 add_watcher() i=3 2024-04-23T19:10:29.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 20 add_watcher() i=4 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 20 add_watcher() i=5 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.874+0000 7f22d0e98800 20 add_watcher() i=6 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.875+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94331767471840] v30'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.875+0000 7f22d0e98800 20 add_watcher() i=7 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.875+0000 7f22d0e98800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.875+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb537df890 con 0x55cb537cba60 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.875+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.875+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94331767470640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537dcd50 con 0x55cb537cba60 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.876+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94331767470640] v30'100 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.876+0000 7f21857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.876+0000 7f22d0e98800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.876+0000 7f22d0e98800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.876+0000 7f22d0e98800 5 note: GC not initialized 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f20fffff640 20 reqs_thread_entry: start 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22d0e98800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22d0e98800 20 Filter name: none 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f20fdffb640 20 reqs_thread_entry: start 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53812dc0 con 0x7f22c80123a0 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.877+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53813650 con 0x55cb537cba60 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53812180 con 0x7f22c80123a0 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f22c0089030 con 0x7f22c80123a0 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 20 rados->read ofs=0 len=0 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53814c90 con 0x55cb537cba60 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:29.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 4.20 4:061ef416:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53814670 con 0x55cb537cba60 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 4.3a 4:5ce4e6b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb538195b0 con 0x55cb537d1ca0 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 4.11 4:88bce1a7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb5381bf00 con 0x7f22c0078060 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.878+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 4.3d 4:bd56d65b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb538195b0 con 0x55cb537d1ca0 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 4.7 4:e0c3751e:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53820d70 con 0x7f22c0078060 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 4.25 4:a745f109:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53823c30 con 0x55cb537d1ca0 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 4.22 4:4491d676:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb538266a0 con 0x55cb537cba60 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 4.3e 4:7f79def2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53829170 con 0x7f22c80123a0 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:74 4.1d 4:b89cda5a:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb5382bf80 con 0x55cb537d1ca0 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 4.16 4:681bb7d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb5382eac0 con 0x55cb537d1ca0 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:76 4.d 4:b3b67914:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55cb53831840 con 0x55cb537d1ca0 2024-04-23T19:10:29.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(69 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(68 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5 [call out=124b] v0'0 uv3 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(73 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c00ae040 con 0x7f22c80123a0 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(75 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(74 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(72 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.879+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(76 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.880+0000 7f21857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:29.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.880+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94331767470640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb53831840 con 0x55cb537cba60 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.881+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94331767470640] v30'101 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.881+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94331767448304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x7f22c80123a0 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.882+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94331767448304] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c00ae040 con 0x7f22c80123a0 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.882+0000 7f22d0e98800 20 remove_watcher() i=0 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.882+0000 7f22d0e98800 2 removed watcher, disabling cache 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.882+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94331767452080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x55cb537cba60 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.884+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94331767452080] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.884+0000 7f22d0e98800 20 remove_watcher() i=1 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.884+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94331767456976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x55cb537d1ca0 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.885+0000 7f22cf19f640 1 -- 172.21.15.63:0/4149301810 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94331767456976] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c8029070 con 0x55cb537d1ca0 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.885+0000 7f22d0e98800 20 remove_watcher() i=2 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.885+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94331767454208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x7f22c80123a0 2024-04-23T19:10:29.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.885+0000 7f22ce99e640 1 -- 172.21.15.63:0/4149301810 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94331767454208] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c00ae040 con 0x7f22c80123a0 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.886+0000 7f22d0e98800 20 remove_watcher() i=3 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.886+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94331767463360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x55cb537cba60 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.886+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94331767463360] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x55cb537cba60 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.887+0000 7f22d0e98800 20 remove_watcher() i=4 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.887+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94331767468288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x7f22c0078060 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.888+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94331767468288] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.888+0000 7f22d0e98800 20 remove_watcher() i=5 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.888+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94331767459728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x7f22c0078060 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.889+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94331767459728] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.889+0000 7f22d0e98800 20 remove_watcher() i=6 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.889+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94331767471840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55cb537fb340 con 0x7f22c0078060 2024-04-23T19:10:29.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.889+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94331767471840] v30'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f22c402a040 con 0x7f22c0078060 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 20 remove_watcher() i=7 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f22c0078060 msgr2=0x7f22c008caf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f22c0078060 0x7f22c008caf0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb537cba60 msgr2=0x55cb537cdf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb537cba60 0x55cb537cdf20 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f22c80123a0 msgr2=0x7f22c80468c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f22c80123a0 0x7f22c80468c0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cb537d1ca0 msgr2=0x55cb537d4090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cb537d1ca0 0x55cb537d4090 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2198036eb0 msgr2=0x7f2198039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2198036eb0 0x7f2198039370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55cb5379d4d0 tx=0x7f22c0083000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 msgr2=0x55cb537bfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22ce19d640 1 -- 172.21.15.63:0/4149301810 reap_dead start 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537bfda0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f22c8015ac0 tx=0x7f22c804cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 shutdown_connections 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4149301810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537b88c0 0x55cb537bfda0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 >> 172.21.15.63:0/4149301810 conn(0x55cb537b4b90 msgr2=0x55cb537b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 shutdown_connections 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4149301810 wait complete. 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21a8036e60 msgr2=0x7f21a8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21a8036e60 0x7f21a8039320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22c005b490 tx=0x7f22c0000bc0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537ab510 msgr2=0x55cb537ab930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537ab510 0x55cb537ab930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55cb53791210 tx=0x7f22c80178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 shutdown_connections 2024-04-23T19:10:29.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21a8036e60 0x7f21a8039320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 --2- 172.21.15.63:0/4176677552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb537ab510 0x55cb537ab930 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 >> 172.21.15.63:0/4176677552 conn(0x55cb5379f5b0 msgr2=0x55cb5379f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 shutdown_connections 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.890+0000 7f22d0e98800 1 -- 172.21.15.63:0/4176677552 wait complete. 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cb5379a780 msgr2=0x55cb5379cb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cb5379a780 0x55cb5379cb70 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb53698570 msgr2=0x55cb53797050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb53698570 0x55cb53797050 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21b4036eb0 msgr2=0x7f21b4039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21b4036eb0 0x7f21b4039370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22c4000f60 tx=0x7f22c40010a0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 msgr2=0x55cb5378fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb5378fe10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f22c0052540 tx=0x7f22c0003020 comp rx=0 tx=0).stop 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 shutdown_connections 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55cb5379a780 0x55cb5379cb70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cb53698570 0x55cb53797050 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f21b4036eb0 0x7f21b4039370 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 --2- 172.21.15.63:0/633951463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb53695c40 0x55cb5378fe10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 >> 172.21.15.63:0/633951463 conn(0x55cb5368f630 msgr2=0x55cb53694360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 shutdown_connections 2024-04-23T19:10:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.891+0000 7f22d0e98800 1 -- 172.21.15.63:0/633951463 wait complete. 2024-04-23T19:10:29.917 INFO:teuthology.orchestra.run.smithi063.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"8815d3af-8342-460f-8220-31ec38570cc3","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","marker":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-23T19:10:29.012224Z","creation_time":"2024-04-23T19:10:29.008577Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":3,"size_actual":4096,"size_utilized":3,"size_kb":1,"size_kb_actual":4,"size_kb_utilized":1,"num_objects":1}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-04-23T19:10:29.917 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:10:29.012224Z', 'creation_time': '2024-04-23T19:10:29.008577Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-23T19:10:29.917 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-23T19:10:29.918 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-23T19:10:29.918 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-04-23T19:10:29.963 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:29.963 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:29.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.963+0000 7f98045af800 1 Processor -- start 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.963+0000 7f98045af800 1 -- start start 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.963+0000 7f98045af800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x564eadad3b90 0x564eadad5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.963+0000 7f98045af800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564ead9857f0 con 0x564eadad64c0 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.963+0000 7f98045af800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x564ead974e20 con 0x564eadad3b90 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.964+0000 7f98020ae640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x564eadad3b90 0x564eadad5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.964+0000 7f98018ad640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x564eadad64c0 0x564eadad88b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38756/0 (socket says 172.21.15.63:38756) 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.964+0000 7f98018ad640 1 -- 172.21.15.63:0/411904074 learned_addr learned my addr 172.21.15.63:0/411904074 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:29.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.964+0000 7f98010ac640 1 -- 172.21.15.63:0/411904074 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511112581 0 0) 0x564ead9857f0 con 0x564eadad64c0 2024-04-23T19:10:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.964+0000 7f98010ac640 1 -- 172.21.15.63:0/411904074 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97f0001e10 con 0x564eadad64c0 2024-04-23T19:10:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.964+0000 7f98020ae640 1 -- 172.21.15.63:0/411904074 >> v1:172.21.15.16:6789/0 conn(0x564eadad64c0 legacy=0x564eadad88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.964+0000 7f98020ae640 1 -- 172.21.15.63:0/411904074 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ead993b50 con 0x564eadad3b90 2024-04-23T19:10:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.966+0000 7f98020ae640 1 --2- 172.21.15.63:0/411904074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad5f80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f97f4005fd0 tx=0x7f97f401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd300e684747798d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.966+0000 7f98010ac640 1 -- 172.21.15.63:0/411904074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97f4052560 con 0x564eadad3b90 2024-04-23T19:10:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.966+0000 7f98010ac640 1 -- 172.21.15.63:0/411904074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f4052700 con 0x564eadad3b90 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.966+0000 7f98010ac640 1 -- 172.21.15.63:0/411904074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97f4052a60 con 0x564eadad3b90 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.967+0000 7f98045af800 1 -- 172.21.15.63:0/411904074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 msgr2=0x564eadad5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.967+0000 7f98045af800 1 --2- 172.21.15.63:0/411904074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad5f80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f97f4005fd0 tx=0x7f97f401cbe0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.967+0000 7f98045af800 1 -- 172.21.15.63:0/411904074 shutdown_connections 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.967+0000 7f98045af800 1 --2- 172.21.15.63:0/411904074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad5f80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.967+0000 7f98045af800 1 -- 172.21.15.63:0/411904074 >> 172.21.15.63:0/411904074 conn(0x564eadacd600 msgr2=0x564eadacfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.967+0000 7f98045af800 1 -- 172.21.15.63:0/411904074 shutdown_connections 2024-04-23T19:10:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.967+0000 7f98045af800 1 -- 172.21.15.63:0/411904074 wait complete. 2024-04-23T19:10:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.979+0000 7f98045af800 1 Processor -- start 2024-04-23T19:10:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.979+0000 7f98045af800 1 -- start start 2024-04-23T19:10:29.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.979+0000 7f98045af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.979+0000 7f98045af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564eada1e6f0 con 0x564eadad3b90 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.980+0000 7f98020ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.980+0000 7f98020ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54374/0 (socket says 172.21.15.63:54374) 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.980+0000 7f98020ae640 1 -- 172.21.15.63:0/4278138737 learned_addr learned my addr 172.21.15.63:0/4278138737 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.980+0000 7f98020ae640 1 -- 172.21.15.63:0/4278138737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eadad1a90 con 0x564eadad3b90 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.980+0000 7f98020ae640 1 --2- 172.21.15.63:0/4278138737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad34d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f97f4007000 tx=0x7f97f404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd9041421d4cf78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.981+0000 7f96f0608640 1 -- 172.21.15.63:0/4278138737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97f40528b0 con 0x564eadad3b90 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.981+0000 7f96f0608640 1 -- 172.21.15.63:0/4278138737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f4052a50 con 0x564eadad3b90 2024-04-23T19:10:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.981+0000 7f96f0608640 1 -- 172.21.15.63:0/4278138737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97f4052d80 con 0x564eadad3b90 2024-04-23T19:10:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.981+0000 7f98045af800 1 -- 172.21.15.63:0/4278138737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 msgr2=0x564eadad34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.981+0000 7f98045af800 1 --2- 172.21.15.63:0/4278138737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad34d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f97f4007000 tx=0x7f97f404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 -- 172.21.15.63:0/4278138737 shutdown_connections 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 --2- 172.21.15.63:0/4278138737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadad34d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 -- 172.21.15.63:0/4278138737 >> 172.21.15.63:0/4278138737 conn(0x564eadacd600 msgr2=0x564eadbbf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 -- 172.21.15.63:0/4278138737 shutdown_connections 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 -- 172.21.15.63:0/4278138737 wait complete. 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 Processor -- start 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 -- start start 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadbcde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98045af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564eadada270 con 0x564eadad3b90 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98020ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadbcde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98020ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadbcde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54380/0 (socket says 172.21.15.63:54380) 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.982+0000 7f98020ae640 1 -- 172.21.15.63:0/1684206858 learned_addr learned my addr 172.21.15.63:0/1684206858 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.983+0000 7f98020ae640 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eadbce360 con 0x564eadad3b90 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.983+0000 7f98020ae640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadbcde20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f97f4007000 tx=0x7f97f4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.983+0000 7f96eee05640 1 -- 172.21.15.63:0/1684206858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97f4075070 con 0x564eadad3b90 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.983+0000 7f96eee05640 1 -- 172.21.15.63:0/1684206858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f405cb80 con 0x564eadad3b90 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.983+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564eadbce5f0 con 0x564eadad3b90 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.983+0000 7f96eee05640 1 -- 172.21.15.63:0/1684206858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97f4070410 con 0x564eadad3b90 2024-04-23T19:10:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.983+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eadbd12a0 con 0x564eadad3b90 2024-04-23T19:10:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.984+0000 7f96eee05640 1 -- 172.21.15.63:0/1684206858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f97f4078070 con 0x564eadad3b90 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.984+0000 7f96eee05640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96e8036eb0 0x7f96e8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.984+0000 7f96eee05640 1 -- 172.21.15.63:0/1684206858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f97f406d070 con 0x564eadad3b90 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.984+0000 7f98018ad640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96e8036eb0 0x7f96e8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.984+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadad64c0 0x564eadbd5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.984+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadbd55e0 con 0x564eadad64c0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.984+0000 7f98028af640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadad64c0 0x564eadbd5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.985+0000 7f98018ad640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96e8036eb0 0x7f96e8039370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97ec000f60 tx=0x7f97ec0010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.985+0000 7f98028af640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadad64c0 0x564eadbd5060 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.985+0000 7f98028af640 1 -- 172.21.15.63:0/1684206858 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97fc0083e0 con 0x564eadad64c0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.985+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadbd25f0 con 0x564eadad64c0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.985+0000 7f98028af640 1 -- 172.21.15.63:0/1684206858 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f97fc0083e0 con 0x564eadad64c0 2024-04-23T19:10:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.985+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadbd7450 con 0x564eadad64c0 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.986+0000 7f98028af640 1 -- 172.21.15.63:0/1684206858 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f97fc0083e0 con 0x564eadad64c0 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.986+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadbd6790 con 0x564eadad64c0 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.986+0000 7f98028af640 1 -- 172.21.15.63:0/1684206858 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f97fc0083e0 con 0x564eadad64c0 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.986+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564eadbd8800 0x564eadbdabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.986+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadbdb170 con 0x564eadbd8800 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.986+0000 7f98020ae640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564eadbd8800 0x564eadbdabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.987+0000 7f98020ae640 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564eadbd8800 0x564eadbdabf0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.987+0000 7f98020ae640 1 -- 172.21.15.63:0/1684206858 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f97f4077030 con 0x564eadbd8800 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.987+0000 7f98045af800 1 Processor -- start 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.987+0000 7f98045af800 1 -- start start 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.987+0000 7f98045af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbe9590 0x564eadbe99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.987+0000 7f98045af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ead9c4e30 con 0x564eadbe9590 2024-04-23T19:10:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.987+0000 7f98028af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbe9590 0x564eadbe99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f98028af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbe9590 0x564eadbe99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54396/0 (socket says 172.21.15.63:54396) 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f98028af640 1 -- 172.21.15.63:0/2728412414 learned_addr learned my addr 172.21.15.63:0/2728412414 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f98028af640 1 -- 172.21.15.63:0/2728412414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eadbe9ef0 con 0x564eadbe9590 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f98028af640 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbe9590 0x564eadbe99b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x564eadbcf220 tx=0x7f97fc00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f96e7fff640 1 -- 172.21.15.63:0/2728412414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc01c070 con 0x564eadbe9590 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f96e7fff640 1 -- 172.21.15.63:0/2728412414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc00bdc0 con 0x564eadbe9590 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564eadbea960 con 0x564eadbe9590 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f96e7fff640 1 -- 172.21.15.63:0/2728412414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc018540 con 0x564eadbe9590 2024-04-23T19:10:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.988+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eadbeae60 con 0x564eadbe9590 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.989+0000 7f96e7fff640 1 -- 172.21.15.63:0/2728412414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f97fc01f070 con 0x564eadbe9590 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.989+0000 7f96e7fff640 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96dc036eb0 0x7f96dc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.989+0000 7f96e7fff640 1 -- 172.21.15.63:0/2728412414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f97fc04e950 con 0x564eadbe9590 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.989+0000 7f98020ae640 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96dc036eb0 0x7f96dc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.989+0000 7f98045af800 1 Processor -- start 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98045af800 1 -- start start 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98020ae640 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96dc036eb0 0x7f96dc039370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97f406bfd0 tx=0x7f97f4051f40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98045af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbf6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98045af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564eadad1db0 con 0x564eadbf6870 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98028af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbf6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98028af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbf6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54400/0 (socket says 172.21.15.63:54400) 2024-04-23T19:10:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98028af640 1 -- 172.21.15.63:0/3924136714 learned_addr learned my addr 172.21.15.63:0/3924136714 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:29.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98028af640 1 -- 172.21.15.63:0/3924136714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eadbf7260 con 0x564eadbf6870 2024-04-23T19:10:29.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f98028af640 1 --2- 172.21.15.63:0/3924136714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbf6c90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97fc04cca0 tx=0x7f97fc014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787ebd0c2f61cef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:29.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f96e5ffb640 1 -- 172.21.15.63:0/3924136714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc026070 con 0x564eadbf6870 2024-04-23T19:10:29.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f96e5ffb640 1 -- 172.21.15.63:0/3924136714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc029480 con 0x564eadbf6870 2024-04-23T19:10:29.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.990+0000 7f96e5ffb640 1 -- 172.21.15.63:0/3924136714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc029830 con 0x564eadbf6870 2024-04-23T19:10:30.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 -- 172.21.15.63:0/3924136714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 msgr2=0x564eadbf6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 --2- 172.21.15.63:0/3924136714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbf6c90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97fc04cca0 tx=0x7f97fc014690 comp rx=0 tx=0).stop 2024-04-23T19:10:30.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 -- 172.21.15.63:0/3924136714 shutdown_connections 2024-04-23T19:10:30.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 --2- 172.21.15.63:0/3924136714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbf6c90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 -- 172.21.15.63:0/3924136714 >> 172.21.15.63:0/3924136714 conn(0x564eadbf2bd0 msgr2=0x564eadbf0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 -- 172.21.15.63:0/3924136714 shutdown_connections 2024-04-23T19:10:30.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 -- 172.21.15.63:0/3924136714 wait complete. 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 Processor -- start 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 -- start start 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbfde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.991+0000 7f98045af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564eadbf79d0 con 0x564eadbf6870 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f98028af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbfde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f98028af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbfde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54404/0 (socket says 172.21.15.63:54404) 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 learned_addr learned my addr 172.21.15.63:0/3978241360 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eadbfe3b0 con 0x564eadbf6870 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f98028af640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbfde70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f97fc04cca0 tx=0x7f97fc04cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f96d3fff640 1 -- 172.21.15.63:0/3978241360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc0164d0 con 0x564eadbf6870 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f96d3fff640 1 -- 172.21.15.63:0/3978241360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc016670 con 0x564eadbf6870 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564eadbfe640 con 0x564eadbf6870 2024-04-23T19:10:30.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f96d3fff640 1 -- 172.21.15.63:0/3978241360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc0186e0 con 0x564eadbf6870 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.992+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eadbfe640 con 0x564eadbf6870 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.993+0000 7f96d3fff640 1 -- 172.21.15.63:0/3978241360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f97fc018880 con 0x564eadbf6870 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.993+0000 7f96d3fff640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96cc036e60 0x7f96cc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.993+0000 7f96d3fff640 1 -- 172.21.15.63:0/3978241360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f97fc031070 con 0x564eadbf6870 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.993+0000 7f98020ae640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96cc036e60 0x7f96cc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.993+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.993+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadc09710 0x564eadc0bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.993+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc0c320 con 0x564eadc09710 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98018ad640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadc09710 0x564eadc0bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98020ae640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96cc036e60 0x7f96cc039320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f97f4005bb0 tx=0x7f97f408b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98018ad640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadc09710 0x564eadc0bbd0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97ec009050 con 0x564eadc09710 2024-04-23T19:10:30.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 20 realm 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc08b60 con 0x564eadc09710 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97ec009050 con 0x564eadc09710 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.994+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc0ce00 con 0x564eadc09710 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97ec009050 con 0x564eadc09710 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc0d6d0 con 0x564eadc09710 2024-04-23T19:10:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f97ec009050 con 0x564eadc09710 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc0da70 con 0x564eadc09710 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f97ec009050 con 0x564eadc09710 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 20 searching for the correct realm 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564eadc0f950 0x564eadc11d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x564eadc0f950 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.995+0000 7f98028af640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564eadc0f950 0x564eadc11d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.996+0000 7f98028af640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564eadc0f950 0x564eadc11d40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.996+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.996+0000 7f98028af640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f97fc021070 0x7f97fc04d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.996+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x7f97fc021070 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.996+0000 7f98020ae640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f97fc021070 0x7f97fc04d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.996+0000 7f98020ae640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f97fc021070 0x7f97fc04d290 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.997+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.997+0000 7f98020ae640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f97f407a060 0x7f97f405b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.997+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x7f97f407a060 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.997+0000 7f98018ad640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f97f407a060 0x7f97f405b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.998+0000 7f98018ad640 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f97f407a060 0x7f97f405b810 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.998+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec009050 con 0x7f97f407a060 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.998+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x564eadc09710 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.998+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec009050 con 0x564eadc09710 2024-04-23T19:10:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.998+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x564eadc09710 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.998+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.998+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x7f97f407a060 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x7f97f407a060 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x7f97fc021070 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x564eadc0f950 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:29.999+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x564eadc09710 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x7f97f407a060 2024-04-23T19:10:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x564eadc09710 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12410 con 0x7f97fc021070 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.000+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc0f950 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc0f950 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97f407a060 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc0f950 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.001+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc0f950 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc09710 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc09710 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97f407a060 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.002+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc0f950 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.003+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.003+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97fc021070 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.003+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.003+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc09710 2024-04-23T19:10:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.003+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.003+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97f407a060 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc0f950 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97f407a060 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97fc021070 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.004+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc0f950 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.005+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.005+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97f407a060 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.005+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.005+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x564eadc09710 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.005+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.005+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f97f40a8060 con 0x7f97fc021070 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc0e3f0 con 0x564eadc09710 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc0ec90 con 0x564eadc09710 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc0e030 con 0x7f97f407a060 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 zone default found 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 4 Realm: () 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 zonegroup default 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.006+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc12890 con 0x7f97f407a060 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc14720 con 0x564eadc0f950 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc14ac0 con 0x564eadc0f950 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 20 started sync module instance, tier type = 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.007+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc13b60 con 0x7f97fc021070 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.008+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.008+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94895922560896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc137c0 con 0x7f97fc021070 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.008+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc15460 con 0x564eadc09710 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.009+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94895922560896] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.010+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.010+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94895922563776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc15880 con 0x564eadc09710 2024-04-23T19:10:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.010+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc162a0 con 0x564eadc0f950 2024-04-23T19:10:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.011+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94895922563776] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.011+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.011+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94895922568592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc16260 con 0x564eadc0f950 2024-04-23T19:10:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.011+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc16d60 con 0x7f97fc021070 2024-04-23T19:10:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.012+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94895922568592] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.012+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.012+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94895922571344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc17180 con 0x7f97fc021070 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.012+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc17bc0 con 0x564eadc09710 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.013+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.013+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94895922574992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc17b80 con 0x564eadc09710 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.013+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc18b70 con 0x7f97f407a060 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.013+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94895922571344] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.014+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.014+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94895922579072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc18b70 con 0x7f97f407a060 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.014+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc195b0 con 0x7f97f407a060 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.014+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94895922574992] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.015+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94895922579072] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.015+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.015+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94895922564832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc19100 con 0x7f97f407a060 2024-04-23T19:10:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.015+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc19ac0 con 0x7f97f407a060 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94895922564832] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94895922577328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc19ac0 con 0x7f97f407a060 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 20 add_watcher() i=0 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 20 add_watcher() i=1 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 20 add_watcher() i=2 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 20 add_watcher() i=3 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 20 add_watcher() i=4 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 20 add_watcher() i=5 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.016+0000 7f98045af800 20 add_watcher() i=6 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.017+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94895922577328] v30'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.017+0000 7f98045af800 20 add_watcher() i=7 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.017+0000 7f98045af800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.017+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc1cde0 con 0x564eadc09710 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.018+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.018+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94895922595872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc19c30 con 0x564eadc09710 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94895922595872] v30'102 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f96b8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f98045af800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f98045af800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f98045af800 5 note: GC not initialized 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f96337fe640 20 reqs_thread_entry: start 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f98045af800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f98045af800 20 Filter name: none 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.019+0000 7f96317fa640 20 reqs_thread_entry: start 2024-04-23T19:10:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc4fe80 con 0x564eadc09710 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98045af800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc51270 con 0x7f97fc021070 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.020+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc51270 con 0x564eadc09710 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 4.20 4:061ef416:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc52bf0 con 0x564eadc09710 2024-04-23T19:10:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 4.3a 4:5ce4e6b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc558a0 con 0x564eadc0f950 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 4.11 4:88bce1a7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc58260 con 0x7f97f407a060 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 4.3d 4:bd56d65b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc58260 con 0x564eadc0f950 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 4.7 4:e0c3751e:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc5d2d0 con 0x7f97f407a060 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 4.25 4:a745f109:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc5fc60 con 0x564eadc0f950 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 4.22 4:4491d676:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc625b0 con 0x564eadc09710 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 4.3e 4:7f79def2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc65320 con 0x7f97fc021070 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 4.1d 4:b89cda5a:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc680d0 con 0x564eadc0f950 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:74 4.16 4:681bb7d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc6aa60 con 0x564eadc0f950 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 4.d 4:b3b67914:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x564eadc0f950 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.021+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(67 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(72 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(71 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(70 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5 [call out=124b] v0'0 uv3 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(73 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(74 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(75 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.022+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x7f97fc021070 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.023+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(76 foo.buckets [call] v30'15 uv15 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.023+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x7f97fc021070 2024-04-23T19:10:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.024+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(77 foo.buckets [call] v30'16 uv16 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.024+0000 7f98045af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.024+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x564eadc09710 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.024+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(78 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.024+0000 7f98045af800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.024+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x7f97fc021070 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.024+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv16 ondisk = 0) ==== 155+0+58 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.025+0000 7f96b8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.025+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94895922595872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x564eadc09710 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.026+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 94895922595872] v30'103 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.026+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94895922560896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x7f97fc021070 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.027+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 94895922560896] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.027+0000 7f98045af800 20 remove_watcher() i=0 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.027+0000 7f98045af800 2 removed watcher, disabling cache 2024-04-23T19:10:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.027+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94895922563776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x564eadc09710 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.028+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 94895922563776] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.028+0000 7f98045af800 20 remove_watcher() i=1 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.028+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94895922568592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x564eadc0f950 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.029+0000 7f98028af640 1 -- 172.21.15.63:0/3978241360 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 94895922568592] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97fc03b020 con 0x564eadc0f950 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.029+0000 7f98045af800 20 remove_watcher() i=2 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.029+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94895922571344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc15d90 con 0x7f97fc021070 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.030+0000 7f98020ae640 1 -- 172.21.15.63:0/3978241360 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 94895922571344] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97f4099070 con 0x7f97fc021070 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.030+0000 7f98045af800 20 remove_watcher() i=3 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.030+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94895922574992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x564eadc09710 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.031+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 94895922574992] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x564eadc09710 2024-04-23T19:10:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.031+0000 7f98045af800 20 remove_watcher() i=4 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.031+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94895922579072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc16850 con 0x7f97f407a060 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.032+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 94895922579072] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.032+0000 7f98045af800 20 remove_watcher() i=5 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.032+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94895922564832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc15d90 con 0x7f97f407a060 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.033+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 94895922564832] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.033+0000 7f98045af800 20 remove_watcher() i=6 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.033+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94895922577328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x564eadc50a30 con 0x7f97f407a060 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 94895922577328] v30'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97ec02a040 con 0x7f97f407a060 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 20 remove_watcher() i=7 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f97f407a060 msgr2=0x7f97f405b810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f97f407a060 0x7f97f405b810 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadc09710 msgr2=0x564eadc0bbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadc09710 0x564eadc0bbd0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f97fc021070 msgr2=0x7f97fc04d290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f97fc021070 0x7f97fc04d290 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564eadc0f950 msgr2=0x564eadc11d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x564eadc0f950 0x564eadc11d40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96cc036e60 msgr2=0x7f96cc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96cc036e60 0x7f96cc039320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f97f4005bb0 tx=0x7f97f408b000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 msgr2=0x564eadbfde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98018ad640 1 -- 172.21.15.63:0/3978241360 reap_dead start 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.034+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbfde70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f97fc04cca0 tx=0x7f97fc04cc30 comp rx=0 tx=0).stop 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 shutdown_connections 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/3978241360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbf6870 0x564eadbfde70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 >> 172.21.15.63:0/3978241360 conn(0x564eadbf2bd0 msgr2=0x564eadbf0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 shutdown_connections 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/3978241360 wait complete. 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96dc036eb0 msgr2=0x7f96dc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96dc036eb0 0x7f96dc039370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97f406bfd0 tx=0x7f97f4051f40 comp rx=0 tx=0).stop 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbe9590 msgr2=0x564eadbe99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbe9590 0x564eadbe99b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x564eadbcf220 tx=0x7f97fc00ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 shutdown_connections 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96dc036eb0 0x7f96dc039370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/2728412414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadbe9590 0x564eadbe99b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 >> 172.21.15.63:0/2728412414 conn(0x564eadbdd630 msgr2=0x564eadbdda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 shutdown_connections 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/2728412414 wait complete. 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564eadbd8800 msgr2=0x564eadbdabf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564eadbd8800 0x564eadbdabf0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadad64c0 msgr2=0x564eadbd5060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadad64c0 0x564eadbd5060 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96e8036eb0 msgr2=0x7f96e8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.035+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96e8036eb0 0x7f96e8039370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97ec000f60 tx=0x7f97ec0010a0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 msgr2=0x564eadbcde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadbcde20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f97f4007000 tx=0x7f97f4003330 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 shutdown_connections 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x564eadbd8800 0x564eadbdabf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564eadad64c0 0x564eadbd5060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f96e8036eb0 0x7f96e8039370 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 --2- 172.21.15.63:0/1684206858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564eadad3b90 0x564eadbcde20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 >> 172.21.15.63:0/1684206858 conn(0x564eadacd600 msgr2=0x564eadad2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 shutdown_connections 2024-04-23T19:10:30.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.036+0000 7f98045af800 1 -- 172.21.15.63:0/1684206858 wait complete. 2024-04-23T19:10:30.052 INFO:teuthology.orchestra.run.smithi063.stdout:{"stats":{"size":3,"size_actual":4096,"size_kb":1,"size_kb_actual":4,"num_objects":1},"last_stats_sync":"2024-04-23T19:10:30.023631Z","last_stats_update":"2024-04-23T19:10:30.022640Z"} 2024-04-23T19:10:30.052 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-04-23T19:10:30.023631Z', 'last_stats_update': '2024-04-23T19:10:30.022640Z'} 2024-04-23T19:10:30.057 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:30.057 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-23T19:10:30.057 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-23T19:10:30.057 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-04-23T19:10:30.103 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:30.103 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:30.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.103+0000 7f7d6e013800 1 Processor -- start 2024-04-23T19:10:30.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.103+0000 7f7d6e013800 1 -- start start 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d6e013800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56347a6534a0 0x56347a655890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d6e013800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56347a5027f0 con 0x56347a64f010 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d6e013800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x56347a4f1e20 con 0x56347a6534a0 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d68e13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56347a6534a0 0x56347a655890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d69614640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x56347a64f010 0x56347a64f430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38770/0 (socket says 172.21.15.63:38770) 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d69614640 1 -- 172.21.15.63:0/860968944 learned_addr learned my addr 172.21.15.63:0/860968944 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d68e13640 1 -- 172.21.15.63:0/860968944 >> v1:172.21.15.16:6789/0 conn(0x56347a64f010 legacy=0x56347a64f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.104+0000 7f7d68e13640 1 -- 172.21.15.63:0/860968944 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347a510b50 con 0x56347a6534a0 2024-04-23T19:10:30.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.106+0000 7f7d68e13640 1 --2- 172.21.15.63:0/860968944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a6534a0 0x56347a655890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000a470 tx=0x7f7d6001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35be0cbfc97a411 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.106+0000 7f7d5bfff640 1 -- 172.21.15.63:0/860968944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d60057070 con 0x56347a6534a0 2024-04-23T19:10:30.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.106+0000 7f7d5bfff640 1 -- 172.21.15.63:0/860968944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d60052570 con 0x56347a6534a0 2024-04-23T19:10:30.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.106+0000 7f7d5bfff640 1 -- 172.21.15.63:0/860968944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d60052870 con 0x56347a6534a0 2024-04-23T19:10:30.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.107+0000 7f7d6e013800 1 -- 172.21.15.63:0/860968944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a6534a0 msgr2=0x56347a655890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.107+0000 7f7d6e013800 1 --2- 172.21.15.63:0/860968944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a6534a0 0x56347a655890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000a470 tx=0x7f7d6001ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.107+0000 7f7d6e013800 1 -- 172.21.15.63:0/860968944 shutdown_connections 2024-04-23T19:10:30.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.107+0000 7f7d6e013800 1 --2- 172.21.15.63:0/860968944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a6534a0 0x56347a655890 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.107+0000 7f7d6e013800 1 -- 172.21.15.63:0/860968944 >> 172.21.15.63:0/860968944 conn(0x56347a64a630 msgr2=0x56347a64ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.107+0000 7f7d6e013800 1 -- 172.21.15.63:0/860968944 shutdown_connections 2024-04-23T19:10:30.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.107+0000 7f7d6e013800 1 -- 172.21.15.63:0/860968944 wait complete. 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.119+0000 7f7d6e013800 1 Processor -- start 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.119+0000 7f7d6e013800 1 -- start start 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.119+0000 7f7d6e013800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a5c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7d6e013800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56347a59b6f0 con 0x56347a64f010 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7d69614640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a5c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7d69614640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a5c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54420/0 (socket says 172.21.15.63:54420) 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7d69614640 1 -- 172.21.15.63:0/137914736 learned_addr learned my addr 172.21.15.63:0/137914736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7d69614640 1 -- 172.21.15.63:0/137914736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347a7440c0 con 0x56347a64f010 2024-04-23T19:10:30.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7d69614640 1 --2- 172.21.15.63:0/137914736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a5c30e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c001e10 tx=0x7f7d5c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3c8709589942d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7c57606640 1 -- 172.21.15.63:0/137914736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d5c012020 con 0x56347a64f010 2024-04-23T19:10:30.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7c57606640 1 -- 172.21.15.63:0/137914736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c00e5a0 con 0x56347a64f010 2024-04-23T19:10:30.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.120+0000 7f7c57606640 1 -- 172.21.15.63:0/137914736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d5c00e8e0 con 0x56347a64f010 2024-04-23T19:10:30.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 -- 172.21.15.63:0/137914736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 msgr2=0x56347a5c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 --2- 172.21.15.63:0/137914736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a5c30e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c001e10 tx=0x7f7d5c00b780 comp rx=0 tx=0).stop 2024-04-23T19:10:30.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 -- 172.21.15.63:0/137914736 shutdown_connections 2024-04-23T19:10:30.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 --2- 172.21.15.63:0/137914736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a5c30e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 -- 172.21.15.63:0/137914736 >> 172.21.15.63:0/137914736 conn(0x56347a64a630 msgr2=0x56347a73bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 -- 172.21.15.63:0/137914736 shutdown_connections 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 -- 172.21.15.63:0/137914736 wait complete. 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.121+0000 7f7d6e013800 1 Processor -- start 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d6e013800 1 -- start start 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d6e013800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a748fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d6e013800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56347a656160 con 0x56347a64f010 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d69614640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a748fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d69614640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a748fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54430/0 (socket says 172.21.15.63:54430) 2024-04-23T19:10:30.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d69614640 1 -- 172.21.15.63:0/3812036084 learned_addr learned my addr 172.21.15.63:0/3812036084 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d69614640 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347a7494e0 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.122+0000 7f7d69614640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a748fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00b7c0 tx=0x7f7d5c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.123+0000 7f7c55e03640 1 -- 172.21.15.63:0/3812036084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d5c012030 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.123+0000 7f7c55e03640 1 -- 172.21.15.63:0/3812036084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c001470 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.123+0000 7f7c55e03640 1 -- 172.21.15.63:0/3812036084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d5c01a8d0 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.123+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56347a749770 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.123+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347a74c3d0 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.123+0000 7f7c55e03640 1 -- 172.21.15.63:0/3812036084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7d5c015020 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7c55e03640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c50036e60 0x7f7c50039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7c55e03640 1 -- 172.21.15.63:0/3812036084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7d5c020030 con 0x56347a64f010 2024-04-23T19:10:30.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d68e13640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c50036e60 0x7f7c50039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a6534a0 0x56347a750190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a750760 con 0x56347a6534a0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d69e15640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a6534a0 0x56347a750190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d68e13640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c50036e60 0x7f7c50039320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d60054000 tx=0x7f7d6004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d69e15640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a6534a0 0x56347a750190 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d69e15640 1 -- 172.21.15.63:0/3812036084 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d6400a4a0 con 0x56347a6534a0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.124+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a74d760 con 0x56347a6534a0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.125+0000 7f7d69e15640 1 -- 172.21.15.63:0/3812036084 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7d6400a4a0 con 0x56347a6534a0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.125+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a752560 con 0x56347a6534a0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.125+0000 7f7d69e15640 1 -- 172.21.15.63:0/3812036084 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7d6400a4a0 con 0x56347a6534a0 2024-04-23T19:10:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.125+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a7518a0 con 0x56347a6534a0 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.125+0000 7f7d69e15640 1 -- 172.21.15.63:0/3812036084 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7d6400a4a0 con 0x56347a6534a0 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.125+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56347a753910 0x56347a755d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.125+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a756280 con 0x56347a753910 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.126+0000 7f7d69614640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56347a753910 0x56347a755d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.126+0000 7f7d69614640 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56347a753910 0x56347a755d00 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.126+0000 7f7d69614640 1 -- 172.21.15.63:0/3812036084 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x56347a753910 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.126+0000 7f7d6e013800 1 Processor -- start 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d6e013800 1 -- start start 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d6e013800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a741c50 0x56347a7400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d6e013800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56347a541e30 con 0x56347a741c50 2024-04-23T19:10:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d69e15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a741c50 0x56347a7400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d69e15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a741c50 0x56347a7400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54440/0 (socket says 172.21.15.63:54440) 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d69e15640 1 -- 172.21.15.63:0/339848323 learned_addr learned my addr 172.21.15.63:0/339848323 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d69e15640 1 -- 172.21.15.63:0/339848323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347a740600 con 0x56347a741c50 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7d69e15640 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a741c50 0x56347a7400c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56347a750b20 tx=0x7f7d640178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.127+0000 7f7c4effd640 1 -- 172.21.15.63:0/339848323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d64013440 con 0x56347a741c50 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.128+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56347a740890 con 0x56347a741c50 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.128+0000 7f7c4effd640 1 -- 172.21.15.63:0/339848323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d640135e0 con 0x56347a741c50 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.128+0000 7f7c4effd640 1 -- 172.21.15.63:0/339848323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d640138c0 con 0x56347a741c50 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.128+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347a740d90 con 0x56347a741c50 2024-04-23T19:10:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.128+0000 7f7c4effd640 1 -- 172.21.15.63:0/339848323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7d64011040 con 0x56347a741c50 2024-04-23T19:10:30.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7c4effd640 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c44036e60 0x7f7c44039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7c4effd640 1 -- 172.21.15.63:0/339848323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7d6401d070 con 0x56347a741c50 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d69614640 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c44036e60 0x7f7c44039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d6e013800 1 Processor -- start 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d6e013800 1 -- start start 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d69614640 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c44036e60 0x7f7c44039320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c04b550 tx=0x7f7d5c04b1c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d6e013800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a773fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d6e013800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56347a744410 con 0x56347a773bb0 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d69e15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a773fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d69e15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a773fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54442/0 (socket says 172.21.15.63:54442) 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d69e15640 1 -- 172.21.15.63:0/835292620 learned_addr learned my addr 172.21.15.63:0/835292620 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.129+0000 7f7d69e15640 1 -- 172.21.15.63:0/835292620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347a774580 con 0x56347a773bb0 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d69e15640 1 --2- 172.21.15.63:0/835292620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a773fb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d640013e0 tx=0x7f7d64012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712cef24b55e8a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7c4cff9640 1 -- 172.21.15.63:0/835292620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d64043070 con 0x56347a773bb0 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7c4cff9640 1 -- 172.21.15.63:0/835292620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6402ac50 con 0x56347a773bb0 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7c4cff9640 1 -- 172.21.15.63:0/835292620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d640014f0 con 0x56347a773bb0 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d6e013800 1 -- 172.21.15.63:0/835292620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 msgr2=0x56347a773fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d6e013800 1 --2- 172.21.15.63:0/835292620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a773fb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d640013e0 tx=0x7f7d64012c90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d6e013800 1 -- 172.21.15.63:0/835292620 shutdown_connections 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d6e013800 1 --2- 172.21.15.63:0/835292620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a773fb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d6e013800 1 -- 172.21.15.63:0/835292620 >> 172.21.15.63:0/835292620 conn(0x56347a76fe80 msgr2=0x56347a76dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d6e013800 1 -- 172.21.15.63:0/835292620 shutdown_connections 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.130+0000 7f7d6e013800 1 -- 172.21.15.63:0/835292620 wait complete. 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d6e013800 1 Processor -- start 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d6e013800 1 -- start start 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d6e013800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a77b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d6e013800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56347a774d30 con 0x56347a773bb0 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d69e15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a77b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d69e15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a77b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54444/0 (socket says 172.21.15.63:54444) 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 learned_addr learned my addr 172.21.15.63:0/3139160579 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347a77b610 con 0x56347a773bb0 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7d69e15640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a77b0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7d6404cb80 tx=0x7f7d6404cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.131+0000 7f7c3affd640 1 -- 172.21.15.63:0/3139160579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d64043030 con 0x56347a773bb0 2024-04-23T19:10:30.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.132+0000 7f7c3affd640 1 -- 172.21.15.63:0/3139160579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6402ae00 con 0x56347a773bb0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.132+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56347a77b8a0 con 0x56347a773bb0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.132+0000 7f7c3affd640 1 -- 172.21.15.63:0/3139160579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d640126e0 con 0x56347a773bb0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.132+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347a77e550 con 0x56347a773bb0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.132+0000 7f7c3affd640 1 -- 172.21.15.63:0/3139160579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7d6404e110 con 0x56347a773bb0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.132+0000 7f7c3affd640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c34036eb0 0x7f7c34039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.132+0000 7f7c3affd640 1 -- 172.21.15.63:0/3139160579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7d64044070 con 0x56347a773bb0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.133+0000 7f7d69614640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c34036eb0 0x7f7c34039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.133+0000 7f7d69614640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c34036eb0 0x7f7c34039370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c04b2f0 tx=0x7f7d5c033000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.133+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.133+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a786db0 0x56347a789270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.133+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a7899c0 con 0x56347a786db0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.133+0000 7f7d68e13640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a786db0 0x56347a789270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.133+0000 7f7d68e13640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a786db0 0x56347a789270 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 realm 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a789d90 con 0x56347a786db0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:30.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78a660 con 0x56347a786db0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78aa00 con 0x56347a786db0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.134+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78b250 con 0x56347a786db0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.135+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.135+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.135+0000 7f7d6e013800 20 searching for the correct realm 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.135+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56347a78cfc0 0x56347a78f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.135+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.135+0000 7f7d69e15640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56347a78cfc0 0x56347a78f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.135+0000 7f7d69e15640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56347a78cfc0 0x56347a78f3b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69e15640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7d6401bba0 0x7f7d640089b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d6401bba0 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69614640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7d6401bba0 0x7f7d640089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69614640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7d6401bba0 0x7f7d640089b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69614640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7d5c017050 0x7f7d5c03f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.136+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.137+0000 7f7d68e13640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7d5c017050 0x7f7d5c03f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.137+0000 7f7d68e13640 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7d5c017050 0x7f7d5c03f730 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.137+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.137+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.137+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.137+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d6401bba0 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.138+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.139+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d6401bba0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.140+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.141+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d6401bba0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.142+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.143+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.143+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.143+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.143+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.143+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.143+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d6401bba0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a78cfc0 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d5c017050 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x56347a786db0 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.144+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56347a78f940 con 0x7f7d6401bba0 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78bbd0 con 0x56347a786db0 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78c470 con 0x56347a786db0 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.145+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78c470 con 0x7f7d5c017050 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 zone default found 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 4 Realm: () 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 zonegroup default 2024-04-23T19:10:30.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78fdc0 con 0x7f7d5c017050 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a78c190 con 0x56347a78cfc0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a791d70 con 0x56347a78cfc0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 started sync module instance, tier type = 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.146+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7910e0 con 0x7f7d6401bba0 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.148+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.148+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94783393043712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a790d40 con 0x7f7d6401bba0 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.148+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a792ca0 con 0x56347a786db0 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.149+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.149+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94783393043712] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.149+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94783393049040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a793480 con 0x56347a786db0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.149+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a792210 con 0x56347a78cfc0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.150+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.150+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94783393053296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a794340 con 0x56347a78cfc0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.150+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a794e10 con 0x7f7d6401bba0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.150+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94783393049040] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.151+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94783393053296] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.151+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.151+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94783393050320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a794340 con 0x7f7d6401bba0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.151+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a795720 con 0x56347a786db0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.152+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94783393050320] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.152+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.152+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94783393054352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a795b10 con 0x56347a786db0 2024-04-23T19:10:30.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.152+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7964d0 con 0x7f7d5c017050 2024-04-23T19:10:30.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.153+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94783393054352] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.153+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.153+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94783393059376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7964d0 con 0x7f7d5c017050 2024-04-23T19:10:30.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.153+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a796ee0 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.154+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94783393059376] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.154+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.154+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94783393063424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a796ee0 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.154+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7978a0 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94783393063424] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94783393066000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7978a0 con 0x7f7d5c017050 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 20 add_watcher() i=0 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 20 add_watcher() i=1 2024-04-23T19:10:30.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 20 add_watcher() i=2 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 20 add_watcher() i=3 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 20 add_watcher() i=4 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 20 add_watcher() i=5 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.155+0000 7f7d6e013800 20 add_watcher() i=6 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.156+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94783393066000] v30'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.156+0000 7f7d6e013800 20 add_watcher() i=7 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.156+0000 7f7d6e013800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.156+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a79ac00 con 0x56347a786db0 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.156+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.157+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94783393080832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a797a10 con 0x56347a786db0 2024-04-23T19:10:30.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.157+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94783393080832] v30'104 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7c1f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 5 note: GC not initialized 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7c1e7e4640 20 reqs_thread_entry: start 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 20 Filter name: none 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7b8ffff640 20 reqs_thread_entry: start 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.158+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a7ce0c0 con 0x56347a786db0 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a7d0040 con 0x7f7d6401bba0 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:30.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.159+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a7d1b20 con 0x56347a786db0 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:30.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.160+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7d5260 con 0x7f7d6401bba0 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(65 foo.buckets [call] v30'17 uv17 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a7d43d0 con 0x7f7d6401bba0 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.161+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7d4ce0 con 0x7f7d6401bba0 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v30'18 uv18 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7d6e013800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7d6e013800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7d6e013800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94783393318832 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56347a7d66b0 con 0x7f7d5c017050 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== watch-notify(notify (1) cookie 94783393063424 notify 128849018882 ret 0) ==== 380+0+0 (crc 0 0 0) 0x56347a7d1f10 con 0x7f7d5c017050 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7c4cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018882 cookie 94783393063424 notifier 4478 bl.length()=338 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7c4cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7c4cff9640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(68 notify.6 [notify cookie 94783393318832 out=8b] v0'0 uv105 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.162+0000 7f7c4cff9640 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f7c2c003ea0 con 0x7f7d5c017050 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.163+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.163+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== watch-notify(notify_complete (2) cookie 94783393318832 notify 128849018882 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56347a7cfcc0 con 0x7f7d5c017050 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.164+0000 7f7c1f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.164+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94783393080832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a7d47b0 con 0x56347a786db0 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.165+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94783393080832] v30'105 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.165+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94783393043712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a79a600 con 0x7f7d6401bba0 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.166+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94783393043712] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.166+0000 7f7d6e013800 20 remove_watcher() i=0 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.166+0000 7f7d6e013800 2 removed watcher, disabling cache 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.166+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94783393049040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a791500 con 0x56347a786db0 2024-04-23T19:10:30.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.167+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94783393049040] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.167+0000 7f7d6e013800 20 remove_watcher() i=1 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.167+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94783393053296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a791500 con 0x56347a78cfc0 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.168+0000 7f7d69e15640 1 -- 172.21.15.63:0/3139160579 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94783393053296] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d64029070 con 0x56347a78cfc0 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.168+0000 7f7d6e013800 20 remove_watcher() i=2 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.168+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94783393050320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a791500 con 0x7f7d6401bba0 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.169+0000 7f7d69614640 1 -- 172.21.15.63:0/3139160579 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94783393050320] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d5c02dcf0 con 0x7f7d6401bba0 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.169+0000 7f7d6e013800 20 remove_watcher() i=3 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.169+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94783393054352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a791500 con 0x56347a786db0 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.170+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94783393054352] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x56347a786db0 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.170+0000 7f7d6e013800 20 remove_watcher() i=4 2024-04-23T19:10:30.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.170+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94783393059376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a791500 con 0x7f7d5c017050 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.171+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94783393059376] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.171+0000 7f7d6e013800 20 remove_watcher() i=5 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.171+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94783393063424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a791500 con 0x7f7d5c017050 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.172+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94783393063424] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.172+0000 7f7d6e013800 20 remove_watcher() i=6 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.172+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94783393066000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56347a791500 con 0x7f7d5c017050 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94783393066000] v30'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d6005a040 con 0x7f7d5c017050 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 20 remove_watcher() i=7 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7d5c017050 msgr2=0x7f7d5c03f730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7d5c017050 0x7f7d5c03f730 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a786db0 msgr2=0x56347a789270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a786db0 0x56347a789270 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7d6401bba0 msgr2=0x7f7d640089b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7d6401bba0 0x7f7d640089b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56347a78cfc0 msgr2=0x56347a78f3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56347a78cfc0 0x56347a78f3b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c34036eb0 msgr2=0x7f7c34039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c34036eb0 0x7f7c34039370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c04b2f0 tx=0x7f7d5c033000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 msgr2=0x56347a77b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3139160579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a773bb0 0x56347a77b0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7d6404cb80 tx=0x7f7d6404cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 reap_dead start 2024-04-23T19:10:30.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d68e13640 1 -- 172.21.15.63:0/3139160579 reap_dead start 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 shutdown_connections 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 >> 172.21.15.63:0/3139160579 conn(0x56347a76fe80 msgr2=0x56347a76d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 shutdown_connections 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.173+0000 7f7d6e013800 1 -- 172.21.15.63:0/3139160579 wait complete. 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c44036e60 msgr2=0x7f7c44039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c44036e60 0x7f7c44039320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c04b550 tx=0x7f7d5c04b1c0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a741c50 msgr2=0x56347a7400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a741c50 0x56347a7400c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56347a750b20 tx=0x7f7d640178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 shutdown_connections 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c44036e60 0x7f7c44039320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/339848323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a741c50 0x56347a7400c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 >> 172.21.15.63:0/339848323 conn(0x56347a7586d0 msgr2=0x56347a758ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 shutdown_connections 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/339848323 wait complete. 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56347a753910 msgr2=0x56347a755d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56347a753910 0x56347a755d00 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a6534a0 msgr2=0x56347a750190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a6534a0 0x56347a750190 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c50036e60 msgr2=0x7f7c50039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c50036e60 0x7f7c50039320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d60054000 tx=0x7f7d6004e000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 msgr2=0x56347a748fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a748fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00b7c0 tx=0x7f7d5c000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 shutdown_connections 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56347a753910 0x56347a755d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56347a6534a0 0x56347a750190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7c50036e60 0x7f7c50039320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 --2- 172.21.15.63:0/3812036084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56347a64f010 0x56347a748fa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 >> 172.21.15.63:0/3812036084 conn(0x56347a64a630 msgr2=0x56347a73a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 shutdown_connections 2024-04-23T19:10:30.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.174+0000 7f7d6e013800 1 -- 172.21.15.63:0/3812036084 wait complete. 2024-04-23T19:10:30.200 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-23T19:10:30.200 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-23T19:10:30.200 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-23T19:10:30.252 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:30.253 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7735546800 1 Processor -- start 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7735546800 1 -- start start 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7735546800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559a630c64d0 0x559a630c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7735546800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559a62f757f0 con 0x559a630c2040 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7735546800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x559a62f64e20 con 0x559a630c64d0 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7732844640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559a630c64d0 0x559a630c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7733045640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x559a630c2040 0x559a630c2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38774/0 (socket says 172.21.15.63:38774) 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.253+0000 7f7733045640 1 -- 172.21.15.63:0/702302897 learned_addr learned my addr 172.21.15.63:0/702302897 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.254+0000 7f7732844640 1 -- 172.21.15.63:0/702302897 >> v1:172.21.15.16:6789/0 conn(0x559a630c2040 legacy=0x559a630c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.254+0000 7f7732844640 1 -- 172.21.15.63:0/702302897 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a62f83b50 con 0x559a630c64d0 2024-04-23T19:10:30.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.255+0000 7f7732844640 1 --2- 172.21.15.63:0/702302897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c64d0 0x559a630c88c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f771c00a470 tx=0x7f771c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5add86321fb605e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7732043640 1 -- 172.21.15.63:0/702302897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f771c057070 con 0x559a630c64d0 2024-04-23T19:10:30.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7732043640 1 -- 172.21.15.63:0/702302897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771c052570 con 0x559a630c64d0 2024-04-23T19:10:30.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7732043640 1 -- 172.21.15.63:0/702302897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f771c052870 con 0x559a630c64d0 2024-04-23T19:10:30.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7735546800 1 -- 172.21.15.63:0/702302897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c64d0 msgr2=0x559a630c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7735546800 1 --2- 172.21.15.63:0/702302897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c64d0 0x559a630c88c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f771c00a470 tx=0x7f771c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7735546800 1 -- 172.21.15.63:0/702302897 shutdown_connections 2024-04-23T19:10:30.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7735546800 1 --2- 172.21.15.63:0/702302897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c64d0 0x559a630c88c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7735546800 1 -- 172.21.15.63:0/702302897 >> 172.21.15.63:0/702302897 conn(0x559a630bd660 msgr2=0x559a630bfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7735546800 1 -- 172.21.15.63:0/702302897 shutdown_connections 2024-04-23T19:10:30.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.256+0000 7f7735546800 1 -- 172.21.15.63:0/702302897 wait complete. 2024-04-23T19:10:30.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.269+0000 7f7735546800 1 Processor -- start 2024-04-23T19:10:30.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.269+0000 7f7735546800 1 -- start start 2024-04-23T19:10:30.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.269+0000 7f7735546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.269+0000 7f7735546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a6300e720 con 0x559a630c2040 2024-04-23T19:10:30.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.269+0000 7f7733045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.269+0000 7f7733045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54462/0 (socket says 172.21.15.63:54462) 2024-04-23T19:10:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.269+0000 7f7733045640 1 -- 172.21.15.63:0/2695048158 learned_addr learned my addr 172.21.15.63:0/2695048158 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.270+0000 7f7733045640 1 -- 172.21.15.63:0/2695048158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a631b3950 con 0x559a630c2040 2024-04-23T19:10:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.270+0000 7f7733045640 1 --2- 172.21.15.63:0/2695048158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631b3380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7724001e10 tx=0x7f772400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bcc169738c52c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.270+0000 7f762160a640 1 -- 172.21.15.63:0/2695048158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772400b200 con 0x559a630c2040 2024-04-23T19:10:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.270+0000 7f762160a640 1 -- 172.21.15.63:0/2695048158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7724009460 con 0x559a630c2040 2024-04-23T19:10:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.270+0000 7f762160a640 1 -- 172.21.15.63:0/2695048158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7724009760 con 0x559a630c2040 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 -- 172.21.15.63:0/2695048158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 msgr2=0x559a631b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 --2- 172.21.15.63:0/2695048158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631b3380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7724001e10 tx=0x7f772400b910 comp rx=0 tx=0).stop 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 -- 172.21.15.63:0/2695048158 shutdown_connections 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 --2- 172.21.15.63:0/2695048158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631b3380 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 -- 172.21.15.63:0/2695048158 >> 172.21.15.63:0/2695048158 conn(0x559a630bd660 msgr2=0x559a631aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 -- 172.21.15.63:0/2695048158 shutdown_connections 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 -- 172.21.15.63:0/2695048158 wait complete. 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 Processor -- start 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 -- start start 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631be240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.271+0000 7f7735546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a630c9190 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f7733045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f7733045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54474/0 (socket says 172.21.15.63:54474) 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f7733045640 1 -- 172.21.15.63:0/3103247600 learned_addr learned my addr 172.21.15.63:0/3103247600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f7733045640 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a631be780 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f7733045640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631be240 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7724000fd0 tx=0x7f7724000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f761fe07640 1 -- 172.21.15.63:0/3103247600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772401b040 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a631bea10 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f761fe07640 1 -- 172.21.15.63:0/3103247600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7724009b10 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f761fe07640 1 -- 172.21.15.63:0/3103247600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7724009e30 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.272+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a631c16c0 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.273+0000 7f761fe07640 1 -- 172.21.15.63:0/3103247600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7724016430 con 0x559a630c2040 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.273+0000 7f761fe07640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7618036eb0 0x7f7618039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.273+0000 7f761fe07640 1 -- 172.21.15.63:0/3103247600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f7724011070 con 0x559a630c2040 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7732844640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7618036eb0 0x7f7618039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a630c64d0 0x559a631c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631c5a20 con 0x559a630c64d0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7733846640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a630c64d0 0x559a631c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7732844640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7618036eb0 0x7f7618039370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f771c054000 tx=0x7f771c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7733846640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a630c64d0 0x559a631c54e0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7733846640 1 -- 172.21.15.63:0/3103247600 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f772c00a4a0 con 0x559a630c64d0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.274+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631c2a90 con 0x559a630c64d0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.275+0000 7f7733846640 1 -- 172.21.15.63:0/3103247600 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f772c00a4a0 con 0x559a630c64d0 2024-04-23T19:10:30.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.275+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631c77e0 con 0x559a630c64d0 2024-04-23T19:10:30.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.275+0000 7f7733846640 1 -- 172.21.15.63:0/3103247600 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f772c00a4a0 con 0x559a630c64d0 2024-04-23T19:10:30.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.275+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631c6b20 con 0x559a630c64d0 2024-04-23T19:10:30.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.275+0000 7f7733846640 1 -- 172.21.15.63:0/3103247600 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f772c00a4a0 con 0x559a630c64d0 2024-04-23T19:10:30.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.275+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a631c8b90 0x559a631caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.275+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631cb500 con 0x559a631c8b90 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.276+0000 7f7733045640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a631c8b90 0x559a631caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.276+0000 7f7733045640 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a631c8b90 0x559a631caf80 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.276+0000 7f7733045640 1 -- 172.21.15.63:0/3103247600 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7724009040 con 0x559a631c8b90 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7735546800 1 Processor -- start 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7735546800 1 -- start start 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7735546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631d98b0 0x559a631d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7735546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a62fb4e30 con 0x559a631d98b0 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7733846640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631d98b0 0x559a631d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7733846640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631d98b0 0x559a631d9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54486/0 (socket says 172.21.15.63:54486) 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7733846640 1 -- 172.21.15.63:0/820655787 learned_addr learned my addr 172.21.15.63:0/820655787 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7733846640 1 -- 172.21.15.63:0/820655787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a631da210 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.277+0000 7f7733846640 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631d98b0 0x559a631d9cd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x559a631bf640 tx=0x7f772c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.278+0000 7f761d602640 1 -- 172.21.15.63:0/820655787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772c013440 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.278+0000 7f761d602640 1 -- 172.21.15.63:0/820655787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c0135e0 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.278+0000 7f761d602640 1 -- 172.21.15.63:0/820655787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772c0138c0 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.278+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a631dac80 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.278+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a631db180 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.278+0000 7f761d602640 1 -- 172.21.15.63:0/820655787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f772c011040 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f761d602640 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f760c036eb0 0x7f760c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f761d602640 1 -- 172.21.15.63:0/820655787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f772c01d070 con 0x559a631d98b0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7733045640 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f760c036eb0 0x7f760c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7735546800 1 Processor -- start 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7735546800 1 -- start start 2024-04-23T19:10:30.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7733045640 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f760c036eb0 0x7f760c039370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7724000c00 tx=0x7f772404a770 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7735546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7735546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a631b3cd0 con 0x559a631e6b90 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7733846640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7733846640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54488/0 (socket says 172.21.15.63:54488) 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.279+0000 7f7733846640 1 -- 172.21.15.63:0/3894937551 learned_addr learned my addr 172.21.15.63:0/3894937551 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7733846640 1 -- 172.21.15.63:0/3894937551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a631e7580 con 0x559a631e6b90 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7733846640 1 --2- 172.21.15.63:0/3894937551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631e6fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f772c04cca0 tx=0x7f772c0127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f6af8cad2667d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7616ffd640 1 -- 172.21.15.63:0/3894937551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772c042020 con 0x559a631e6b90 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7616ffd640 1 -- 172.21.15.63:0/3894937551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c012cc0 con 0x559a631e6b90 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7616ffd640 1 -- 172.21.15.63:0/3894937551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772c027040 con 0x559a631e6b90 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7735546800 1 -- 172.21.15.63:0/3894937551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 msgr2=0x559a631e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7735546800 1 --2- 172.21.15.63:0/3894937551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631e6fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f772c04cca0 tx=0x7f772c0127a0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7735546800 1 -- 172.21.15.63:0/3894937551 shutdown_connections 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7735546800 1 --2- 172.21.15.63:0/3894937551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631e6fb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7735546800 1 -- 172.21.15.63:0/3894937551 >> 172.21.15.63:0/3894937551 conn(0x559a631e2ef0 msgr2=0x559a631e0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7735546800 1 -- 172.21.15.63:0/3894937551 shutdown_connections 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.280+0000 7f7735546800 1 -- 172.21.15.63:0/3894937551 wait complete. 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7735546800 1 Processor -- start 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7735546800 1 -- start start 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7735546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7735546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a631e7cf0 con 0x559a631e6b90 2024-04-23T19:10:30.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7733846640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7733846640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631ee1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54492/0 (socket says 172.21.15.63:54492) 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 learned_addr learned my addr 172.21.15.63:0/891021382 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a631ee6e0 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.281+0000 7f7733846640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631ee1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f772c0127e0 tx=0x7f772c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.282+0000 7f76157fa640 1 -- 172.21.15.63:0/891021382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772c027040 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.282+0000 7f76157fa640 1 -- 172.21.15.63:0/891021382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c021c60 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.282+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a631ee970 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.282+0000 7f76157fa640 1 -- 172.21.15.63:0/891021382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f772c02a350 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.282+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a631f1620 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.282+0000 7f76157fa640 1 -- 172.21.15.63:0/891021382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f772c045020 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.282+0000 7f76157fa640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f75fc036eb0 0x7f75fc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.283+0000 7f76157fa640 1 -- 172.21.15.63:0/891021382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f772c033030 con 0x559a631e6b90 2024-04-23T19:10:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.283+0000 7f7733045640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f75fc036eb0 0x7f75fc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.283+0000 7f7733045640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f75fc036eb0 0x7f75fc039370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7724009860 tx=0x7f7724041000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.283+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.283+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a631f9c50 0x559a631fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.283+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631fc860 con 0x559a631f9c50 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.283+0000 7f7732844640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a631f9c50 0x559a631fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7732844640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a631f9c50 0x559a631fc110 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 20 realm 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631fcc30 con 0x559a631f9c50 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631fd500 con 0x559a631f9c50 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.284+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631fd8a0 con 0x559a631f9c50 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7735546800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631fe0f0 con 0x559a631f9c50 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7735546800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7735546800 20 searching for the correct realm 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a631ffe60 0x559a63202250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7733846640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a631ffe60 0x559a63202250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.285+0000 7f7733846640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a631ffe60 0x559a63202250 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.286+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.286+0000 7f7733846640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f772c0123a0 0x7f772c02b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.286+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f772c0123a0 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.286+0000 7f7733045640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f772c0123a0 0x7f772c02b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.286+0000 7f7733045640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f772c0123a0 0x7f772c02b3e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.287+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.287+0000 7f7733045640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f77240277a0 0x7f772401f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.287+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.287+0000 7f7732844640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f77240277a0 0x7f772401f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.287+0000 7f7732844640 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f77240277a0 0x7f772401f730 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.287+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.287+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.288+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.288+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.288+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.288+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.288+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.288+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f772c0123a0 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.289+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.290+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.290+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.290+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.290+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f772c0123a0 2024-04-23T19:10:30.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.290+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.290+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.291+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.292+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f772c0123a0 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.293+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.294+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.294+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.294+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.294+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f772c0123a0 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.294+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.294+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631ffe60 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.295+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.295+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f77240277a0 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.295+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.295+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x559a631f9c50 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.295+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.295+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559a632027e0 con 0x7f772c0123a0 2024-04-23T19:10:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631feaa0 con 0x559a631f9c50 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631ff340 con 0x559a631f9c50 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631ff340 con 0x7f77240277a0 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.296+0000 7f7735546800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 zone default found 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 4 Realm: () 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 zonegroup default 2024-04-23T19:10:30.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a63202c60 con 0x7f77240277a0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a631ff100 con 0x559a631ffe60 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a63204be0 con 0x559a631ffe60 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 started sync module instance, tier type = 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:30.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.297+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63203f80 con 0x7f772c0123a0 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.298+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.298+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94121576383344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63203be0 con 0x7f772c0123a0 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.298+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63205b10 con 0x559a631f9c50 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.299+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94121576383344] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.300+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.300+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94121576388192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63205a80 con 0x559a631f9c50 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.300+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a632069d0 con 0x559a631ffe60 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.301+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94121576388192] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.301+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.301+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94121576391968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63206dc0 con 0x559a631ffe60 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.301+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a632077e0 con 0x7f772c0123a0 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.302+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94121576391968] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.302+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.302+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94121576395472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63207bd0 con 0x7f772c0123a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.302+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63208630 con 0x559a631f9c50 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.303+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.303+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94121576399168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a632085e0 con 0x559a631f9c50 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.303+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63209600 con 0x7f77240277a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.303+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94121576395472] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.304+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.304+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94121576399168] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.304+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94121576403152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a632095c0 con 0x7f77240277a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.304+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63208a50 con 0x7f77240277a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.305+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(56 notify.6 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.305+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94121576405856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63208a50 con 0x7f77240277a0 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.305+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a6320aa50 con 0x7f77240277a0 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.305+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94121576403152] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94121576405856] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94121576406912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a6320a5a0 con 0x7f77240277a0 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 20 add_watcher() i=0 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 20 add_watcher() i=1 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 20 add_watcher() i=2 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 20 add_watcher() i=3 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 20 add_watcher() i=4 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 20 add_watcher() i=5 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.306+0000 7f7735546800 20 add_watcher() i=6 2024-04-23T19:10:30.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.308+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94121576406912] v30'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.308+0000 7f7735546800 20 add_watcher() i=7 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.308+0000 7f7735546800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.308+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a6320da80 con 0x559a631f9c50 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.308+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.308+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94121576408480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a6320ae70 con 0x559a631f9c50 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.309+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94121576408480] v30'106 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.309+0000 7f75e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.309+0000 7f7735546800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.309+0000 7f7735546800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.309+0000 7f7735546800 5 note: GC not initialized 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.309+0000 7f75e8fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:30.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 20 Filter name: none 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f75627fc640 20 reqs_thread_entry: start 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a63240d20 con 0x7f77240277a0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.310+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a6323f4c0 con 0x7f77240277a0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.311+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.311+0000 7f7735546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.311+0000 7f7735546800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.311+0000 7f7735546800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.311+0000 7f7735546800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.311+0000 7f7735546800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.311+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a6323fd60 con 0x7f77240277a0 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v30'1 uv1 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7735546800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7735546800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7735546800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94121576626352 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a6323fd10 con 0x7f77240277a0 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== watch-notify(notify (1) cookie 94121576403152 notify 128849018883 ret 0) ==== 598+0+0 (crc 0 0 0) 0x559a63240100 con 0x7f77240277a0 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7616ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018883 cookie 94121576403152 notifier 4485 bl.length()=556 2024-04-23T19:10:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7616ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-23T19:10:30.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7616ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:30.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(65 notify.5 [notify cookie 94121576626352 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7616ffd640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f75f8003ea0 con 0x7f77240277a0 2024-04-23T19:10:30.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== watch-notify(notify_complete (2) cookie 94121576626352 notify 128849018883 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559a6323f5b0 con 0x7f77240277a0 2024-04-23T19:10:30.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.312+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a6323fd60 con 0x7f77240277a0 2024-04-23T19:10:30.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.313+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v30'3 uv3 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.313+0000 7f7735546800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:10:30.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.313+0000 7f7735546800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:30.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.313+0000 7f7735546800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:10:30.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.313+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94121576626352 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559a6323fcd0 con 0x559a631ffe60 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== watch-notify(notify (1) cookie 94121576391968 notify 128849018882 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7f772c053030 con 0x559a631ffe60 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7616ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018882 cookie 94121576391968 notifier 4485 bl.length()=177 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7616ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7616ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 notify.2 [notify cookie 94121576626352 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7616ffd640 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f75f8003ea0 con 0x559a631ffe60 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.314+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== watch-notify(notify_complete (2) cookie 94121576626352 notify 128849018882 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559a631ee970 con 0x559a631ffe60 2024-04-23T19:10:30.329 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-23T19:10:30.311592Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:30.311570Z","tags":[],"group_ids":[]} 2024-04-23T19:10:30.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.315+0000 7f75e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.315+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94121576408480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a631f5ec0 con 0x559a631f9c50 2024-04-23T19:10:30.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.316+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94121576408480] v30'107 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.316+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94121576383344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a631f5ec0 con 0x7f772c0123a0 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.317+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94121576383344] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.317+0000 7f7735546800 20 remove_watcher() i=0 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.317+0000 7f7735546800 2 removed watcher, disabling cache 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.317+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94121576388192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63204370 con 0x559a631f9c50 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.318+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94121576388192] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.318+0000 7f7735546800 20 remove_watcher() i=1 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.318+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94121576391968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63204370 con 0x559a631ffe60 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.319+0000 7f7733846640 1 -- 172.21.15.63:0/891021382 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94121576391968] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f772c029070 con 0x559a631ffe60 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.319+0000 7f7735546800 20 remove_watcher() i=2 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.319+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94121576395472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63206520 con 0x7f772c0123a0 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.320+0000 7f7733045640 1 -- 172.21.15.63:0/891021382 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94121576395472] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7724009040 con 0x7f772c0123a0 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.320+0000 7f7735546800 20 remove_watcher() i=3 2024-04-23T19:10:30.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.320+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94121576399168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63204370 con 0x559a631f9c50 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.321+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94121576399168] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x559a631f9c50 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.321+0000 7f7735546800 20 remove_watcher() i=4 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.321+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94121576403152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63208140 con 0x7f77240277a0 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.322+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94121576403152] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.322+0000 7f7735546800 20 remove_watcher() i=5 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.322+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94121576405856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63204370 con 0x7f77240277a0 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.323+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94121576405856] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.323+0000 7f7735546800 20 remove_watcher() i=6 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.323+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94121576406912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559a63208140 con 0x7f77240277a0 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94121576406912] v30'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f771c05a040 con 0x7f77240277a0 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 20 remove_watcher() i=7 2024-04-23T19:10:30.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f77240277a0 msgr2=0x7f772401f730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f77240277a0 0x7f772401f730 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a631f9c50 msgr2=0x559a631fc110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a631f9c50 0x559a631fc110 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f772c0123a0 msgr2=0x7f772c02b3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f772c0123a0 0x7f772c02b3e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a631ffe60 msgr2=0x559a63202250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a631ffe60 0x559a63202250 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f75fc036eb0 msgr2=0x7f75fc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f75fc036eb0 0x7f75fc039370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7724009860 tx=0x7f7724041000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 msgr2=0x559a631ee1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631ee1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f772c0127e0 tx=0x7f772c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:30.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.324+0000 7f7732844640 1 -- 172.21.15.63:0/891021382 reap_dead start 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 shutdown_connections 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/891021382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631e6b90 0x559a631ee1a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 >> 172.21.15.63:0/891021382 conn(0x559a631e2ef0 msgr2=0x559a631e0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 shutdown_connections 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/891021382 wait complete. 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f760c036eb0 msgr2=0x7f760c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f760c036eb0 0x7f760c039370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7724000c00 tx=0x7f772404a770 comp rx=0 tx=0).stop 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631d98b0 msgr2=0x559a631d9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631d98b0 0x559a631d9cd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x559a631bf640 tx=0x7f772c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 shutdown_connections 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f760c036eb0 0x7f760c039370 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/820655787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a631d98b0 0x559a631d9cd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 >> 172.21.15.63:0/820655787 conn(0x559a631cd950 msgr2=0x559a631cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 shutdown_connections 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/820655787 wait complete. 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a631c8b90 msgr2=0x559a631caf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a631c8b90 0x559a631caf80 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a630c64d0 msgr2=0x559a631c54e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a630c64d0 0x559a631c54e0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7618036eb0 msgr2=0x7f7618039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7618036eb0 0x7f7618039370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f771c054000 tx=0x7f771c04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 msgr2=0x559a631be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.325+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631be240 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7724000fd0 tx=0x7f7724000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 shutdown_connections 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a631c8b90 0x559a631caf80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a630c64d0 0x559a631c54e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7618036eb0 0x7f7618039370 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 --2- 172.21.15.63:0/3103247600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a630c2040 0x559a631be240 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 >> 172.21.15.63:0/3103247600 conn(0x559a630bd660 msgr2=0x559a63035770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 shutdown_connections 2024-04-23T19:10:30.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.326+0000 7f7735546800 1 -- 172.21.15.63:0/3103247600 wait complete. 2024-04-23T19:10:30.341 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-23T19:10:30.311592Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:30.311570Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:30.346 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-23T19:10:30.351 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:30.351 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-04-23T19:10:30.351 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-04-23T19:10:30.352 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-04-23T19:10:30.395 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:30.395 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:30.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.395+0000 7ff3f3726800 1 Processor -- start 2024-04-23T19:10:30.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3f3726800 1 -- start start 2024-04-23T19:10:30.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3f3726800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56003bbc0620 0x56003bbc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3f3726800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56003ba6e7f0 con 0x56003bbba400 2024-04-23T19:10:30.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3f3726800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x56003ba5de20 con 0x56003bbc0620 2024-04-23T19:10:30.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3eec27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56003bbc0620 0x56003bbc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3ef428640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x56003bbba400 0x56003bbba820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38778/0 (socket says 172.21.15.63:38778) 2024-04-23T19:10:30.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3ef428640 1 -- 172.21.15.63:0/2707226796 learned_addr learned my addr 172.21.15.63:0/2707226796 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:30.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3eec27640 1 -- 172.21.15.63:0/2707226796 >> v1:172.21.15.16:6789/0 conn(0x56003bbba400 legacy=0x56003bbba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.396+0000 7ff3eec27640 1 -- 172.21.15.63:0/2707226796 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003ba7cb50 con 0x56003bbc0620 2024-04-23T19:10:30.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.398+0000 7ff3eec27640 1 --2- 172.21.15.63:0/2707226796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbc0620 0x56003bbc2a10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400a470 tx=0x7ff3e401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64f5c96cf54d0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.398+0000 7ff3ee426640 1 -- 172.21.15.63:0/2707226796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e4057070 con 0x56003bbc0620 2024-04-23T19:10:30.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.398+0000 7ff3ee426640 1 -- 172.21.15.63:0/2707226796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e4052570 con 0x56003bbc0620 2024-04-23T19:10:30.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.398+0000 7ff3ee426640 1 -- 172.21.15.63:0/2707226796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e4052870 con 0x56003bbc0620 2024-04-23T19:10:30.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.399+0000 7ff3f3726800 1 -- 172.21.15.63:0/2707226796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbc0620 msgr2=0x56003bbc2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.399+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2707226796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbc0620 0x56003bbc2a10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400a470 tx=0x7ff3e401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.399+0000 7ff3f3726800 1 -- 172.21.15.63:0/2707226796 shutdown_connections 2024-04-23T19:10:30.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.399+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2707226796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbc0620 0x56003bbc2a10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.399+0000 7ff3f3726800 1 -- 172.21.15.63:0/2707226796 >> 172.21.15.63:0/2707226796 conn(0x56003bbb5db0 msgr2=0x56003bbb81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.399+0000 7ff3f3726800 1 -- 172.21.15.63:0/2707226796 shutdown_connections 2024-04-23T19:10:30.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.399+0000 7ff3f3726800 1 -- 172.21.15.63:0/2707226796 wait complete. 2024-04-23T19:10:30.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3f3726800 1 Processor -- start 2024-04-23T19:10:30.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3f3726800 1 -- start start 2024-04-23T19:10:30.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3f3726800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bcad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3f3726800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56003bb076f0 con 0x56003bbba400 2024-04-23T19:10:30.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3ef428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bcad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3ef428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bcad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54508/0 (socket says 172.21.15.63:54508) 2024-04-23T19:10:30.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3ef428640 1 -- 172.21.15.63:0/4156233089 learned_addr learned my addr 172.21.15.63:0/4156233089 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.412+0000 7ff3ef428640 1 -- 172.21.15.63:0/4156233089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003bcad920 con 0x56003bbba400 2024-04-23T19:10:30.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.413+0000 7ff3ef428640 1 --2- 172.21.15.63:0/4156233089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bcad350 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0001d50 tx=0x7ff3e000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dca1d1b49523f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.413+0000 7ff2dd60a640 1 -- 172.21.15.63:0/4156233089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e0013030 con 0x56003bbba400 2024-04-23T19:10:30.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.413+0000 7ff2dd60a640 1 -- 172.21.15.63:0/4156233089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e000e470 con 0x56003bbba400 2024-04-23T19:10:30.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.413+0000 7ff2dd60a640 1 -- 172.21.15.63:0/4156233089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e000e810 con 0x56003bbba400 2024-04-23T19:10:30.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 -- 172.21.15.63:0/4156233089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 msgr2=0x56003bcad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 --2- 172.21.15.63:0/4156233089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bcad350 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0001d50 tx=0x7ff3e000b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 -- 172.21.15.63:0/4156233089 shutdown_connections 2024-04-23T19:10:30.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 --2- 172.21.15.63:0/4156233089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bcad350 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 -- 172.21.15.63:0/4156233089 >> 172.21.15.63:0/4156233089 conn(0x56003bbb5db0 msgr2=0x56003bca8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 -- 172.21.15.63:0/4156233089 shutdown_connections 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 -- 172.21.15.63:0/4156233089 wait complete. 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 Processor -- start 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 -- start start 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bbbcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3f3726800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56003bbc32b0 con 0x56003bbba400 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3ef428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bbbcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3ef428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bbbcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54514/0 (socket says 172.21.15.63:54514) 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.414+0000 7ff3ef428640 1 -- 172.21.15.63:0/2190099758 learned_addr learned my addr 172.21.15.63:0/2190099758 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.415+0000 7ff3ef428640 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003bbbd220 con 0x56003bbba400 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.415+0000 7ff3ef428640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bbbcce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56003bcae0e0 tx=0x7ff3e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.415+0000 7ff2dbe07640 1 -- 172.21.15.63:0/2190099758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e0027070 con 0x56003bbba400 2024-04-23T19:10:30.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.415+0000 7ff2dbe07640 1 -- 172.21.15.63:0/2190099758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e0019900 con 0x56003bbba400 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.415+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56003bbbb420 con 0x56003bbba400 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.415+0000 7ff2dbe07640 1 -- 172.21.15.63:0/2190099758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e0019c40 con 0x56003bbba400 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.415+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003bbbb940 con 0x56003bbba400 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.416+0000 7ff2dbe07640 1 -- 172.21.15.63:0/2190099758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff3e002a070 con 0x56003bbba400 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.416+0000 7ff2dbe07640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2d4036e60 0x7ff2d4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.416+0000 7ff2dbe07640 1 -- 172.21.15.63:0/2190099758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff3e001f070 con 0x56003bbba400 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.416+0000 7ff3eec27640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2d4036e60 0x7ff2d4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.416+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bbc0620 0x56003bcbe350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.416+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcbe920 con 0x56003bbc0620 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.416+0000 7ff3efc29640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bbc0620 0x56003bcbe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.417+0000 7ff3eec27640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2d4036e60 0x7ff2d4039320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4054000 tx=0x7ff3e404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.417+0000 7ff3efc29640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bbc0620 0x56003bcbe350 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.417+0000 7ff3efc29640 1 -- 172.21.15.63:0/2190099758 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bbc0620 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.417+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bbbbe40 con 0x56003bbc0620 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.417+0000 7ff3efc29640 1 -- 172.21.15.63:0/2190099758 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bbc0620 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.417+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcc0720 con 0x56003bbc0620 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.418+0000 7ff3efc29640 1 -- 172.21.15.63:0/2190099758 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bbc0620 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.418+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcbfa60 con 0x56003bbc0620 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.418+0000 7ff3efc29640 1 -- 172.21.15.63:0/2190099758 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bbc0620 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.418+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56003bcc1ad0 0x56003bcc3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.418+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcc4440 con 0x56003bcc1ad0 2024-04-23T19:10:30.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.418+0000 7ff3ef428640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56003bcc1ad0 0x56003bcc3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.419+0000 7ff3ef428640 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56003bcc1ad0 0x56003bcc3ec0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.419+0000 7ff3ef428640 1 -- 172.21.15.63:0/2190099758 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff3e0029030 con 0x56003bcc1ad0 2024-04-23T19:10:30.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.419+0000 7ff3f3726800 1 Processor -- start 2024-04-23T19:10:30.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.419+0000 7ff3f3726800 1 -- start start 2024-04-23T19:10:30.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.419+0000 7ff3f3726800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcd27d0 0x56003bcd2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.419+0000 7ff3f3726800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56003baade30 con 0x56003bcd27d0 2024-04-23T19:10:30.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff3efc29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcd27d0 0x56003bcd2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff3efc29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcd27d0 0x56003bcd2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54516/0 (socket says 172.21.15.63:54516) 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff3efc29640 1 -- 172.21.15.63:0/4184961302 learned_addr learned my addr 172.21.15.63:0/4184961302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff3efc29640 1 -- 172.21.15.63:0/4184961302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003bcd3130 con 0x56003bcd27d0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff3efc29640 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcd27d0 0x56003bcd2bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x56003bcbece0 tx=0x7ff3e80178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff2d9602640 1 -- 172.21.15.63:0/4184961302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e8013440 con 0x56003bcd27d0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff2d9602640 1 -- 172.21.15.63:0/4184961302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e80135e0 con 0x56003bcd27d0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56003bcd3ba0 con 0x56003bcd27d0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff2d9602640 1 -- 172.21.15.63:0/4184961302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e80138c0 con 0x56003bcd27d0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.420+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003bcd40a0 con 0x56003bcd27d0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.421+0000 7ff2d9602640 1 -- 172.21.15.63:0/4184961302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff3e8011040 con 0x56003bcd27d0 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.421+0000 7ff2d9602640 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2c8036eb0 0x7ff2c8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.421+0000 7ff2d9602640 1 -- 172.21.15.63:0/4184961302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff3e801d070 con 0x56003bcd27d0 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.421+0000 7ff3ef428640 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2c8036eb0 0x7ff2c8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.421+0000 7ff3f3726800 1 Processor -- start 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3ef428640 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2c8036eb0 0x7ff2c8039370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x56003bcc4820 tx=0x7ff3e000ef30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3f3726800 1 -- start start 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3f3726800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bcdff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3f3726800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56003bcadca0 con 0x56003bcdfb00 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3efc29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bcdff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3efc29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bcdff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54524/0 (socket says 172.21.15.63:54524) 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3efc29640 1 -- 172.21.15.63:0/3660686076 learned_addr learned my addr 172.21.15.63:0/3660686076 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3efc29640 1 -- 172.21.15.63:0/3660686076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003bce04f0 con 0x56003bcdfb00 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff3efc29640 1 --2- 172.21.15.63:0/3660686076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bcdff20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff3e804ccf0 tx=0x7ff3e802b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8dd137bf0e7f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff2d2ffd640 1 -- 172.21.15.63:0/3660686076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e802bc90 con 0x56003bcdfb00 2024-04-23T19:10:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff2d2ffd640 1 -- 172.21.15.63:0/3660686076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e802be30 con 0x56003bcdfb00 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.422+0000 7ff2d2ffd640 1 -- 172.21.15.63:0/3660686076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e802b4f0 con 0x56003bcdfb00 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 -- 172.21.15.63:0/3660686076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 msgr2=0x56003bcdff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3660686076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bcdff20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff3e804ccf0 tx=0x7ff3e802b8e0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 -- 172.21.15.63:0/3660686076 shutdown_connections 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3660686076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bcdff20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 -- 172.21.15.63:0/3660686076 >> 172.21.15.63:0/3660686076 conn(0x56003bcdbdd0 msgr2=0x56003bcd9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 -- 172.21.15.63:0/3660686076 shutdown_connections 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 -- 172.21.15.63:0/3660686076 wait complete. 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 Processor -- start 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 -- start start 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bce7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3f3726800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56003bce0c80 con 0x56003bcdfb00 2024-04-23T19:10:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3efc29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bce7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.423+0000 7ff3efc29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bce7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54534/0 (socket says 172.21.15.63:54534) 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 learned_addr learned my addr 172.21.15.63:0/3959075150 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003bce7540 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff3efc29640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bce7000 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3e804bc10 tx=0x7ff3e804cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff2d17fa640 1 -- 172.21.15.63:0/3959075150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e8013b90 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff2d17fa640 1 -- 172.21.15.63:0/3959075150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e8013d30 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff2d17fa640 1 -- 172.21.15.63:0/3959075150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3e802a480 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56003bce7830 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.424+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003bcea480 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff2d17fa640 1 -- 172.21.15.63:0/3959075150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff3e802a620 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff2d17fa640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2b8036e60 0x7ff2b8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff2d17fa640 1 -- 172.21.15.63:0/3959075150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff3e8032070 con 0x56003bcdfb00 2024-04-23T19:10:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff3ef428640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2b8036e60 0x7ff2b8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bcf2b70 0x56003bcf5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf5780 con 0x56003bcf2b70 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff3eec27640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bcf2b70 0x56003bcf5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.425+0000 7ff3ef428640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2b8036e60 0x7ff2b8039320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0046010 tx=0x7ff3e0043000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3eec27640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bcf2b70 0x56003bcf5030 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 20 realm 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf5b50 con 0x56003bcf2b70 2024-04-23T19:10:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf6420 con 0x56003bcf2b70 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.426+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf67c0 con 0x56003bcf2b70 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3f3726800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf6be0 con 0x56003bcf2b70 2024-04-23T19:10:30.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3f3726800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3f3726800 20 searching for the correct realm 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56003bcf8be0 0x56003bcfb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3efc29640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56003bcf8be0 0x56003bcfb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.427+0000 7ff3efc29640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56003bcf8be0 0x56003bcfb030 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.428+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.428+0000 7ff3efc29640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff3e804ec20 0x7ff3e8046860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.428+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e804ec20 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.428+0000 7ff3ef428640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff3e804ec20 0x7ff3e8046860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.428+0000 7ff3ef428640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff3e804ec20 0x7ff3e8046860 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.429+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.429+0000 7ff3ef428640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff3e002c060 0x7ff3e004c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.429+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.429+0000 7ff3eec27640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff3e002c060 0x7ff3e004c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.429+0000 7ff3eec27640 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff3e002c060 0x7ff3e004c370 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.430+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.430+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.430+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.430+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.430+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.430+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e804ec20 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.431+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.432+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.432+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.432+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.432+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.432+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e804ec20 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.432+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.432+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.433+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.434+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e804ec20 2024-04-23T19:10:30.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.435+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e804ec20 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf8be0 2024-04-23T19:10:30.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.436+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e002c060 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x56003bcf2b70 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfb700 con 0x7ff3e804ec20 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3f3726800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3f3726800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3f3726800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3f3726800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3f3726800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:30.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3f3726800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.437+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf76b0 con 0x56003bcf2b70 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf7f50 con 0x56003bcf2b70 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf72f0 con 0x7ff3e002c060 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 zone default found 2024-04-23T19:10:30.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 4 Realm: () 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 zonegroup default 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.438+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcf7f50 con 0x7ff3e002c060 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfd5b0 con 0x56003bcf8be0 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfd950 con 0x56003bcf8be0 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 20 started sync module instance, tier type = 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.439+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfca50 con 0x7ff3e804ec20 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.440+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.440+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94559003463280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfe230 con 0x7ff3e804ec20 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.441+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfea80 con 0x56003bcf2b70 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.442+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94559003463280] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.442+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.442+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94559003467136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfeb40 con 0x56003bcf2b70 2024-04-23T19:10:30.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.442+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcff640 con 0x56003bcf8be0 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.443+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94559003467136] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.443+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.443+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94559003472176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcff600 con 0x56003bcf8be0 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.443+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd000f0 con 0x7ff3e804ec20 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.444+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.444+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94559003472176] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.444+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94559003474896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd00990 con 0x7ff3e804ec20 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.444+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfe610 con 0x56003bcf2b70 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.445+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94559003474896] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.446+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.446+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94559003477216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd01400 con 0x56003bcf2b70 2024-04-23T19:10:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.446+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd01dc0 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.447+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.447+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94559003482384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd021e0 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.447+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd02c60 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.447+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94559003477216] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.448+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94559003482384] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.448+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.448+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94559003486032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd01d30 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.448+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd032f0 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94559003486032] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94559003479888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd032f0 con 0x7ff3e002c060 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 20 add_watcher() i=0 2024-04-23T19:10:30.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 20 add_watcher() i=1 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 20 add_watcher() i=2 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 20 add_watcher() i=3 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 20 add_watcher() i=4 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 20 add_watcher() i=5 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.449+0000 7ff3f3726800 20 add_watcher() i=6 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.450+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94559003479888] v30'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.450+0000 7ff3f3726800 20 add_watcher() i=7 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.450+0000 7ff3f3726800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.450+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bd06480 con 0x56003bcf2b70 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.450+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.450+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94559003487808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd03710 con 0x56003bcf2b70 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.451+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94559003487808] v30'108 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.451+0000 7ff2a5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.451+0000 7ff3f3726800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.451+0000 7ff3f3726800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.451+0000 7ff3f3726800 5 note: GC not initialized 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.451+0000 7ff2a4fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.452+0000 7ff3f3726800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.452+0000 7ff3f3726800 20 Filter name: none 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.452+0000 7ff21e7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.452+0000 7ff3f3726800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.452+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56003bd39360 con 0x7ff3e804ec20 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.452+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv18 ondisk = 0) ==== 275+0+304 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.452+0000 7ff3f3726800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.453+0000 7ff2a5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.453+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94559003487808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd37320 con 0x56003bcf2b70 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.454+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94559003487808] v30'109 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.454+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94559003463280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd37320 con 0x7ff3e804ec20 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.455+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94559003463280] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.455+0000 7ff3f3726800 20 remove_watcher() i=0 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.455+0000 7ff3f3726800 2 removed watcher, disabling cache 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.455+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94559003467136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcfce70 con 0x56003bcf2b70 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.456+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94559003467136] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.456+0000 7ff3f3726800 20 remove_watcher() i=1 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.456+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94559003472176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd37320 con 0x56003bcf8be0 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.457+0000 7ff3efc29640 1 -- 172.21.15.63:0/3959075150 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94559003472176] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e800a4a0 con 0x56003bcf8be0 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.457+0000 7ff3f3726800 20 remove_watcher() i=2 2024-04-23T19:10:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.457+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94559003474896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd37320 con 0x7ff3e804ec20 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.458+0000 7ff3ef428640 1 -- 172.21.15.63:0/3959075150 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94559003474896] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e003f070 con 0x7ff3e804ec20 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.458+0000 7ff3f3726800 20 remove_watcher() i=3 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.458+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94559003477216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcffbd0 con 0x56003bcf2b70 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.459+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94559003477216] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x56003bcf2b70 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.459+0000 7ff3f3726800 20 remove_watcher() i=4 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.459+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94559003482384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd37320 con 0x7ff3e002c060 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.460+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94559003482384] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.460+0000 7ff3f3726800 20 remove_watcher() i=5 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.460+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94559003486032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bd004e0 con 0x7ff3e002c060 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.461+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94559003486032] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.461+0000 7ff3f3726800 20 remove_watcher() i=6 2024-04-23T19:10:30.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.461+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94559003479888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56003bcffbd0 con 0x7ff3e002c060 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94559003479888] v30'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff3e405a040 con 0x7ff3e002c060 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 20 remove_watcher() i=7 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff3e002c060 msgr2=0x7ff3e004c370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff3e002c060 0x7ff3e004c370 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bcf2b70 msgr2=0x56003bcf5030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bcf2b70 0x56003bcf5030 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff3e804ec20 msgr2=0x7ff3e8046860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff3e804ec20 0x7ff3e8046860 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56003bcf8be0 msgr2=0x56003bcfb030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56003bcf8be0 0x56003bcfb030 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2b8036e60 msgr2=0x7ff2b8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2b8036e60 0x7ff2b8039320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0046010 tx=0x7ff3e0043000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 msgr2=0x56003bce7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bce7000 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3e804bc10 tx=0x7ff3e804cc80 comp rx=0 tx=0).stop 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3eec27640 1 -- 172.21.15.63:0/3959075150 reap_dead start 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 shutdown_connections 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/3959075150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcdfb00 0x56003bce7000 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 >> 172.21.15.63:0/3959075150 conn(0x56003bcdbdd0 msgr2=0x56003bcd9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 shutdown_connections 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/3959075150 wait complete. 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2c8036eb0 msgr2=0x7ff2c8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2c8036eb0 0x7ff2c8039370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x56003bcc4820 tx=0x7ff3e000ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcd27d0 msgr2=0x56003bcd2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcd27d0 0x56003bcd2bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x56003bcbece0 tx=0x7ff3e80178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 shutdown_connections 2024-04-23T19:10:30.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2c8036eb0 0x7ff2c8039370 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 --2- 172.21.15.63:0/4184961302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bcd27d0 0x56003bcd2bf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 >> 172.21.15.63:0/4184961302 conn(0x56003bcc6870 msgr2=0x56003bcc6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 shutdown_connections 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.463+0000 7ff3f3726800 1 -- 172.21.15.63:0/4184961302 wait complete. 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56003bcc1ad0 msgr2=0x56003bcc3ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56003bcc1ad0 0x56003bcc3ec0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bbc0620 msgr2=0x56003bcbe350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bbc0620 0x56003bcbe350 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2d4036e60 msgr2=0x7ff2d4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2d4036e60 0x7ff2d4039320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4054000 tx=0x7ff3e404e000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 msgr2=0x56003bbbcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bbbcce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56003bcae0e0 tx=0x7ff3e0000bc0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 shutdown_connections 2024-04-23T19:10:30.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56003bcc1ad0 0x56003bcc3ec0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56003bbc0620 0x56003bcbe350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff2d4036e60 0x7ff2d4039320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 --2- 172.21.15.63:0/2190099758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56003bbba400 0x56003bbbcce0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 >> 172.21.15.63:0/2190099758 conn(0x56003bbb5db0 msgr2=0x56003bca7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 shutdown_connections 2024-04-23T19:10:30.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.464+0000 7ff3f3726800 1 -- 172.21.15.63:0/2190099758 wait complete. 2024-04-23T19:10:30.475 INFO:teuthology.orchestra.run.smithi063.stdout:{"key":"bucket:myfoo","ver":{"tag":"_siR-_l8IW5NHLzKGDeVzF_v","ver":2},"mtime":"2024-04-23T19:10:30.161942Z","data":{"bucket":{"name":"myfoo","marker":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","bucket_id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","tenant":"","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"foo","creation_time":"2024-04-23T19:10:29.008577Z","linked":false,"has_bucket_info":false}} 2024-04-23T19:10:30.475 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_siR-_l8IW5NHLzKGDeVzF_v', 'ver': 2}, 'mtime': '2024-04-23T19:10:30.161942Z', 'data': {'bucket': {'name': 'myfoo', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'bucket_id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-04-23T19:10:29.008577Z', 'linked': False, 'has_bucket_info': False}} 2024-04-23T19:10:30.475 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1'] 2024-04-23T19:10:30.476 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1'] 2024-04-23T19:10:30.476 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:30.524 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:30.524 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f389211f800 1 Processor -- start 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f389211f800 1 -- start start 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f389211f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x555e4e7ca590 0x555e4e7cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f389211f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555e4e6787f0 con 0x555e4e7c61a0 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f389211f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x555e4e667e20 con 0x555e4e7ca590 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f388d5e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x555e4e7ca590 0x555e4e7cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f388dde7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x555e4e7c61a0 0x555e4e7c65c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38794/0 (socket says 172.21.15.63:38794) 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f388dde7640 1 -- 172.21.15.63:0/912917897 learned_addr learned my addr 172.21.15.63:0/912917897 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f388d5e6640 1 -- 172.21.15.63:0/912917897 >> v1:172.21.15.16:6789/0 conn(0x555e4e7c61a0 legacy=0x555e4e7c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.525+0000 7f388d5e6640 1 -- 172.21.15.63:0/912917897 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4e686b50 con 0x555e4e7ca590 2024-04-23T19:10:30.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.527+0000 7f388d5e6640 1 --2- 172.21.15.63:0/912917897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7ca590 0x555e4e7cc980 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f387800a470 tx=0x7f387801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65249d87f933d14e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.527+0000 7f388cde5640 1 -- 172.21.15.63:0/912917897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878057070 con 0x555e4e7ca590 2024-04-23T19:10:30.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.527+0000 7f388cde5640 1 -- 172.21.15.63:0/912917897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3878052570 con 0x555e4e7ca590 2024-04-23T19:10:30.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.527+0000 7f388cde5640 1 -- 172.21.15.63:0/912917897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878052870 con 0x555e4e7ca590 2024-04-23T19:10:30.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.528+0000 7f389211f800 1 -- 172.21.15.63:0/912917897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7ca590 msgr2=0x555e4e7cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.528+0000 7f389211f800 1 --2- 172.21.15.63:0/912917897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7ca590 0x555e4e7cc980 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f387800a470 tx=0x7f387801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.528+0000 7f389211f800 1 -- 172.21.15.63:0/912917897 shutdown_connections 2024-04-23T19:10:30.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.528+0000 7f389211f800 1 --2- 172.21.15.63:0/912917897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7ca590 0x555e4e7cc980 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.528+0000 7f389211f800 1 -- 172.21.15.63:0/912917897 >> 172.21.15.63:0/912917897 conn(0x555e4e7c1800 msgr2=0x555e4e7c3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.528+0000 7f389211f800 1 -- 172.21.15.63:0/912917897 shutdown_connections 2024-04-23T19:10:30.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.528+0000 7f389211f800 1 -- 172.21.15.63:0/912917897 wait complete. 2024-04-23T19:10:30.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.541+0000 7f389211f800 1 Processor -- start 2024-04-23T19:10:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.541+0000 7f389211f800 1 -- start start 2024-04-23T19:10:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.541+0000 7f389211f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e8b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.541+0000 7f389211f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e4e711710 con 0x555e4e7c61a0 2024-04-23T19:10:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.541+0000 7f388dde7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e8b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.541+0000 7f388dde7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e8b7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54558/0 (socket says 172.21.15.63:54558) 2024-04-23T19:10:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.541+0000 7f388dde7640 1 -- 172.21.15.63:0/2505394501 learned_addr learned my addr 172.21.15.63:0/2505394501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.542+0000 7f388dde7640 1 -- 172.21.15.63:0/2505394501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4e8b7a10 con 0x555e4e7c61a0 2024-04-23T19:10:30.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.542+0000 7f388dde7640 1 --2- 172.21.15.63:0/2505394501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e8b7440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3880001e10 tx=0x7f388000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2603b65463605a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.542+0000 7f377be07640 1 -- 172.21.15.63:0/2505394501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3880012020 con 0x555e4e7c61a0 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.542+0000 7f377be07640 1 -- 172.21.15.63:0/2505394501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388000e5a0 con 0x555e4e7c61a0 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.542+0000 7f377be07640 1 -- 172.21.15.63:0/2505394501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388000e8e0 con 0x555e4e7c61a0 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 -- 172.21.15.63:0/2505394501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 msgr2=0x555e4e8b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 --2- 172.21.15.63:0/2505394501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e8b7440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3880001e10 tx=0x7f388000b780 comp rx=0 tx=0).stop 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 -- 172.21.15.63:0/2505394501 shutdown_connections 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 --2- 172.21.15.63:0/2505394501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e8b7440 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 -- 172.21.15.63:0/2505394501 >> 172.21.15.63:0/2505394501 conn(0x555e4e7c1800 msgr2=0x555e4e8b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 -- 172.21.15.63:0/2505394501 shutdown_connections 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 -- 172.21.15.63:0/2505394501 wait complete. 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 Processor -- start 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 -- start start 2024-04-23T19:10:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e73ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f389211f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e4e7cd250 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f388dde7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e73ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f388dde7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e73ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54562/0 (socket says 172.21.15.63:54562) 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.543+0000 7f388dde7640 1 -- 172.21.15.63:0/1283585937 learned_addr learned my addr 172.21.15.63:0/1283585937 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.544+0000 7f388dde7640 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4e73bfa0 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.544+0000 7f388dde7640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e73ba60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f388000b7c0 tx=0x7f3880000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.544+0000 7f377a604640 1 -- 172.21.15.63:0/1283585937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3880012030 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.544+0000 7f377a604640 1 -- 172.21.15.63:0/1283585937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3880001470 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.544+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e4e73a1a0 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.544+0000 7f377a604640 1 -- 172.21.15.63:0/1283585937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388001a8d0 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.544+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e4e73a6c0 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f377a604640 1 -- 172.21.15.63:0/1283585937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3880015020 con 0x555e4e7c61a0 2024-04-23T19:10:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f377a604640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3774036eb0 0x7f3774039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f377a604640 1 -- 172.21.15.63:0/1283585937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f3880020030 con 0x555e4e7c61a0 2024-04-23T19:10:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f388d5e6640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3774036eb0 0x7f3774039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e7ca590 0x555e4e8c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e8c89c0 con 0x555e4e7ca590 2024-04-23T19:10:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f388e5e8640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e7ca590 0x555e4e8c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.545+0000 7f388d5e6640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3774036eb0 0x7f3774039370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3878054000 tx=0x7f387804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.546+0000 7f388e5e8640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e7ca590 0x555e4e8c8440 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.546+0000 7f388e5e8640 1 -- 172.21.15.63:0/1283585937 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f388800a4a0 con 0x555e4e7ca590 2024-04-23T19:10:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.546+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e73abc0 con 0x555e4e7ca590 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.546+0000 7f388e5e8640 1 -- 172.21.15.63:0/1283585937 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f388800a4a0 con 0x555e4e7ca590 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.546+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e8ca7c0 con 0x555e4e7ca590 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.546+0000 7f388e5e8640 1 -- 172.21.15.63:0/1283585937 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f388800a4a0 con 0x555e4e7ca590 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.546+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e8c9b00 con 0x555e4e7ca590 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.547+0000 7f388e5e8640 1 -- 172.21.15.63:0/1283585937 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f388800a4a0 con 0x555e4e7ca590 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.547+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555e4e8cbb70 0x555e4e8cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.547+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e8ce4e0 con 0x555e4e8cbb70 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.547+0000 7f388dde7640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555e4e8cbb70 0x555e4e8cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.547+0000 7f388dde7640 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555e4e8cbb70 0x555e4e8cdf60 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.547+0000 7f388dde7640 1 -- 172.21.15.63:0/1283585937 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3880018950 con 0x555e4e8cbb70 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f389211f800 1 Processor -- start 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f389211f800 1 -- start start 2024-04-23T19:10:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f389211f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8dc890 0x555e4e8dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f389211f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e4e6b7e30 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f388e5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8dc890 0x555e4e8dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f388e5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8dc890 0x555e4e8dccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54570/0 (socket says 172.21.15.63:54570) 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f388e5e8640 1 -- 172.21.15.63:0/387062629 learned_addr learned my addr 172.21.15.63:0/387062629 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.548+0000 7f388e5e8640 1 -- 172.21.15.63:0/387062629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4e8dd1f0 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.549+0000 7f388e5e8640 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8dc890 0x555e4e8dccb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x555e4e8c8d80 tx=0x7f38880178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.549+0000 7f37737fe640 1 -- 172.21.15.63:0/387062629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3888013440 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.549+0000 7f37737fe640 1 -- 172.21.15.63:0/387062629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38880135e0 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.549+0000 7f37737fe640 1 -- 172.21.15.63:0/387062629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38880138c0 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.549+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e4e8ddc60 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.549+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e4e8de160 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.549+0000 7f37737fe640 1 -- 172.21.15.63:0/387062629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3888011040 con 0x555e4e8dc890 2024-04-23T19:10:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f37737fe640 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3768036e60 0x7f3768039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f37737fe640 1 -- 172.21.15.63:0/387062629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f388801d070 con 0x555e4e8dc890 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f388dde7640 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3768036e60 0x7f3768039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f389211f800 1 Processor -- start 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f389211f800 1 -- start start 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f388dde7640 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3768036e60 0x7f3768039320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3880018c60 tx=0x7f388004b190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f389211f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f389211f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e4e8b7d90 con 0x555e4e8e9c70 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f388e5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f388e5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8ea090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54576/0 (socket says 172.21.15.63:54576) 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.550+0000 7f388e5e8640 1 -- 172.21.15.63:0/2924505221 learned_addr learned my addr 172.21.15.63:0/2924505221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f388e5e8640 1 -- 172.21.15.63:0/2924505221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4e8ea660 con 0x555e4e8e9c70 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f388e5e8640 1 --2- 172.21.15.63:0/2924505221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8ea090 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38880129a0 tx=0x7f3888012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63f6dcbe3888d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f37717fa640 1 -- 172.21.15.63:0/2924505221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388802ad80 con 0x555e4e8e9c70 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f37717fa640 1 -- 172.21.15.63:0/2924505221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38880014f0 con 0x555e4e8e9c70 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f37717fa640 1 -- 172.21.15.63:0/2924505221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38880017d0 con 0x555e4e8e9c70 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f389211f800 1 -- 172.21.15.63:0/2924505221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 msgr2=0x555e4e8ea090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f389211f800 1 --2- 172.21.15.63:0/2924505221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8ea090 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38880129a0 tx=0x7f3888012a90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f389211f800 1 -- 172.21.15.63:0/2924505221 shutdown_connections 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f389211f800 1 --2- 172.21.15.63:0/2924505221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8ea090 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f389211f800 1 -- 172.21.15.63:0/2924505221 >> 172.21.15.63:0/2924505221 conn(0x555e4e8e5ef0 msgr2=0x555e4e8e3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f389211f800 1 -- 172.21.15.63:0/2924505221 shutdown_connections 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.551+0000 7f389211f800 1 -- 172.21.15.63:0/2924505221 wait complete. 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f389211f800 1 Processor -- start 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f389211f800 1 -- start start 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f389211f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f389211f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e4e8ead90 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f388e5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f388e5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54586/0 (socket says 172.21.15.63:54586) 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 learned_addr learned my addr 172.21.15.63:0/565971925 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.552+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4e8f1660 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.553+0000 7f388e5e8640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8f1120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38880129a0 tx=0x7f388804cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.553+0000 7f375f7fe640 1 -- 172.21.15.63:0/565971925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38880124d0 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.553+0000 7f375f7fe640 1 -- 172.21.15.63:0/565971925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3888012670 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.553+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e4e8f18f0 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.553+0000 7f375f7fe640 1 -- 172.21.15.63:0/565971925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388804e110 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.553+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e4e8f45a0 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f375f7fe640 1 -- 172.21.15.63:0/565971925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3888041070 con 0x555e4e8e9c70 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f375f7fe640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3758036eb0 0x7f3758039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f375f7fe640 1 -- 172.21.15.63:0/565971925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f388800e040 con 0x555e4e8e9c70 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f388dde7640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3758036eb0 0x7f3758039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e8fce30 0x555e4e8ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e8ffa40 con 0x555e4e8fce30 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f388dde7640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3758036eb0 0x7f3758039370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f388001a6f0 tx=0x7f3880036000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.554+0000 7f388d5e6640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e8fce30 0x555e4e8ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f388d5e6640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e8fce30 0x555e4e8ff2f0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 20 realm 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e8ffe10 con 0x555e4e8fce30 2024-04-23T19:10:30.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9006e0 con 0x555e4e8fce30 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.555+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e900a80 con 0x555e4e8fce30 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f389211f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9012d0 con 0x555e4e8fce30 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f389211f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f389211f800 20 searching for the correct realm 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555e4e903040 0x555e4e905430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f388e5e8640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555e4e903040 0x555e4e905430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.556+0000 7f388e5e8640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555e4e903040 0x555e4e905430 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.557+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.557+0000 7f388e5e8640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f388803b2a0 0x7f38880227b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.557+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388803b2a0 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.557+0000 7f388dde7640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f388803b2a0 0x7f38880227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.557+0000 7f388dde7640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f388803b2a0 0x7f38880227b0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.558+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.558+0000 7f388dde7640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f388003a040 0x7f388003f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.558+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.558+0000 7f388d5e6640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f388003a040 0x7f388003f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.558+0000 7f388d5e6640 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f388003a040 0x7f388003f700 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.559+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388803b2a0 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.560+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.561+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.561+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.561+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.561+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388803b2a0 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.561+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.561+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.561+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.562+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.563+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388803b2a0 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.564+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.564+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.564+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.564+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.564+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.564+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388803b2a0 2024-04-23T19:10:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e903040 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.565+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388003a040 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x555e4e8fce30 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9059c0 con 0x7f388803b2a0 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.566+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e901c50 con 0x555e4e8fce30 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9024f0 con 0x555e4e8fce30 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9024f0 con 0x7f388003a040 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 zone default found 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 4 Realm: () 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 zonegroup default 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.567+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e905e40 con 0x7f388003a040 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e902270 con 0x555e4e903040 2024-04-23T19:10:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e907df0 con 0x555e4e903040 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 20 started sync module instance, tier type = 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.568+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e907160 con 0x7f388803b2a0 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.569+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.569+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93863533376896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e908700 con 0x7f388803b2a0 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.569+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e908f50 con 0x555e4e8fce30 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.570+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.570+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93863533382304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9094f0 con 0x555e4e8fce30 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.570+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90a010 con 0x555e4e903040 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.570+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93863533376896] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.571+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93863533382304] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.571+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.572+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93863533386464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e908f40 con 0x555e4e903040 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.572+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90aa20 con 0x7f388803b2a0 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.572+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93863533386464] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.573+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.573+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93863533389168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90adc0 con 0x7f388803b2a0 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.573+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90b780 con 0x555e4e8fce30 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.574+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.574+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93863533393072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90bf40 con 0x555e4e8fce30 2024-04-23T19:10:30.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.574+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90ca60 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.574+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93863533389168] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.575+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93863533393072] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.575+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.575+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93863533397296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90ca20 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.575+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90d300 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.576+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93863533397296] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.576+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.576+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93863533380176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90ca20 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.576+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90d920 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93863533380176] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93863533399552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90d920 con 0x7f388003a040 2024-04-23T19:10:30.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 20 add_watcher() i=0 2024-04-23T19:10:30.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 20 add_watcher() i=1 2024-04-23T19:10:30.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 20 add_watcher() i=2 2024-04-23T19:10:30.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 20 add_watcher() i=3 2024-04-23T19:10:30.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 20 add_watcher() i=4 2024-04-23T19:10:30.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 20 add_watcher() i=5 2024-04-23T19:10:30.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.577+0000 7f389211f800 20 add_watcher() i=6 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.578+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93863533399552] v30'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.578+0000 7f389211f800 20 add_watcher() i=7 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.578+0000 7f389211f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.578+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e910c30 con 0x555e4e8fce30 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.578+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.578+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93863533401568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90deb0 con 0x555e4e8fce30 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.579+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93863533401568] v30'110 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.579+0000 7f373f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.579+0000 7f389211f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:30.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.579+0000 7f389211f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.579+0000 7f389211f800 5 note: GC not initialized 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f373e7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 20 Filter name: none 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f36affff640 20 reqs_thread_entry: start 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e944110 con 0x7f388003a040 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:30.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.580+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e946090 con 0x7f388803b2a0 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv18 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9479a0 con 0x555e4e8fce30 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(64 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:30.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.581+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e94bf80 con 0x7f388803b2a0 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.582+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(65 foo.buckets [call] v30'19 uv17 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.582+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e951ee0 con 0x555e4e8fce30 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(66 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v30'4 uv4 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:30.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 93863533638224 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e94c3b0 con 0x7f388003a040 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== watch-notify(notify (1) cookie 93863533397296 notify 128849018884 ret 0) ==== 794+0+0 (crc 0 0 0) 0x555e4e947d90 con 0x7f388003a040 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f37717fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018884 cookie 93863533397296 notifier 4499 bl.length()=752 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f37717fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f37717fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f37717fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(67 notify.5 [notify cookie 93863533638224 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f37717fa640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f3750003ea0 con 0x7f388003a040 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== watch-notify(notify_complete (2) cookie 93863533638224 notify 128849018884 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555e4e946ef0 con 0x7f388003a040 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_sources= new_sources= 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential sources added=[] removed=[] 2024-04-23T19:10:30.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_dests= new_dests= 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential dests added=[] removed=[] 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.584+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9506a0 con 0x7f388003a040 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.585+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(69 fud.buckets [call] v30'1 uv1 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.585+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e9516b0 con 0x7f388803b2a0 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v30'20 uv20 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f389211f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f389211f800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f389211f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93863533638224 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x555e4e950b50 con 0x7f388003a040 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== watch-notify(notify (1) cookie 93863533380176 notify 128849018885 ret 0) ==== 356+0+0 (crc 0 0 0) 0x555e4e686b50 con 0x7f388003a040 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f37717fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018885 cookie 93863533380176 notifier 4499 bl.length()=314 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f37717fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-23T19:10:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f37717fa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f37717fa640 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f3750003ea0 con 0x7f388003a040 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(71 notify.6 [notify cookie 93863533638224 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.587+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== watch-notify(notify_complete (2) cookie 93863533638224 notify 128849018885 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f387808bab0 con 0x7f388003a040 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.588+0000 7f373f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.588+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93863533401568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e948930 con 0x555e4e8fce30 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.589+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 93863533401568] v30'111 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.589+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93863533376896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e948930 con 0x7f388803b2a0 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.590+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 93863533376896] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.590+0000 7f389211f800 20 remove_watcher() i=0 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.590+0000 7f389211f800 2 removed watcher, disabling cache 2024-04-23T19:10:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.590+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93863533382304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e907580 con 0x555e4e8fce30 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.591+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 93863533382304] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.591+0000 7f389211f800 20 remove_watcher() i=1 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.591+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93863533386464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e907580 con 0x555e4e903040 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.592+0000 7f388e5e8640 1 -- 172.21.15.63:0/565971925 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 93863533386464] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3888029070 con 0x555e4e903040 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.592+0000 7f389211f800 20 remove_watcher() i=2 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.592+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93863533389168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e909ae0 con 0x7f388803b2a0 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.593+0000 7f388dde7640 1 -- 172.21.15.63:0/565971925 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 93863533389168] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3880053040 con 0x7f388803b2a0 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.593+0000 7f389211f800 20 remove_watcher() i=3 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.593+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93863533393072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e907580 con 0x555e4e8fce30 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.594+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 93863533393072] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x555e4e8fce30 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.594+0000 7f389211f800 20 remove_watcher() i=4 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.594+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93863533397296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90a570 con 0x7f388003a040 2024-04-23T19:10:30.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.595+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 93863533397296] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.595+0000 7f389211f800 20 remove_watcher() i=5 2024-04-23T19:10:30.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.595+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93863533380176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e90b4b0 con 0x7f388003a040 2024-04-23T19:10:30.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.596+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 93863533380176] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.596+0000 7f389211f800 20 remove_watcher() i=6 2024-04-23T19:10:30.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.596+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93863533399552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x555e4e907580 con 0x7f388003a040 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 93863533399552] v30'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f387805a040 con 0x7f388003a040 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 20 remove_watcher() i=7 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f388003a040 msgr2=0x7f388003f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f388003a040 0x7f388003f700 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e8fce30 msgr2=0x555e4e8ff2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e8fce30 0x555e4e8ff2f0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f388803b2a0 msgr2=0x7f38880227b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f388803b2a0 0x7f38880227b0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555e4e903040 msgr2=0x555e4e905430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555e4e903040 0x555e4e905430 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3758036eb0 msgr2=0x7f3758039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3758036eb0 0x7f3758039370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f388001a6f0 tx=0x7f3880036000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 msgr2=0x555e4e8f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.597+0000 7f389211f800 1 --2- 172.21.15.63:0/565971925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8e9c70 0x555e4e8f1120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38880129a0 tx=0x7f388804cbd0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 reap_dead start 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f388d5e6640 1 -- 172.21.15.63:0/565971925 reap_dead start 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 shutdown_connections 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 >> 172.21.15.63:0/565971925 conn(0x555e4e8e5ef0 msgr2=0x555e4e8e3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 shutdown_connections 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/565971925 wait complete. 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3768036e60 msgr2=0x7f3768039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3768036e60 0x7f3768039320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3880018c60 tx=0x7f388004b190 comp rx=0 tx=0).stop 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8dc890 msgr2=0x555e4e8dccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8dc890 0x555e4e8dccb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x555e4e8c8d80 tx=0x7f38880178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 shutdown_connections 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3768036e60 0x7f3768039320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/387062629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e8dc890 0x555e4e8dccb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 >> 172.21.15.63:0/387062629 conn(0x555e4e8d0930 msgr2=0x555e4e8d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 shutdown_connections 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/387062629 wait complete. 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555e4e8cbb70 msgr2=0x555e4e8cdf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555e4e8cbb70 0x555e4e8cdf60 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e7ca590 msgr2=0x555e4e8c8440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e7ca590 0x555e4e8c8440 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3774036eb0 msgr2=0x7f3774039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3774036eb0 0x7f3774039370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3878054000 tx=0x7f387804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 msgr2=0x555e4e73ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.598+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e73ba60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f388000b7c0 tx=0x7f3880000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 shutdown_connections 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555e4e8cbb70 0x555e4e8cdf60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555e4e7ca590 0x555e4e8c8440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3774036eb0 0x7f3774039370 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 --2- 172.21.15.63:0/1283585937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4e7c61a0 0x555e4e73ba60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 >> 172.21.15.63:0/1283585937 conn(0x555e4e7c1800 msgr2=0x555e4e8b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 shutdown_connections 2024-04-23T19:10:30.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.599+0000 7f389211f800 1 -- 172.21.15.63:0/1283585937 wait complete. 2024-04-23T19:10:30.613 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-23T19:10:30.613 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-23T19:10:30.613 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-23T19:10:30.657 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:30.657 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:30.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.657+0000 7fd1f67af800 1 Processor -- start 2024-04-23T19:10:30.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.657+0000 7fd1f67af800 1 -- start start 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.657+0000 7fd1f67af800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55d2a2e6f180 0x55d2a2e6f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.657+0000 7fd1f67af800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d2a2d217f0 con 0x55d2a2e70340 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.657+0000 7fd1f67af800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55d2a2d10e20 con 0x55d2a2e6f180 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.658+0000 7fd1f42b5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55d2a2e6f180 0x55d2a2e6f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.658+0000 7fd1f3ab4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55d2a2e70340 0x55d2a2e707c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38798/0 (socket says 172.21.15.63:38798) 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.658+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3767475648 learned_addr learned my addr 172.21.15.63:0/3767475648 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.658+0000 7fd1f32b3640 1 -- 172.21.15.63:0/3767475648 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369764228 0 0) 0x55d2a2d217f0 con 0x55d2a2e70340 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.658+0000 7fd1f32b3640 1 -- 172.21.15.63:0/3767475648 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1dc001e10 con 0x55d2a2e70340 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.658+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3767475648 >> v1:172.21.15.16:6789/0 conn(0x55d2a2e70340 legacy=0x55d2a2e707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.658+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3767475648 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2d2fb50 con 0x55d2a2e6f180 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.660+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3767475648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2e6f5a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400a470 tx=0x7fd1e401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16fcfac419ab4af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.660+0000 7fd1f32b3640 1 -- 172.21.15.63:0/3767475648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e4057070 con 0x55d2a2e6f180 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.660+0000 7fd1f32b3640 1 -- 172.21.15.63:0/3767475648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e4052570 con 0x55d2a2e6f180 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.660+0000 7fd1f32b3640 1 -- 172.21.15.63:0/3767475648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e4052850 con 0x55d2a2e6f180 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.660+0000 7fd1f67af800 1 -- 172.21.15.63:0/3767475648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 msgr2=0x55d2a2e6f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.660+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3767475648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2e6f5a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400a470 tx=0x7fd1e401cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.661+0000 7fd1f67af800 1 -- 172.21.15.63:0/3767475648 shutdown_connections 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.661+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3767475648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2e6f5a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.661+0000 7fd1f67af800 1 -- 172.21.15.63:0/3767475648 >> 172.21.15.63:0/3767475648 conn(0x55d2a2e6a7e0 msgr2=0x55d2a2e6cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.661+0000 7fd1f67af800 1 -- 172.21.15.63:0/3767475648 shutdown_connections 2024-04-23T19:10:30.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.661+0000 7fd1f67af800 1 -- 172.21.15.63:0/3767475648 wait complete. 2024-04-23T19:10:30.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.673+0000 7fd1f67af800 1 Processor -- start 2024-04-23T19:10:30.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.673+0000 7fd1f67af800 1 -- start start 2024-04-23T19:10:30.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.673+0000 7fd1f67af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2de4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.673+0000 7fd1f67af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2dba6f0 con 0x55d2a2e6f180 2024-04-23T19:10:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd1f42b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2de4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd1f42b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2de4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54604/0 (socket says 172.21.15.63:54604) 2024-04-23T19:10:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd1f42b5640 1 -- 172.21.15.63:0/1293140831 learned_addr learned my addr 172.21.15.63:0/1293140831 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd1f42b5640 1 -- 172.21.15.63:0/1293140831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2de3050 con 0x55d2a2e6f180 2024-04-23T19:10:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/1293140831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2de4a90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4055040 tx=0x7fd1e404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3c164e48cab7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd0e260c640 1 -- 172.21.15.63:0/1293140831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e4057040 con 0x55d2a2e6f180 2024-04-23T19:10:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd0e260c640 1 -- 172.21.15.63:0/1293140831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e4052950 con 0x55d2a2e6f180 2024-04-23T19:10:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.674+0000 7fd0e260c640 1 -- 172.21.15.63:0/1293140831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e4052c70 con 0x55d2a2e6f180 2024-04-23T19:10:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 -- 172.21.15.63:0/1293140831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 msgr2=0x55d2a2de4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 --2- 172.21.15.63:0/1293140831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2de4a90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4055040 tx=0x7fd1e404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 -- 172.21.15.63:0/1293140831 shutdown_connections 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 --2- 172.21.15.63:0/1293140831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2de4a90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 -- 172.21.15.63:0/1293140831 >> 172.21.15.63:0/1293140831 conn(0x55d2a2e6a7e0 msgr2=0x55d2a2f5be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 -- 172.21.15.63:0/1293140831 shutdown_connections 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 -- 172.21.15.63:0/1293140831 wait complete. 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 Processor -- start 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.675+0000 7fd1f67af800 1 -- start start 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd1f67af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2f6a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd1f67af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2e787b0 con 0x55d2a2e6f180 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd1f42b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2f6a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd1f42b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2f6a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54606/0 (socket says 172.21.15.63:54606) 2024-04-23T19:10:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3736830656 learned_addr learned my addr 172.21.15.63:0/3736830656 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2f6a7e0 con 0x55d2a2e6f180 2024-04-23T19:10:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2f6a2a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4055040 tx=0x7fd1e4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.676+0000 7fd0e0e09640 1 -- 172.21.15.63:0/3736830656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e4073070 con 0x55d2a2e6f180 2024-04-23T19:10:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.677+0000 7fd0e0e09640 1 -- 172.21.15.63:0/3736830656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e405d590 con 0x55d2a2e6f180 2024-04-23T19:10:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.677+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a2f6aa70 con 0x55d2a2e6f180 2024-04-23T19:10:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.677+0000 7fd0e0e09640 1 -- 172.21.15.63:0/3736830656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e4057070 con 0x55d2a2e6f180 2024-04-23T19:10:30.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.677+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a2f6d720 con 0x55d2a2e6f180 2024-04-23T19:10:30.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.677+0000 7fd0e0e09640 1 -- 172.21.15.63:0/3736830656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd1e4076070 con 0x55d2a2e6f180 2024-04-23T19:10:30.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.677+0000 7fd0e0e09640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0dc036eb0 0x7fd0dc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.678+0000 7fd0e0e09640 1 -- 172.21.15.63:0/3736830656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd1e408e330 con 0x55d2a2e6f180 2024-04-23T19:10:30.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.678+0000 7fd1f3ab4640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0dc036eb0 0x7fd0dc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.678+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2e70340 0x55d2a2f714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.678+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2f71a60 con 0x55d2a2e70340 2024-04-23T19:10:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.678+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2e70340 0x55d2a2f714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.678+0000 7fd1f3ab4640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0dc036eb0 0x7fd0dc039370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000f60 tx=0x7fd1e80010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.678+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2e70340 0x55d2a2f714e0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.679+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3736830656 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1ec00a4a0 con 0x55d2a2e70340 2024-04-23T19:10:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.679+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2f6ea90 con 0x55d2a2e70340 2024-04-23T19:10:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.679+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3736830656 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd1ec00a4a0 con 0x55d2a2e70340 2024-04-23T19:10:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.679+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2f73860 con 0x55d2a2e70340 2024-04-23T19:10:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.679+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3736830656 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd1ec00a4a0 con 0x55d2a2e70340 2024-04-23T19:10:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.679+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2f72ba0 con 0x55d2a2e70340 2024-04-23T19:10:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.679+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3736830656 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd1ec00a4a0 con 0x55d2a2e70340 2024-04-23T19:10:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.680+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d2a2f74c10 0x55d2a2f77000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.680+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2f77580 con 0x55d2a2f74c10 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.680+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d2a2f74c10 0x55d2a2f77000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.680+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d2a2f74c10 0x55d2a2f77000 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.680+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3736830656 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd1e4075030 con 0x55d2a2f74c10 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f67af800 1 Processor -- start 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f67af800 1 -- start start 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f67af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f85930 0x55d2a2f85d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f67af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2d60e30 con 0x55d2a2f85930 2024-04-23T19:10:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f4ab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f85930 0x55d2a2f85d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f4ab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f85930 0x55d2a2f85d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54614/0 (socket says 172.21.15.63:54614) 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3705423377 learned_addr learned my addr 172.21.15.63:0/3705423377 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.681+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3705423377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2f86290 con 0x55d2a2f85930 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.682+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f85930 0x55d2a2f85d50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55d2a2f6b6a0 tx=0x7fd1ec0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.682+0000 7fd0d9ffb640 1 -- 172.21.15.63:0/3705423377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ec013440 con 0x55d2a2f85930 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.682+0000 7fd0d9ffb640 1 -- 172.21.15.63:0/3705423377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec0135e0 con 0x55d2a2f85930 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.682+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a2f86d00 con 0x55d2a2f85930 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.682+0000 7fd0d9ffb640 1 -- 172.21.15.63:0/3705423377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ec0138c0 con 0x55d2a2f85930 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.682+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a2f87200 con 0x55d2a2f85930 2024-04-23T19:10:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd0d9ffb640 1 -- 172.21.15.63:0/3705423377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd1ec011040 con 0x55d2a2f85930 2024-04-23T19:10:30.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd0d9ffb640 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0cc036e60 0x7fd0cc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd0d9ffb640 1 -- 172.21.15.63:0/3705423377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd1ec01d070 con 0x55d2a2f85930 2024-04-23T19:10:30.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0cc036e60 0x7fd0cc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f67af800 1 Processor -- start 2024-04-23T19:10:30.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f67af800 1 -- start start 2024-04-23T19:10:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0cc036e60 0x7fd0cc039320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55d2a2f77960 tx=0x7fd1e408c710 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f67af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f67af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2de3310 con 0x55d2a2f92c90 2024-04-23T19:10:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f4ab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.683+0000 7fd1f4ab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f930b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54616/0 (socket says 172.21.15.63:54616) 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.684+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/2964122519 learned_addr learned my addr 172.21.15.63:0/2964122519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.684+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/2964122519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2f93680 con 0x55d2a2f92c90 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.684+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/2964122519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f930b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec04ccb0 tx=0x7fd1ec001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7238fb11acf444f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.684+0000 7fd0d37fe640 1 -- 172.21.15.63:0/2964122519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ec012df0 con 0x55d2a2f92c90 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.684+0000 7fd0d37fe640 1 -- 172.21.15.63:0/2964122519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec001310 con 0x55d2a2f92c90 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.684+0000 7fd0d37fe640 1 -- 172.21.15.63:0/2964122519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ec001630 con 0x55d2a2f92c90 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 -- 172.21.15.63:0/2964122519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 msgr2=0x55d2a2f930b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 --2- 172.21.15.63:0/2964122519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f930b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec04ccb0 tx=0x7fd1ec001f90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 -- 172.21.15.63:0/2964122519 shutdown_connections 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 --2- 172.21.15.63:0/2964122519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f930b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 -- 172.21.15.63:0/2964122519 >> 172.21.15.63:0/2964122519 conn(0x55d2a2f8efb0 msgr2=0x55d2a2f8d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 -- 172.21.15.63:0/2964122519 shutdown_connections 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 -- 172.21.15.63:0/2964122519 wait complete. 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 Processor -- start 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 -- start start 2024-04-23T19:10:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f9a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f67af800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2f93e00 con 0x55d2a2f92c90 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.685+0000 7fd1f4ab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f9a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd1f4ab6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f9a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54626/0 (socket says 172.21.15.63:54626) 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 learned_addr learned my addr 172.21.15.63:0/3581489750 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2f9a7c0 con 0x55d2a2f92c90 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f9a280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec04bb80 tx=0x7fd1ec04cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd0d1ffb640 1 -- 172.21.15.63:0/3581489750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ec021c60 con 0x55d2a2f92c90 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd0d1ffb640 1 -- 172.21.15.63:0/3581489750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec021e00 con 0x55d2a2f92c90 2024-04-23T19:10:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a2f9aa50 con 0x55d2a2f92c90 2024-04-23T19:10:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd0d1ffb640 1 -- 172.21.15.63:0/3581489750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ec02a350 con 0x55d2a2f92c90 2024-04-23T19:10:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.686+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a2f9d700 con 0x55d2a2f92c90 2024-04-23T19:10:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.687+0000 7fd0d1ffb640 1 -- 172.21.15.63:0/3581489750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd1ec040040 con 0x55d2a2f92c90 2024-04-23T19:10:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.687+0000 7fd0d1ffb640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0c4036e60 0x7fd0c4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.687+0000 7fd0d1ffb640 1 -- 172.21.15.63:0/3581489750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd1ec00e040 con 0x55d2a2f92c90 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.687+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0c4036e60 0x7fd0c4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0c4036e60 0x7fd0c4039320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4052d50 tx=0x7fd1e4089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2fa5f20 0x55d2a2fa83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fa8b30 con 0x55d2a2fa5f20 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f3ab4640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2fa5f20 0x55d2a2fa83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f3ab4640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2fa5f20 0x55d2a2fa83e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1e8009050 con 0x55d2a2fa5f20 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f67af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f67af800 20 realm 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.688+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fa5370 con 0x55d2a2fa5f20 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1e8009050 con 0x55d2a2fa5f20 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fa9620 con 0x55d2a2fa5f20 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1e8009050 con 0x55d2a2fa5f20 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fa9ef0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd1e8009050 con 0x55d2a2fa5f20 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faa290 con 0x55d2a2fa5f20 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd1e8009050 con 0x55d2a2fa5f20 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 20 searching for the correct realm 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d2a2fac170 0x55d2a2fae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.689+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d2a2fac170 0x55d2a2fae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.690+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d2a2fac170 0x55d2a2fae5c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.690+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.690+0000 7fd1f4ab6640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1ec05b040 0x7fd1ec0277f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.690+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1ec05b040 2024-04-23T19:10:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.690+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1ec05b040 0x7fd1ec0277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.691+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1ec05b040 0x7fd1ec0277f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.691+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.691+0000 7fd1f42b5640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1e409c040 0x7fd1e408c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.691+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.691+0000 7fd1f3ab4640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1e409c040 0x7fd1e408c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.691+0000 7fd1f3ab4640 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1e409c040 0x7fd1e408c8c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8009050 con 0x7fd1e409c040 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8009050 con 0x55d2a2fa5f20 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.692+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1ec05b040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.693+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1ec05b040 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.694+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.695+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.695+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.695+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.695+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.695+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.695+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.696+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1ec05b040 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.697+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1ec05b040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fac170 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.698+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1e409c040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x55d2a2fa5f20 2024-04-23T19:10:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faec90 con 0x7fd1ec05b040 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.699+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faac40 con 0x55d2a2fa5f20 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fab4e0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faf110 con 0x7fd1e409c040 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 zone default found 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 4 Realm: () 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:30.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 zonegroup default 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.700+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2faf5c0 con 0x7fd1e409c040 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb0fa0 con 0x55d2a2fac170 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb1460 con 0x55d2a2fac170 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 20 started sync module instance, tier type = 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.701+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb0430 con 0x7fd1ec05b040 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.702+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.702+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94363165853728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb0090 con 0x7fd1ec05b040 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.702+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb2390 con 0x55d2a2fa5f20 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.703+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.703+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94363165860032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb2b30 con 0x55d2a2fa5f20 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.703+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb3630 con 0x55d2a2fac170 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.703+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94363165853728] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.705+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94363165860032] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.705+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.705+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94363165864224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb2580 con 0x55d2a2fac170 2024-04-23T19:10:30.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.705+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb4070 con 0x7fd1ec05b040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.706+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94363165864224] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.706+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.706+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94363165866944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb4410 con 0x7fd1ec05b040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.706+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb4dd0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.707+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94363165866944] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.707+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.707+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94363165870368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb51f0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.707+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb5bb0 con 0x7fd1e409c040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.708+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94363165870368] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.708+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.708+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94363165871904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb5bb0 con 0x7fd1e409c040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.708+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb6570 con 0x7fd1e409c040 2024-04-23T19:10:30.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.709+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94363165871904] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.709+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.709+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94363165876896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb5bb0 con 0x7fd1e409c040 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.709+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb6f80 con 0x7fd1e409c040 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94363165876896] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94363165874400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb6f80 con 0x7fd1e409c040 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 20 add_watcher() i=0 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 20 add_watcher() i=1 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 20 add_watcher() i=2 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 20 add_watcher() i=3 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 20 add_watcher() i=4 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 20 add_watcher() i=5 2024-04-23T19:10:30.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.710+0000 7fd1f67af800 20 add_watcher() i=6 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.711+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94363165874400] v30'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.711+0000 7fd1f67af800 20 add_watcher() i=7 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.711+0000 7fd1f67af800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.711+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fba2c0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.712+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.712+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94363165879360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb7510 con 0x55d2a2fa5f20 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.713+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94363165879360] v30'112 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.713+0000 7fd0aa7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.713+0000 7fd1f67af800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.713+0000 7fd1f67af800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.713+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb9a50 con 0x55d2a2fa5f20 2024-04-23T19:10:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.714+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.714+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1e409c040 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.715+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.715+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.716+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.716+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fa5f20 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.717+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'113 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.717+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3c70 con 0x7fd1ec05b040 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.718+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'32 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.718+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc44c0 con 0x55d2a2fac170 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.719+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'21 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.719+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc44c0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.720+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.720+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc44c0 con 0x7fd1e409c040 2024-04-23T19:10:30.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.721+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'21 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.721+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc44c0 con 0x7fd1ec05b040 2024-04-23T19:10:30.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.722+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.722+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc4aa0 con 0x55d2a2fac170 2024-04-23T19:10:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.723+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'20 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.723+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1ec05b040 2024-04-23T19:10:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.724+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'33 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.724+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1e409c040 2024-04-23T19:10:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.725+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.725+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1ec05b040 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.726+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.726+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1e409c040 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.727+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'27 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.727+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fa5f20 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.728+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'114 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.728+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.729+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'27 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.729+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1ec05b040 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.730+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'34 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.730+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.731+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.731+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fa5f20 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.732+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.732+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fa5f20 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.733+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'115 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.733+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.734+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'28 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.734+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1ec05b040 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.735+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.735+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.736+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.736+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.737+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.737+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.738+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.738+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.739+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'22 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.739+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1e409c040 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.740+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.740+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.741+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.741+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1e409c040 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.742+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'22 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.742+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.743+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'29 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.743+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1e409c040 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.744+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'28 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.744+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fc3380 con 0x7fd1ec05b040 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.745+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.745+0000 7fd0a97e2640 20 reqs_thread_entry: start 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.745+0000 7fd1f67af800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.745+0000 7fd1f67af800 20 Filter name: none 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.745+0000 7fd01affd640 20 reqs_thread_entry: start 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.745+0000 7fd1f67af800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.745+0000 7fd1f67af800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fef980 con 0x7fd1e409c040 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f67af800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:10:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f67af800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f67af800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f67af800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2ff2290 con 0x7fd1e409c040 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.746+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(95 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.747+0000 7fd0aa7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.747+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94363165879360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fa23c0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.748+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94363165879360] v30'116 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.748+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94363165853728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb6c40 con 0x7fd1ec05b040 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.749+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94363165853728] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.749+0000 7fd1f67af800 20 remove_watcher() i=0 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.749+0000 7fd1f67af800 2 removed watcher, disabling cache 2024-04-23T19:10:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.749+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94363165860032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb6c40 con 0x55d2a2fa5f20 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.750+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94363165860032] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.750+0000 7fd1f67af800 20 remove_watcher() i=1 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.750+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94363165864224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb20c0 con 0x55d2a2fac170 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.751+0000 7fd1f4ab6640 1 -- 172.21.15.63:0/3581489750 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94363165864224] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ec029070 con 0x55d2a2fac170 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.751+0000 7fd1f67af800 20 remove_watcher() i=2 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.751+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94363165866944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb6c40 con 0x7fd1ec05b040 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.752+0000 7fd1f42b5640 1 -- 172.21.15.63:0/3581489750 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94363165866944] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e4075030 con 0x7fd1ec05b040 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.752+0000 7fd1f67af800 20 remove_watcher() i=3 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.752+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94363165870368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb20c0 con 0x55d2a2fa5f20 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.753+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94363165870368] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x55d2a2fa5f20 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.753+0000 7fd1f67af800 20 remove_watcher() i=4 2024-04-23T19:10:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.753+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94363165871904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb3bc0 con 0x7fd1e409c040 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.754+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94363165871904] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.754+0000 7fd1f67af800 20 remove_watcher() i=5 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.754+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94363165876896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb6c40 con 0x7fd1e409c040 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.755+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94363165876896] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.755+0000 7fd1f67af800 20 remove_watcher() i=6 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.755+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94363165874400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d2a2fb4920 con 0x7fd1e409c040 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94363165874400] v30'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e802a040 con 0x7fd1e409c040 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 20 remove_watcher() i=7 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1e409c040 msgr2=0x7fd1e408c8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1e409c040 0x7fd1e408c8c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2fa5f20 msgr2=0x55d2a2fa83e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2fa5f20 0x55d2a2fa83e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1ec05b040 msgr2=0x7fd1ec0277f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1ec05b040 0x7fd1ec0277f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d2a2fac170 msgr2=0x55d2a2fae5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d2a2fac170 0x55d2a2fae5c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0c4036e60 msgr2=0x7fd0c4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0c4036e60 0x7fd0c4039320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4052d50 tx=0x7fd1e4089000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 msgr2=0x55d2a2f9a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f9a280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec04bb80 tx=0x7fd1ec04cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f3ab4640 1 -- 172.21.15.63:0/3581489750 reap_dead start 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 shutdown_connections 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.756+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3581489750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f92c90 0x55d2a2f9a280 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 >> 172.21.15.63:0/3581489750 conn(0x55d2a2f8efb0 msgr2=0x55d2a2f8c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 shutdown_connections 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3581489750 wait complete. 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0cc036e60 msgr2=0x7fd0cc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0cc036e60 0x7fd0cc039320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55d2a2f77960 tx=0x7fd1e408c710 comp rx=0 tx=0).stop 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f85930 msgr2=0x55d2a2f85d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f85930 0x55d2a2f85d50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55d2a2f6b6a0 tx=0x7fd1ec0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 shutdown_connections 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0cc036e60 0x7fd0cc039320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3705423377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2f85930 0x55d2a2f85d50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 >> 172.21.15.63:0/3705423377 conn(0x55d2a2f799d0 msgr2=0x55d2a2f79dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 shutdown_connections 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3705423377 wait complete. 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d2a2f74c10 msgr2=0x55d2a2f77000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d2a2f74c10 0x55d2a2f77000 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2e70340 msgr2=0x55d2a2f714e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2e70340 0x55d2a2f714e0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0dc036eb0 msgr2=0x7fd0dc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0dc036eb0 0x7fd0dc039370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000f60 tx=0x7fd1e80010a0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 msgr2=0x55d2a2f6a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2f6a2a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4055040 tx=0x7fd1e4003020 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 shutdown_connections 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d2a2f74c10 0x55d2a2f77000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d2a2e70340 0x55d2a2f714e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0dc036eb0 0x7fd0dc039370 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 --2- 172.21.15.63:0/3736830656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2a2e6f180 0x55d2a2f6a2a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.757+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 >> 172.21.15.63:0/3736830656 conn(0x55d2a2e6a7e0 msgr2=0x55d2a2de3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.758+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 shutdown_connections 2024-04-23T19:10:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.758+0000 7fd1f67af800 1 -- 172.21.15.63:0/3736830656 wait complete. 2024-04-23T19:10:30.776 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-23T19:10:30.777 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-23T19:10:30.777 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-23T19:10:30.777 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-04-23T19:10:30.863 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:30.863 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:30.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.863+0000 7f4fa6528800 1 Processor -- start 2024-04-23T19:10:30.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.863+0000 7f4fa6528800 1 -- start start 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa6528800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557820c746a0 0x557820c76a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa6528800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557820b227f0 con 0x557820c70b90 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa6528800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x557820b11e20 con 0x557820c746a0 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa19a6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557820c746a0 0x557820c76a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa21a7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x557820c70b90 0x557820c72f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38804/0 (socket says 172.21.15.63:38804) 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa21a7640 1 -- 172.21.15.63:0/2468797047 learned_addr learned my addr 172.21.15.63:0/2468797047 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa19a6640 1 -- 172.21.15.63:0/2468797047 >> v1:172.21.15.16:6789/0 conn(0x557820c70b90 legacy=0x557820c72f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.864+0000 7f4fa19a6640 1 -- 172.21.15.63:0/2468797047 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557820b30b50 con 0x557820c746a0 2024-04-23T19:10:30.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.866+0000 7f4fa19a6640 1 --2- 172.21.15.63:0/2468797047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c746a0 0x557820c76a90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800a470 tx=0x7f4f9801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd98745dd1b510c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.866+0000 7f4fa11a5640 1 -- 172.21.15.63:0/2468797047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98057070 con 0x557820c746a0 2024-04-23T19:10:30.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.866+0000 7f4fa11a5640 1 -- 172.21.15.63:0/2468797047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f98052570 con 0x557820c746a0 2024-04-23T19:10:30.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.866+0000 7f4fa11a5640 1 -- 172.21.15.63:0/2468797047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98052870 con 0x557820c746a0 2024-04-23T19:10:30.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.867+0000 7f4fa6528800 1 -- 172.21.15.63:0/2468797047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c746a0 msgr2=0x557820c76a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.867+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2468797047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c746a0 0x557820c76a90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800a470 tx=0x7f4f9801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:30.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.867+0000 7f4fa6528800 1 -- 172.21.15.63:0/2468797047 shutdown_connections 2024-04-23T19:10:30.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.867+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2468797047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c746a0 0x557820c76a90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.867+0000 7f4fa6528800 1 -- 172.21.15.63:0/2468797047 >> 172.21.15.63:0/2468797047 conn(0x557820c6a600 msgr2=0x557820c6ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.867+0000 7f4fa6528800 1 -- 172.21.15.63:0/2468797047 shutdown_connections 2024-04-23T19:10:30.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.867+0000 7f4fa6528800 1 -- 172.21.15.63:0/2468797047 wait complete. 2024-04-23T19:10:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.879+0000 7f4fa6528800 1 Processor -- start 2024-04-23T19:10:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.880+0000 7f4fa6528800 1 -- start start 2024-04-23T19:10:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.880+0000 7f4fa6528800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820d61380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.880+0000 7f4fa6528800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557820bbb6f0 con 0x557820c70b90 2024-04-23T19:10:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.880+0000 7f4fa21a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820d61380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.880+0000 7f4fa21a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820d61380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54652/0 (socket says 172.21.15.63:54652) 2024-04-23T19:10:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.880+0000 7f4fa21a7640 1 -- 172.21.15.63:0/3433718249 learned_addr learned my addr 172.21.15.63:0/3433718249 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.880+0000 7f4fa21a7640 1 -- 172.21.15.63:0/3433718249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557820d61950 con 0x557820c70b90 2024-04-23T19:10:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.881+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/3433718249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820d61380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4f94001de0 tx=0x7f4f9400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0121e86aadf6ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.881+0000 7f4e90608640 1 -- 172.21.15.63:0/3433718249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f94012020 con 0x557820c70b90 2024-04-23T19:10:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.881+0000 7f4e90608640 1 -- 172.21.15.63:0/3433718249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9400e5a0 con 0x557820c70b90 2024-04-23T19:10:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.881+0000 7f4e90608640 1 -- 172.21.15.63:0/3433718249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9400e8c0 con 0x557820c70b90 2024-04-23T19:10:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.881+0000 7f4fa6528800 1 -- 172.21.15.63:0/3433718249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 msgr2=0x557820d61380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.881+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3433718249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820d61380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4f94001de0 tx=0x7f4f9400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 -- 172.21.15.63:0/3433718249 shutdown_connections 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3433718249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820d61380 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 -- 172.21.15.63:0/3433718249 >> 172.21.15.63:0/3433718249 conn(0x557820c6a600 msgr2=0x557820d5cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 -- 172.21.15.63:0/3433718249 shutdown_connections 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 -- 172.21.15.63:0/3433718249 wait complete. 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 Processor -- start 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 -- start start 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820c704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa6528800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557820c772d0 con 0x557820c70b90 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa21a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820c704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa21a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820c704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54660/0 (socket says 172.21.15.63:54660) 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.882+0000 7f4fa21a7640 1 -- 172.21.15.63:0/2805810029 learned_addr learned my addr 172.21.15.63:0/2805810029 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.883+0000 7f4fa21a7640 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557820c6ebc0 con 0x557820c70b90 2024-04-23T19:10:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.883+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820c704d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f94001de0 tx=0x7f4f94000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.883+0000 7f4e8ee05640 1 -- 172.21.15.63:0/2805810029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f94012030 con 0x557820c70b90 2024-04-23T19:10:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.883+0000 7f4e8ee05640 1 -- 172.21.15.63:0/2805810029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9401a7a0 con 0x557820c70b90 2024-04-23T19:10:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.883+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557820c6ee50 con 0x557820c70b90 2024-04-23T19:10:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.883+0000 7f4e8ee05640 1 -- 172.21.15.63:0/2805810029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9401aac0 con 0x557820c70b90 2024-04-23T19:10:30.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.883+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557820c6f370 con 0x557820c70b90 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4e8ee05640 1 -- 172.21.15.63:0/2805810029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4f94015020 con 0x557820c70b90 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4e8ee05640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e88036e60 0x7f4e88039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4e8ee05640 1 -- 172.21.15.63:0/2805810029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4f94020070 con 0x557820c70b90 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4fa19a6640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e88036e60 0x7f4e88039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820c746a0 0x557820d722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820d72840 con 0x557820c746a0 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820c746a0 0x557820d722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.884+0000 7f4fa19a6640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e88036e60 0x7f4e88039320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f98054000 tx=0x7f4f9804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.885+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820c746a0 0x557820d722b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.885+0000 7f4fa29a8640 1 -- 172.21.15.63:0/2805810029 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f9c00a4a0 con 0x557820c746a0 2024-04-23T19:10:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.885+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820d73120 con 0x557820c746a0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.885+0000 7f4fa29a8640 1 -- 172.21.15.63:0/2805810029 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4f9c00a4a0 con 0x557820c746a0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.885+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820d74510 con 0x557820c746a0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.885+0000 7f4fa29a8640 1 -- 172.21.15.63:0/2805810029 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4f9c00a4a0 con 0x557820c746a0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.885+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820d73970 con 0x557820c746a0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.886+0000 7f4fa29a8640 1 -- 172.21.15.63:0/2805810029 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4f9c00a4a0 con 0x557820c746a0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.886+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557820d759c0 0x557820d77db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.886+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820d78520 con 0x557820d759c0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.886+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557820d759c0 0x557820d77db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.886+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557820d759c0 0x557820d77db0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa21a7640 1 -- 172.21.15.63:0/2805810029 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4f94024980 con 0x557820d759c0 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa6528800 1 Processor -- start 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa6528800 1 -- start start 2024-04-23T19:10:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa6528800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d869a0 0x557820d86dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa6528800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557820c70a10 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d869a0 0x557820d86dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d869a0 0x557820d86dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54664/0 (socket says 172.21.15.63:54664) 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa29a8640 1 -- 172.21.15.63:0/3632174966 learned_addr learned my addr 172.21.15.63:0/3632174966 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.887+0000 7f4fa29a8640 1 -- 172.21.15.63:0/3632174966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557820d87300 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.888+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d869a0 0x557820d86dc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x557820c70210 tx=0x7f4f9c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.888+0000 7f4e87fff640 1 -- 172.21.15.63:0/3632174966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c013440 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.888+0000 7f4e87fff640 1 -- 172.21.15.63:0/3632174966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c0135e0 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.888+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557820d87d70 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.888+0000 7f4e87fff640 1 -- 172.21.15.63:0/3632174966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c0138c0 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.888+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557820d88270 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4e87fff640 1 -- 172.21.15.63:0/3632174966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4f9c011040 con 0x557820d869a0 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4e87fff640 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e7c036e60 0x7f4e7c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4e87fff640 1 -- 172.21.15.63:0/3632174966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4f9c01d070 con 0x557820d869a0 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e7c036e60 0x7f4e7c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4fa6528800 1 Processor -- start 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4fa6528800 1 -- start start 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e7c036e60 0x7f4e7c039320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x557820c6f9b0 tx=0x7f4f9401ae40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4fa6528800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d94130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.889+0000 7f4fa6528800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557820b61e30 con 0x557820d93d10 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4fa29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d94130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4fa29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d94130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54674/0 (socket says 172.21.15.63:54674) 2024-04-23T19:10:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4fa29a8640 1 -- 172.21.15.63:0/3024346926 learned_addr learned my addr 172.21.15.63:0/3024346926 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4fa29a8640 1 -- 172.21.15.63:0/3024346926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557820d94700 con 0x557820d93d10 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/3024346926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d94130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c04cca0 tx=0x7f4f9c0127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ba94ab16054414 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4e85ffb640 1 -- 172.21.15.63:0/3024346926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c042020 con 0x557820d93d10 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4e85ffb640 1 -- 172.21.15.63:0/3024346926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c012cc0 con 0x557820d93d10 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.890+0000 7f4e85ffb640 1 -- 172.21.15.63:0/3024346926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c027040 con 0x557820d93d10 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 -- 172.21.15.63:0/3024346926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 msgr2=0x557820d94130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3024346926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d94130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c04cca0 tx=0x7f4f9c0127a0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 -- 172.21.15.63:0/3024346926 shutdown_connections 2024-04-23T19:10:30.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3024346926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d94130 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 -- 172.21.15.63:0/3024346926 >> 172.21.15.63:0/3024346926 conn(0x557820d8ffe0 msgr2=0x557820d8e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 -- 172.21.15.63:0/3024346926 shutdown_connections 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 -- 172.21.15.63:0/3024346926 wait complete. 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 Processor -- start 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 -- start start 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d9b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.891+0000 7f4fa6528800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557820d94cd0 con 0x557820d93d10 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4fa29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d9b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4fa29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d9b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54676/0 (socket says 172.21.15.63:54676) 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 learned_addr learned my addr 172.21.15.63:0/869614347 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557820d9b610 con 0x557820d93d10 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d9b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c001df0 tx=0x7f4f9c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4e73fff640 1 -- 172.21.15.63:0/869614347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c027040 con 0x557820d93d10 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4e73fff640 1 -- 172.21.15.63:0/869614347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c02ae30 con 0x557820d93d10 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557820d9b900 con 0x557820d93d10 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4e73fff640 1 -- 172.21.15.63:0/869614347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c021c60 con 0x557820d93d10 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.892+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557820d9e550 con 0x557820d93d10 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.893+0000 7f4e73fff640 1 -- 172.21.15.63:0/869614347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4f9c045020 con 0x557820d93d10 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.893+0000 7f4e73fff640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e6c036e60 0x7f4e6c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.893+0000 7f4e73fff640 1 -- 172.21.15.63:0/869614347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f4f9c033070 con 0x557820d93d10 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.893+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e6c036e60 0x7f4e6c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e6c036e60 0x7f4e6c039320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f9401afb0 tx=0x7f4f94043000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820da6df0 0x557820da92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820da9a00 con 0x557820da6df0 2024-04-23T19:10:30.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa19a6640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820da6df0 0x557820da92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa19a6640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820da6df0 0x557820da92b0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa6528800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa6528800 20 realm 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.894+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820da9dd0 con 0x557820da6df0 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820daa6a0 con 0x557820da6df0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820daaa40 con 0x557820da6df0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820dab290 con 0x557820da6df0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.895+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:30.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa6528800 20 searching for the correct realm 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557820dad000 0x557820daf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557820dad000 0x557820daf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557820dad000 0x557820daf3f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa29a8640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f9c0123a0 0x7f4f9c04be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.896+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.897+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f9c0123a0 0x7f4f9c04be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.897+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f9c0123a0 0x7f4f9c04be40 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.897+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.897+0000 7f4fa21a7640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f94017050 0x7f4f94044140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.897+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.897+0000 7f4fa19a6640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f94017050 0x7f4f94044140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.898+0000 7f4fa19a6640 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f94017050 0x7f4f94044140 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.898+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.898+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.898+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.898+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.898+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.898+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.899+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.900+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.900+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.900+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.900+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.900+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.900+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.901+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.901+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.901+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.901+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.901+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.901+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.902+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.902+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.902+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.902+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.902+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.902+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.903+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.904+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.905+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.905+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.905+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.905+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820dad000 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.905+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.905+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f94017050 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x557820da6df0 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x557820daf980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820dabc10 con 0x557820da6df0 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.906+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820dac4b0 con 0x557820da6df0 2024-04-23T19:10:30.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820dac4b0 con 0x7f4f94017050 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 20 zone default found 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 4 Realm: () 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 20 zonegroup default 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.907+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820dafe00 con 0x7f4f94017050 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820db1c90 con 0x557820dad000 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820db2030 con 0x557820dad000 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 20 started sync module instance, tier type = 2024-04-23T19:10:30.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.908+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db10d0 con 0x7f4f9c0123a0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.909+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.909+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93974435662880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db23d0 con 0x7f4f9c0123a0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.909+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db2c60 con 0x557820da6df0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.910+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93974435662880] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.911+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.911+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93974435665136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db3050 con 0x557820da6df0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.911+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db3ab0 con 0x557820dad000 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.911+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93974435665136] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.912+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.912+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93974435669872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db3ab0 con 0x557820dad000 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.912+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db4520 con 0x7f4f9c0123a0 2024-04-23T19:10:30.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.913+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.913+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93974435676176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db4df0 con 0x7f4f9c0123a0 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.913+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db58d0 con 0x557820da6df0 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.913+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93974435669872] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.914+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93974435676176] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.914+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.914+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93974435678528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db3590 con 0x557820da6df0 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.914+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db6170 con 0x7f4f94017050 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.915+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93974435678528] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.915+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.915+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93974435681248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db6120 con 0x7f4f94017050 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.915+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db6b80 con 0x7f4f94017050 2024-04-23T19:10:30.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.916+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93974435681248] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.916+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.916+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93974435683440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db6d60 con 0x7f4f94017050 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.916+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db75b0 con 0x7f4f94017050 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93974435683440] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93974435687056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db73c0 con 0x7f4f94017050 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 20 add_watcher() i=0 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 20 add_watcher() i=1 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 20 add_watcher() i=2 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 20 add_watcher() i=3 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 20 add_watcher() i=4 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 20 add_watcher() i=5 2024-04-23T19:10:30.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.917+0000 7f4fa6528800 20 add_watcher() i=6 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.918+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93974435687056] v30'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.918+0000 7f4fa6528800 20 add_watcher() i=7 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.918+0000 7f4fa6528800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.918+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820dba9c0 con 0x557820da6df0 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.918+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.919+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93974435688112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db7d80 con 0x557820da6df0 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.919+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93974435688112] v30'117 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4e58fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 5 note: GC not initialized 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4dd37fe640 20 reqs_thread_entry: start 2024-04-23T19:10:30.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 20 Filter name: none 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4dd17fa640 20 reqs_thread_entry: start 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.920+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820dedd70 con 0x7f4f94017050 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820defcf0 con 0x7f4f9c0123a0 2024-04-23T19:10:30.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.921+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820df1610 con 0x557820da6df0 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(64 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv4 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:30.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.922+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820df2b40 con 0x7f4f94017050 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.923+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(65 fud.buckets [call] v30'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.923+0000 7f4fa6528800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.923+0000 7f4fa6528800 20 rados->read ofs=0 len=0 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.923+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820df2f40 con 0x7f4f9c0123a0 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.924+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.924+0000 7f4fa6528800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.924+0000 7f4fa6528800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.924+0000 7f4fa6528800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.924+0000 7f4fa6528800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:30.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.924+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820df8680 con 0x7f4f9c0123a0 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v30'21 uv21 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4fa6528800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4fa6528800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4fa6528800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93974435931728 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x557820df5cc0 con 0x7f4f94017050 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== watch-notify(notify (1) cookie 93974435683440 notify 128849018886 ret 0) ==== 380+0+0 (crc 0 0 0) 0x557820df1a00 con 0x7f4f94017050 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4e85ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018886 cookie 93974435683440 notifier 4513 bl.length()=338 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4e85ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4e85ffb640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4e85ffb640 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f4e68003ea0 con 0x7f4f94017050 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.925+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(68 notify.6 [notify cookie 93974435931728 out=8b] v0'0 uv120 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.926+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.926+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== watch-notify(notify_complete (2) cookie 93974435931728 notify 128849018886 ret 0) ==== 42+0+48 (crc 0 0 0) 0x557820df0b30 con 0x7f4f94017050 2024-04-23T19:10:30.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.927+0000 7f4e58fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:30.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.927+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93974435688112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820df2650 con 0x557820da6df0 2024-04-23T19:10:30.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.928+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93974435688112] v30'118 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.928+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93974435662880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db6eb0 con 0x7f4f9c0123a0 2024-04-23T19:10:30.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.929+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93974435662880] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.929+0000 7f4fa6528800 20 remove_watcher() i=0 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.929+0000 7f4fa6528800 2 removed watcher, disabling cache 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.929+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93974435665136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db0c20 con 0x557820da6df0 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.930+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93974435665136] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.930+0000 7f4fa6528800 20 remove_watcher() i=1 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.930+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93974435669872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db14f0 con 0x557820dad000 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.931+0000 7f4fa29a8640 1 -- 172.21.15.63:0/869614347 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93974435669872] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9c029070 con 0x557820dad000 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.931+0000 7f4fa6528800 20 remove_watcher() i=2 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.931+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93974435676176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db14f0 con 0x7f4f9c0123a0 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.932+0000 7f4fa21a7640 1 -- 172.21.15.63:0/869614347 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93974435676176] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f94024980 con 0x7f4f9c0123a0 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.932+0000 7f4fa6528800 20 remove_watcher() i=3 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.932+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93974435678528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db14f0 con 0x557820da6df0 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.932+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93974435678528] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x557820da6df0 2024-04-23T19:10:30.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.933+0000 7f4fa6528800 20 remove_watcher() i=4 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.933+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93974435681248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db14f0 con 0x7f4f94017050 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.933+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93974435681248] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.933+0000 7f4fa6528800 20 remove_watcher() i=5 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.933+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93974435683440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db53e0 con 0x7f4f94017050 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.934+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93974435683440] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.934+0000 7f4fa6528800 20 remove_watcher() i=6 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.934+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93974435687056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x557820db14f0 con 0x7f4f94017050 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93974435687056] v30'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4f9805a040 con 0x7f4f94017050 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 20 remove_watcher() i=7 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f94017050 msgr2=0x7f4f94044140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4f94017050 0x7f4f94044140 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820da6df0 msgr2=0x557820da92b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820da6df0 0x557820da92b0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f9c0123a0 msgr2=0x7f4f9c04be40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4f9c0123a0 0x7f4f9c04be40 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557820dad000 msgr2=0x557820daf3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.935+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557820dad000 0x557820daf3f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e6c036e60 msgr2=0x7f4e6c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e6c036e60 0x7f4e6c039320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f9401afb0 tx=0x7f4f94043000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa19a6640 1 -- 172.21.15.63:0/869614347 reap_dead start 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 msgr2=0x557820d9b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d9b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c001df0 tx=0x7f4f9c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 shutdown_connections 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/869614347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d93d10 0x557820d9b0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 >> 172.21.15.63:0/869614347 conn(0x557820d8ffe0 msgr2=0x557820d8d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 shutdown_connections 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/869614347 wait complete. 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e7c036e60 msgr2=0x7f4e7c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e7c036e60 0x7f4e7c039320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x557820c6f9b0 tx=0x7f4f9401ae40 comp rx=0 tx=0).stop 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d869a0 msgr2=0x557820d86dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d869a0 0x557820d86dc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x557820c70210 tx=0x7f4f9c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 shutdown_connections 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e7c036e60 0x7f4e7c039320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/3632174966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820d869a0 0x557820d86dc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 >> 172.21.15.63:0/3632174966 conn(0x557820d7aa40 msgr2=0x557820d7ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 shutdown_connections 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/3632174966 wait complete. 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557820d759c0 msgr2=0x557820d77db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557820d759c0 0x557820d77db0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820c746a0 msgr2=0x557820d722b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820c746a0 0x557820d722b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.936+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e88036e60 msgr2=0x7f4e88039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e88036e60 0x7f4e88039320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f98054000 tx=0x7f4f9804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 msgr2=0x557820c704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820c704d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f94001de0 tx=0x7f4f94000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 shutdown_connections 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557820d759c0 0x557820d77db0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557820c746a0 0x557820d722b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4e88036e60 0x7f4e88039320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 --2- 172.21.15.63:0/2805810029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557820c70b90 0x557820c704d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 >> 172.21.15.63:0/2805810029 conn(0x557820c6a600 msgr2=0x557820d5b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:30.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 shutdown_connections 2024-04-23T19:10:30.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:30.937+0000 7f4fa6528800 1 -- 172.21.15.63:0/2805810029 wait complete. 2024-04-23T19:10:30.951 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1'] 2024-04-23T19:10:30.951 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1'] 2024-04-23T19:10:30.951 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:31.000 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:31.000 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:31.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.000+0000 7fe614344800 1 Processor -- start 2024-04-23T19:10:31.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.000+0000 7fe614344800 1 -- start start 2024-04-23T19:10:31.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe614344800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55fe278a64e0 0x55fe278a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe614344800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55fe277557f0 con 0x55fe278a2530 2024-04-23T19:10:31.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe614344800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55fe27744e20 con 0x55fe278a64e0 2024-04-23T19:10:31.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe611642640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55fe278a64e0 0x55fe278a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe611e43640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55fe278a2530 0x55fe278a2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38808/0 (socket says 172.21.15.63:38808) 2024-04-23T19:10:31.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe611e43640 1 -- 172.21.15.63:0/4027959285 learned_addr learned my addr 172.21.15.63:0/4027959285 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:31.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe611642640 1 -- 172.21.15.63:0/4027959285 >> v1:172.21.15.16:6789/0 conn(0x55fe278a2530 legacy=0x55fe278a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.001+0000 7fe611642640 1 -- 172.21.15.63:0/4027959285 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe27763b50 con 0x55fe278a64e0 2024-04-23T19:10:31.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.003+0000 7fe611642640 1 --2- 172.21.15.63:0/4027959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a64e0 0x55fe278a88d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe60800a470 tx=0x7fe60801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309741fff27fc924 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.003+0000 7fe610e41640 1 -- 172.21.15.63:0/4027959285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe608057070 con 0x55fe278a64e0 2024-04-23T19:10:31.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.003+0000 7fe610e41640 1 -- 172.21.15.63:0/4027959285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe608052570 con 0x55fe278a64e0 2024-04-23T19:10:31.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.003+0000 7fe610e41640 1 -- 172.21.15.63:0/4027959285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe608052870 con 0x55fe278a64e0 2024-04-23T19:10:31.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.004+0000 7fe614344800 1 -- 172.21.15.63:0/4027959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a64e0 msgr2=0x55fe278a88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.004+0000 7fe614344800 1 --2- 172.21.15.63:0/4027959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a64e0 0x55fe278a88d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe60800a470 tx=0x7fe60801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.004+0000 7fe614344800 1 -- 172.21.15.63:0/4027959285 shutdown_connections 2024-04-23T19:10:31.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.004+0000 7fe614344800 1 --2- 172.21.15.63:0/4027959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a64e0 0x55fe278a88d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.004+0000 7fe614344800 1 -- 172.21.15.63:0/4027959285 >> 172.21.15.63:0/4027959285 conn(0x55fe2789cdd0 msgr2=0x55fe2789f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.004+0000 7fe614344800 1 -- 172.21.15.63:0/4027959285 shutdown_connections 2024-04-23T19:10:31.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.004+0000 7fe614344800 1 -- 172.21.15.63:0/4027959285 wait complete. 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.016+0000 7fe614344800 1 Processor -- start 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.016+0000 7fe614344800 1 -- start start 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.016+0000 7fe614344800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe278a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.016+0000 7fe614344800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe277ee710 con 0x55fe278a2530 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe611e43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe278a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe611e43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe278a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54706/0 (socket says 172.21.15.63:54706) 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe611e43640 1 -- 172.21.15.63:0/41322157 learned_addr learned my addr 172.21.15.63:0/41322157 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe611e43640 1 -- 172.21.15.63:0/41322157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe27994f20 con 0x55fe278a2530 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe611e43640 1 --2- 172.21.15.63:0/41322157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe278a1c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe604001d50 tx=0x7fe60400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2edd3d00c37f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe4ffe07640 1 -- 172.21.15.63:0/41322157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604013030 con 0x55fe278a2530 2024-04-23T19:10:31.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe4ffe07640 1 -- 172.21.15.63:0/41322157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60400e470 con 0x55fe278a2530 2024-04-23T19:10:31.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.017+0000 7fe4ffe07640 1 -- 172.21.15.63:0/41322157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60400e810 con 0x55fe278a2530 2024-04-23T19:10:31.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 -- 172.21.15.63:0/41322157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 msgr2=0x55fe278a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 --2- 172.21.15.63:0/41322157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe278a1c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe604001d50 tx=0x7fe60400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 -- 172.21.15.63:0/41322157 shutdown_connections 2024-04-23T19:10:31.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 --2- 172.21.15.63:0/41322157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe278a1c30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 -- 172.21.15.63:0/41322157 >> 172.21.15.63:0/41322157 conn(0x55fe2789cdd0 msgr2=0x55fe2798ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 -- 172.21.15.63:0/41322157 shutdown_connections 2024-04-23T19:10:31.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 -- 172.21.15.63:0/41322157 wait complete. 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.018+0000 7fe614344800 1 Processor -- start 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe614344800 1 -- start start 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe614344800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe2799e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe614344800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe278a91a0 con 0x55fe278a2530 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe611e43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe2799e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe611e43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe2799e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54716/0 (socket says 172.21.15.63:54716) 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe611e43640 1 -- 172.21.15.63:0/2572095994 learned_addr learned my addr 172.21.15.63:0/2572095994 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe611e43640 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2799e730 con 0x55fe278a2530 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe611e43640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe2799e1f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55fe278a1260 tx=0x7fe604000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe4fe604640 1 -- 172.21.15.63:0/2572095994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604027070 con 0x55fe278a2530 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe4fe604640 1 -- 172.21.15.63:0/2572095994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe604019900 con 0x55fe278a2530 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe4fe604640 1 -- 172.21.15.63:0/2572095994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604019c40 con 0x55fe278a2530 2024-04-23T19:10:31.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.019+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe2799e9c0 con 0x55fe278a2530 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.020+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe279a15d0 con 0x55fe278a2530 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.020+0000 7fe4fe604640 1 -- 172.21.15.63:0/2572095994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe60402a070 con 0x55fe278a2530 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.020+0000 7fe4fe604640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4f8036e60 0x7fe4f8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.020+0000 7fe4fe604640 1 -- 172.21.15.63:0/2572095994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe60401f070 con 0x55fe278a2530 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe611642640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4f8036e60 0x7fe4f8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe278a64e0 0x55fe279a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279a5960 con 0x55fe278a64e0 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe612644640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe278a64e0 0x55fe279a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe611642640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4f8036e60 0x7fe4f8039320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe608054000 tx=0x7fe60804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe612644640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe278a64e0 0x55fe279a5390 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe612644640 1 -- 172.21.15.63:0/2572095994 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe278a64e0 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.021+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279a2960 con 0x55fe278a64e0 2024-04-23T19:10:31.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe612644640 1 -- 172.21.15.63:0/2572095994 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe278a64e0 2024-04-23T19:10:31.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279a7760 con 0x55fe278a64e0 2024-04-23T19:10:31.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe612644640 1 -- 172.21.15.63:0/2572095994 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe278a64e0 2024-04-23T19:10:31.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279a6aa0 con 0x55fe278a64e0 2024-04-23T19:10:31.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe612644640 1 -- 172.21.15.63:0/2572095994 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe278a64e0 2024-04-23T19:10:31.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe279a8b10 0x55fe279aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279ab480 con 0x55fe279a8b10 2024-04-23T19:10:31.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.022+0000 7fe611e43640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe279a8b10 0x55fe279aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.023+0000 7fe611e43640 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe279a8b10 0x55fe279aaf00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.023+0000 7fe611e43640 1 -- 172.21.15.63:0/2572095994 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe604029030 con 0x55fe279a8b10 2024-04-23T19:10:31.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.023+0000 7fe614344800 1 Processor -- start 2024-04-23T19:10:31.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.023+0000 7fe614344800 1 -- start start 2024-04-23T19:10:31.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe614344800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279b9880 0x55fe279b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe614344800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe27794e30 con 0x55fe279b9880 2024-04-23T19:10:31.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe612644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279b9880 0x55fe279b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe612644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279b9880 0x55fe279b9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54724/0 (socket says 172.21.15.63:54724) 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe612644640 1 -- 172.21.15.63:0/3617328285 learned_addr learned my addr 172.21.15.63:0/3617328285 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe612644640 1 -- 172.21.15.63:0/3617328285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe279ba1e0 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe612644640 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279b9880 0x55fe279b9ca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55fe279a5d20 tx=0x7fe60c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe4f77fe640 1 -- 172.21.15.63:0/3617328285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c013440 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe4f77fe640 1 -- 172.21.15.63:0/3617328285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c0135e0 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe279bac50 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe4f77fe640 1 -- 172.21.15.63:0/3617328285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c0138c0 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.024+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe279bb150 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.025+0000 7fe4f77fe640 1 -- 172.21.15.63:0/3617328285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe60c011040 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.025+0000 7fe4f77fe640 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4ec036e60 0x7fe4ec039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.025+0000 7fe4f77fe640 1 -- 172.21.15.63:0/3617328285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe60c01d070 con 0x55fe279b9880 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.025+0000 7fe611e43640 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4ec036e60 0x7fe4ec039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe614344800 1 Processor -- start 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe614344800 1 -- start start 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe611e43640 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4ec036e60 0x7fe4ec039320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55fe279ab860 tx=0x7fe60400ef30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe614344800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe614344800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe279996b0 con 0x55fe279c6c10 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe612644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe612644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54736/0 (socket says 172.21.15.63:54736) 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe612644640 1 -- 172.21.15.63:0/2015525392 learned_addr learned my addr 172.21.15.63:0/2015525392 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe612644640 1 -- 172.21.15.63:0/2015525392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe279c7600 con 0x55fe279c6c10 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe612644640 1 --2- 172.21.15.63:0/2015525392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279c7030 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe60c04cd50 tx=0x7fe60c04d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b438bded21a1a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.026+0000 7fe4f57fa640 1 -- 172.21.15.63:0/2015525392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c015560 con 0x55fe279c6c10 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe4f57fa640 1 -- 172.21.15.63:0/2015525392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c02a350 con 0x55fe279c6c10 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe4f57fa640 1 -- 172.21.15.63:0/2015525392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c012ca0 con 0x55fe279c6c10 2024-04-23T19:10:31.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 -- 172.21.15.63:0/2015525392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 msgr2=0x55fe279c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 --2- 172.21.15.63:0/2015525392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279c7030 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe60c04cd50 tx=0x7fe60c04d000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 -- 172.21.15.63:0/2015525392 shutdown_connections 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 --2- 172.21.15.63:0/2015525392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279c7030 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 -- 172.21.15.63:0/2015525392 >> 172.21.15.63:0/2015525392 conn(0x55fe279c2ee0 msgr2=0x55fe279c0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 -- 172.21.15.63:0/2015525392 shutdown_connections 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 -- 172.21.15.63:0/2015525392 wait complete. 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 Processor -- start 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.027+0000 7fe614344800 1 -- start start 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe614344800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe614344800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe279c7d90 con 0x55fe279c6c10 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe612644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe612644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54746/0 (socket says 172.21.15.63:54746) 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 learned_addr learned my addr 172.21.15.63:0/3609487752 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe279ce600 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe612644640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279ce0c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe60c04d040 tx=0x7fe60c04cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe4e37fe640 1 -- 172.21.15.63:0/3609487752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c02a8e0 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe4e37fe640 1 -- 172.21.15.63:0/3609487752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c013af0 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe279ce890 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe4e37fe640 1 -- 172.21.15.63:0/3609487752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60c013dd0 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.028+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe279d1540 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.029+0000 7fe4e37fe640 1 -- 172.21.15.63:0/3609487752 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe60c02b3a0 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.029+0000 7fe4e37fe640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4dc036e10 0x7fe4dc0392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.029+0000 7fe4e37fe640 1 -- 172.21.15.63:0/3609487752 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe60c032070 con 0x55fe279c6c10 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.029+0000 7fe611e43640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4dc036e10 0x7fe4dc0392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe611e43640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4dc036e10 0x7fe4dc0392d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe604046010 tx=0x7fe604043000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe279d9c20 0x55fe279dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279dc830 con 0x55fe279d9c20 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe611642640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe279d9c20 0x55fe279dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe611642640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe279d9c20 0x55fe279dc0e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe614344800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe614344800 20 realm 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.030+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279dcc00 con 0x55fe279d9c20 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279dd4d0 con 0x55fe279d9c20 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279dd870 con 0x55fe279d9c20 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279de0c0 con 0x55fe279d9c20 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.031+0000 7fe614344800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe614344800 20 searching for the correct realm 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe279dfe30 0x55fe279e2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe612644640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe279dfe30 0x55fe279e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe612644640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe279dfe30 0x55fe279e2220 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe612644640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe60c02c430 0x7fe60c045460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60c02c430 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.032+0000 7fe611e43640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe60c02c430 0x7fe60c045460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.033+0000 7fe611e43640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe60c02c430 0x7fe60c045460 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.033+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.033+0000 7fe611e43640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe60402c060 0x7fe60404c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.033+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.033+0000 7fe611642640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe60402c060 0x7fe60404c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.034+0000 7fe611642640 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe60402c060 0x7fe60404c370 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.034+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.034+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.035+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.035+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.035+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.035+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.035+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.035+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.035+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60c02c430 2024-04-23T19:10:31.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.036+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60c02c430 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.037+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.038+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.038+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.038+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.038+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.038+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.038+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.039+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.039+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.039+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.039+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.039+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.039+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60c02c430 2024-04-23T19:10:31.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.040+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.041+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.041+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.041+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.041+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60c02c430 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.041+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.041+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279dfe30 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60402c060 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x55fe279d9c20 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e27b0 con 0x7fe60c02c430 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe614344800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe614344800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe614344800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe614344800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe614344800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe614344800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.042+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279dea70 con 0x55fe279d9c20 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279df310 con 0x55fe279d9c20 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279df310 con 0x7fe60402c060 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 zone default found 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 4 Realm: () 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 zonegroup default 2024-04-23T19:10:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.043+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e2c30 con 0x7fe60402c060 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279df0e0 con 0x55fe279dfe30 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e4bb0 con 0x55fe279dfe30 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 20 started sync module instance, tier type = 2024-04-23T19:10:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.044+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e3f50 con 0x7fe60c02c430 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.045+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.045+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94550074737472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e54c0 con 0x7fe60c02c430 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.045+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e5d10 con 0x55fe279d9c20 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.046+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94550074737472] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.046+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.047+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94550074740752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e5d70 con 0x55fe279d9c20 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.047+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e6880 con 0x55fe279dfe30 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.047+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94550074740752] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.048+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.048+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94550074745648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e6840 con 0x55fe279dfe30 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.048+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e7330 con 0x7fe60c02c430 2024-04-23T19:10:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.049+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94550074745648] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.049+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.049+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94550074748480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e7720 con 0x7fe60c02c430 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.049+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e8180 con 0x55fe279d9c20 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.050+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.050+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94550074752096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e8a20 con 0x55fe279d9c20 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.050+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e94c0 con 0x7fe60402c060 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.050+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94550074748480] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.051+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.051+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94550074752096] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.051+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94550074757008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e9480 con 0x7fe60402c060 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.051+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e9a20 con 0x7fe60402c060 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.051+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94550074757008] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.052+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.052+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94550074754416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e9a20 con 0x7fe60402c060 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.052+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279ea320 con 0x7fe60402c060 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94550074754416] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94550074742880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279ea320 con 0x7fe60402c060 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 20 add_watcher() i=0 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 20 add_watcher() i=1 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 20 add_watcher() i=2 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 20 add_watcher() i=3 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 20 add_watcher() i=4 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 20 add_watcher() i=5 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.053+0000 7fe614344800 20 add_watcher() i=6 2024-04-23T19:10:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.054+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94550074742880] v30'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.054+0000 7fe614344800 20 add_watcher() i=7 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.054+0000 7fe614344800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.054+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe279ed740 con 0x55fe279d9c20 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.054+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.054+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94550074759920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279eac40 con 0x55fe279d9c20 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94550074759920] v30'119 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe4c37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe614344800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe614344800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe614344800 5 note: GC not initialized 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe4c27fc640 20 reqs_thread_entry: start 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe614344800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.055+0000 7fe614344800 20 Filter name: none 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe433fff640 20 reqs_thread_entry: start 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a20ba0 con 0x55fe279d9c20 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a22b20 con 0x7fe60c02c430 2024-04-23T19:10:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv21 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.056+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a24230 con 0x55fe279d9c20 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv4 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.057+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a28a40 con 0x7fe60402c060 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.058+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(65 fud.buckets [call] v30'3 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.058+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a2e960 con 0x55fe279d9c20 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.059+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v30'5 uv5 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.059+0000 7fe614344800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.059+0000 7fe614344800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.059+0000 7fe614344800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.059+0000 7fe614344800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.059+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94550075001072 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a28e70 con 0x7fe60402c060 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== watch-notify(notify (1) cookie 94550074757008 notify 128849018887 ret 0) ==== 794+0+0 (crc 0 0 0) 0x55fe27a24620 con 0x7fe60402c060 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe4f57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018887 cookie 94550074757008 notifier 4520 bl.length()=752 2024-04-23T19:10:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe4f57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe4f57fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe4f57fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(67 notify.5 [notify cookie 94550075001072 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe4f57fa640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fe4d4003ea0 con 0x7fe60402c060 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== watch-notify(notify_complete (2) cookie 94550075001072 notify 128849018887 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55fe27a23760 con 0x7fe60402c060 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe614344800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_sources= new_sources= 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe614344800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential sources added=[] removed=[] 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe614344800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_dests= new_dests= 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe614344800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential dests added=[] removed=[] 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.060+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a2d160 con 0x7fe60c02c430 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.061+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(69 foo.buckets [call] v30'22 uv22 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.061+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a2e170 con 0x7fe60c02c430 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v30'23 uv23 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe614344800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe614344800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe614344800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94550075001040 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a2d610 con 0x7fe60402c060 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== watch-notify(notify (1) cookie 94550074754416 notify 128849018888 ret 0) ==== 356+0+0 (crc 0 0 0) 0x55fe27763b50 con 0x7fe60402c060 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe4f57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018888 cookie 94550074754416 notifier 4520 bl.length()=314 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe4f57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe4f57fa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(71 notify.6 [notify cookie 94550075001040 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.062+0000 7fe4f57fa640 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7fe4d4003ea0 con 0x7fe60402c060 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.063+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.063+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== watch-notify(notify_complete (2) cookie 94550075001040 notify 128849018888 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fe60808f8e0 con 0x7fe60402c060 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.064+0000 7fe4c37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.064+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94550074759920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe27a248d0 con 0x55fe279d9c20 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.065+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94550074759920] v30'120 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.065+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94550074737472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e9af0 con 0x7fe60c02c430 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.066+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94550074737472] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.066+0000 7fe614344800 20 remove_watcher() i=0 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.066+0000 7fe614344800 2 removed watcher, disabling cache 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.066+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94550074740752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e4340 con 0x55fe279d9c20 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.067+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94550074740752] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.067+0000 7fe614344800 20 remove_watcher() i=1 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.067+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94550074745648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e4340 con 0x55fe279dfe30 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.068+0000 7fe612644640 1 -- 172.21.15.63:0/3609487752 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94550074745648] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60c00a4a0 con 0x55fe279dfe30 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.068+0000 7fe614344800 20 remove_watcher() i=2 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.068+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94550074748480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e4340 con 0x7fe60c02c430 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.069+0000 7fe611e43640 1 -- 172.21.15.63:0/3609487752 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94550074748480] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60403f070 con 0x7fe60c02c430 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.069+0000 7fe614344800 20 remove_watcher() i=3 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.069+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94550074752096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e6e40 con 0x55fe279d9c20 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.069+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94550074752096] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x55fe279d9c20 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.069+0000 7fe614344800 20 remove_watcher() i=4 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.069+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94550074757008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e4340 con 0x7fe60402c060 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.070+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94550074757008] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.070+0000 7fe614344800 20 remove_watcher() i=5 2024-04-23T19:10:31.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.070+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94550074754416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e6e40 con 0x7fe60402c060 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.071+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94550074754416] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.071+0000 7fe614344800 20 remove_watcher() i=6 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.071+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94550074742880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe279e8f90 con 0x7fe60402c060 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94550074742880] v30'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe60805a040 con 0x7fe60402c060 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 20 remove_watcher() i=7 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe60402c060 msgr2=0x7fe60404c370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe60402c060 0x7fe60404c370 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe279d9c20 msgr2=0x55fe279dc0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe279d9c20 0x55fe279dc0e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe60c02c430 msgr2=0x7fe60c045460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe60c02c430 0x7fe60c045460 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe279dfe30 msgr2=0x55fe279e2220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.072+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe279dfe30 0x55fe279e2220 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4dc036e10 msgr2=0x7fe4dc0392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4dc036e10 0x7fe4dc0392d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe604046010 tx=0x7fe604043000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 msgr2=0x55fe279ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/3609487752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279c6c10 0x55fe279ce0c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe60c04d040 tx=0x7fe60c04cce0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 reap_dead start 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe611642640 1 -- 172.21.15.63:0/3609487752 reap_dead start 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 shutdown_connections 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 >> 172.21.15.63:0/3609487752 conn(0x55fe279c2ee0 msgr2=0x55fe279c0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 shutdown_connections 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3609487752 wait complete. 2024-04-23T19:10:31.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4ec036e60 msgr2=0x7fe4ec039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4ec036e60 0x7fe4ec039320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55fe279ab860 tx=0x7fe60400ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279b9880 msgr2=0x55fe279b9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279b9880 0x55fe279b9ca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55fe279a5d20 tx=0x7fe60c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 shutdown_connections 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4ec036e60 0x7fe4ec039320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/3617328285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe279b9880 0x55fe279b9ca0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 >> 172.21.15.63:0/3617328285 conn(0x55fe279ad920 msgr2=0x55fe279add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 shutdown_connections 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/3617328285 wait complete. 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe279a8b10 msgr2=0x55fe279aaf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe279a8b10 0x55fe279aaf00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe278a64e0 msgr2=0x55fe279a5390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.073+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe278a64e0 0x55fe279a5390 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4f8036e60 msgr2=0x7fe4f8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4f8036e60 0x7fe4f8039320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe608054000 tx=0x7fe60804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 msgr2=0x55fe2799e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe2799e1f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55fe278a1260 tx=0x7fe604000bc0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 shutdown_connections 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe279a8b10 0x55fe279aaf00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe278a64e0 0x55fe279a5390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe4f8036e60 0x7fe4f8039320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 --2- 172.21.15.63:0/2572095994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe278a2530 0x55fe2799e1f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 >> 172.21.15.63:0/2572095994 conn(0x55fe2789cdd0 msgr2=0x55fe2798d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 shutdown_connections 2024-04-23T19:10:31.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.074+0000 7fe614344800 1 -- 172.21.15.63:0/2572095994 wait complete. 2024-04-23T19:10:31.089 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-23T19:10:31.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-23T19:10:31.089 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-23T19:10:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.133+0000 7f863cd2d800 1 Processor -- start 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.133+0000 7f863cd2d800 1 -- start start 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.133+0000 7f863cd2d800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55f522255730 0x55f522257b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.133+0000 7f863cd2d800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f5221037f0 con 0x55f522251c20 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.133+0000 7f863cd2d800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55f5220f2e20 con 0x55f522255730 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.133+0000 7f8638177640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55f522255730 0x55f522257b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.133+0000 7f8638978640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55f522251c20 0x55f522254010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38814/0 (socket says 172.21.15.63:38814) 2024-04-23T19:10:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.134+0000 7f8638978640 1 -- 172.21.15.63:0/1012761717 learned_addr learned my addr 172.21.15.63:0/1012761717 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.134+0000 7f8638177640 1 -- 172.21.15.63:0/1012761717 >> v1:172.21.15.16:6789/0 conn(0x55f522251c20 legacy=0x55f522254010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.134+0000 7f8638177640 1 -- 172.21.15.63:0/1012761717 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f522111b50 con 0x55f522255730 2024-04-23T19:10:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.136+0000 7f8638177640 1 --2- 172.21.15.63:0/1012761717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522255730 0x55f522257b20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f862c00a470 tx=0x7f862c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f667824e352243 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.136+0000 7f8637976640 1 -- 172.21.15.63:0/1012761717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f862c057070 con 0x55f522255730 2024-04-23T19:10:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.136+0000 7f8637976640 1 -- 172.21.15.63:0/1012761717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862c052570 con 0x55f522255730 2024-04-23T19:10:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.136+0000 7f8637976640 1 -- 172.21.15.63:0/1012761717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f862c052870 con 0x55f522255730 2024-04-23T19:10:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.136+0000 7f863cd2d800 1 -- 172.21.15.63:0/1012761717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522255730 msgr2=0x55f522257b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.136+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1012761717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522255730 0x55f522257b20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f862c00a470 tx=0x7f862c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.137+0000 7f863cd2d800 1 -- 172.21.15.63:0/1012761717 shutdown_connections 2024-04-23T19:10:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.137+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1012761717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522255730 0x55f522257b20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.137+0000 7f863cd2d800 1 -- 172.21.15.63:0/1012761717 >> 172.21.15.63:0/1012761717 conn(0x55f52224b630 msgr2=0x55f52224da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.137+0000 7f863cd2d800 1 -- 172.21.15.63:0/1012761717 shutdown_connections 2024-04-23T19:10:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.137+0000 7f863cd2d800 1 -- 172.21.15.63:0/1012761717 wait complete. 2024-04-23T19:10:31.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.149+0000 7f863cd2d800 1 Processor -- start 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.149+0000 7f863cd2d800 1 -- start start 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f863cd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f5223423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f863cd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f52219c6f0 con 0x55f522251c20 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8638978640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f5223423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8638978640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f5223423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54770/0 (socket says 172.21.15.63:54770) 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8638978640 1 -- 172.21.15.63:0/1937256922 learned_addr learned my addr 172.21.15.63:0/1937256922 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8638978640 1 -- 172.21.15.63:0/1937256922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f522342980 con 0x55f522251c20 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8638978640 1 --2- 172.21.15.63:0/1937256922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f5223423b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8628001de0 tx=0x7f862800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8345b95ba003b925 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8526e0d640 1 -- 172.21.15.63:0/1937256922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8628012020 con 0x55f522251c20 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8526e0d640 1 -- 172.21.15.63:0/1937256922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862800e5a0 con 0x55f522251c20 2024-04-23T19:10:31.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.150+0000 7f8526e0d640 1 -- 172.21.15.63:0/1937256922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f862800e8c0 con 0x55f522251c20 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 -- 172.21.15.63:0/1937256922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 msgr2=0x55f5223423b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1937256922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f5223423b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8628001de0 tx=0x7f862800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 -- 172.21.15.63:0/1937256922 shutdown_connections 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1937256922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f5223423b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 -- 172.21.15.63:0/1937256922 >> 172.21.15.63:0/1937256922 conn(0x55f52224b630 msgr2=0x55f52233dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 -- 172.21.15.63:0/1937256922 shutdown_connections 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 -- 172.21.15.63:0/1937256922 wait complete. 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.151+0000 7f863cd2d800 1 Processor -- start 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f863cd2d800 1 -- start start 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f863cd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f522251540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f863cd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f52225a370 con 0x55f522251c20 2024-04-23T19:10:31.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f8638978640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f522251540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f8638978640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f522251540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54782/0 (socket says 172.21.15.63:54782) 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f8638978640 1 -- 172.21.15.63:0/3077625608 learned_addr learned my addr 172.21.15.63:0/3077625608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f8638978640 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52224fc30 con 0x55f522251c20 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f8638978640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f522251540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f862800c010 tx=0x7f8628000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f852560a640 1 -- 172.21.15.63:0/3077625608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8628012070 con 0x55f522251c20 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f852560a640 1 -- 172.21.15.63:0/3077625608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862801a7a0 con 0x55f522251c20 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f52224fec0 con 0x55f522251c20 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.152+0000 7f852560a640 1 -- 172.21.15.63:0/3077625608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f862801aa80 con 0x55f522251c20 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.153+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5222503e0 con 0x55f522251c20 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.153+0000 7f852560a640 1 -- 172.21.15.63:0/3077625608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8628015020 con 0x55f522251c20 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.153+0000 7f852560a640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8520036e60 0x7f8520039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.153+0000 7f852560a640 1 -- 172.21.15.63:0/3077625608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f8628020070 con 0x55f522251c20 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f8638177640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8520036e60 0x7f8520039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522255730 0x55f522353470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f5223539b0 con 0x55f522255730 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f8639179640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522255730 0x55f522353470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f8638177640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8520036e60 0x7f8520039320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f862c054000 tx=0x7f862c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f8639179640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522255730 0x55f522353470 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f8639179640 1 -- 172.21.15.63:0/3077625608 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f863000a4a0 con 0x55f522255730 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.154+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f522353e00 con 0x55f522255730 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.155+0000 7f8639179640 1 -- 172.21.15.63:0/3077625608 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f863000a4a0 con 0x55f522255730 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.155+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f522355700 con 0x55f522255730 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.155+0000 7f8639179640 1 -- 172.21.15.63:0/3077625608 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f863000a4a0 con 0x55f522255730 2024-04-23T19:10:31.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.155+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f522354a40 con 0x55f522255730 2024-04-23T19:10:31.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.155+0000 7f8639179640 1 -- 172.21.15.63:0/3077625608 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f863000a4a0 con 0x55f522255730 2024-04-23T19:10:31.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.155+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f522356ab0 0x55f522358ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.155+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f522359420 con 0x55f522356ab0 2024-04-23T19:10:31.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.156+0000 7f8638978640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f522356ab0 0x55f522358ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.156+0000 7f8638978640 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f522356ab0 0x55f522358ea0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.156+0000 7f8638978640 1 -- 172.21.15.63:0/3077625608 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f862802b070 con 0x55f522356ab0 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f863cd2d800 1 Processor -- start 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f863cd2d800 1 -- start start 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f863cd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522367840 0x55f522367c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f863cd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f522142e30 con 0x55f522367840 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f8639179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522367840 0x55f522367c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f8639179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522367840 0x55f522367c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54796/0 (socket says 172.21.15.63:54796) 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f8639179640 1 -- 172.21.15.63:0/1754207432 learned_addr learned my addr 172.21.15.63:0/1754207432 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.157+0000 7f8639179640 1 -- 172.21.15.63:0/1754207432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5223681a0 con 0x55f522367840 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.158+0000 7f8639179640 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522367840 0x55f522367c60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55f522251bf0 tx=0x7f86300178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.158+0000 7f851e7fc640 1 -- 172.21.15.63:0/1754207432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8630013440 con 0x55f522367840 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.158+0000 7f851e7fc640 1 -- 172.21.15.63:0/1754207432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86300135e0 con 0x55f522367840 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.158+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f522368c10 con 0x55f522367840 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.158+0000 7f851e7fc640 1 -- 172.21.15.63:0/1754207432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86300138c0 con 0x55f522367840 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.158+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f522369110 con 0x55f522367840 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.158+0000 7f851e7fc640 1 -- 172.21.15.63:0/1754207432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8630011040 con 0x55f522367840 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f851e7fc640 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8510036e60 0x7f8510039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f851e7fc640 1 -- 172.21.15.63:0/1754207432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f863001d070 con 0x55f522367840 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f8638978640 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8510036e60 0x7f8510039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f863cd2d800 1 Processor -- start 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f863cd2d800 1 -- start start 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f8638978640 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8510036e60 0x7f8510039320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f862804b320 tx=0x7f862804af90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f863cd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f522375010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f863cd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f522342d00 con 0x55f522374bf0 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f8639179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f522375010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f8639179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f522375010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54812/0 (socket says 172.21.15.63:54812) 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.159+0000 7f8639179640 1 -- 172.21.15.63:0/3310610826 learned_addr learned my addr 172.21.15.63:0/3310610826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f8639179640 1 -- 172.21.15.63:0/3310610826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5223755e0 con 0x55f522374bf0 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f8639179640 1 --2- 172.21.15.63:0/3310610826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f522375010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f863004ccb0 tx=0x7f8630012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accf38b6fdeec859 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f850ffff640 1 -- 172.21.15.63:0/3310610826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8630012df0 con 0x55f522374bf0 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f850ffff640 1 -- 172.21.15.63:0/3310610826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8630001310 con 0x55f522374bf0 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f850ffff640 1 -- 172.21.15.63:0/3310610826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8630001630 con 0x55f522374bf0 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f863cd2d800 1 -- 172.21.15.63:0/3310610826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 msgr2=0x55f522375010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3310610826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f522375010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f863004ccb0 tx=0x7f8630012910 comp rx=0 tx=0).stop 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f863cd2d800 1 -- 172.21.15.63:0/3310610826 shutdown_connections 2024-04-23T19:10:31.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3310610826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f522375010 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f863cd2d800 1 -- 172.21.15.63:0/3310610826 >> 172.21.15.63:0/3310610826 conn(0x55f522370ec0 msgr2=0x55f52236ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f863cd2d800 1 -- 172.21.15.63:0/3310610826 shutdown_connections 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.160+0000 7f863cd2d800 1 -- 172.21.15.63:0/3310610826 wait complete. 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f863cd2d800 1 Processor -- start 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f863cd2d800 1 -- start start 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f863cd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f52237c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f863cd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f522375d30 con 0x55f522374bf0 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f8639179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f52237c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f8639179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f52237c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54814/0 (socket says 172.21.15.63:54814) 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 learned_addr learned my addr 172.21.15.63:0/3334736802 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52237c600 con 0x55f522374bf0 2024-04-23T19:10:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.161+0000 7f8639179640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f52237c0c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8630015ac0 tx=0x7f863004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.162+0000 7f850e7fc640 1 -- 172.21.15.63:0/3334736802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8630021c60 con 0x55f522374bf0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.162+0000 7f850e7fc640 1 -- 172.21.15.63:0/3334736802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8630021e40 con 0x55f522374bf0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.162+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f52237c890 con 0x55f522374bf0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.162+0000 7f850e7fc640 1 -- 172.21.15.63:0/3334736802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f863002a350 con 0x55f522374bf0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.162+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52237f540 con 0x55f522374bf0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.162+0000 7f850e7fc640 1 -- 172.21.15.63:0/3334736802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8630040040 con 0x55f522374bf0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f850e7fc640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8504036e60 0x7f8504039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f850e7fc640 1 -- 172.21.15.63:0/3334736802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f8630033030 con 0x55f522374bf0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f8638978640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8504036e60 0x7f8504039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f8638978640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8504036e60 0x7f8504039320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8628022e90 tx=0x7f8628047000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522387c10 0x55f52238a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238a820 con 0x55f522387c10 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.163+0000 7f8638177640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522387c10 0x55f52238a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f8638177640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522387c10 0x55f52238a0d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 20 realm 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238abf0 con 0x55f522387c10 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238b4c0 con 0x55f522387c10 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.164+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238b860 con 0x55f522387c10 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f863cd2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238c0b0 con 0x55f522387c10 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f863cd2d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f863cd2d800 20 searching for the correct realm 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f52238de20 0x55f522390210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x55f52238de20 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f8639179640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f52238de20 0x55f522390210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.165+0000 7f8639179640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f52238de20 0x55f522390210 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8639179640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f86300123a0 0x7f8630022850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x7f86300123a0 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8638978640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f86300123a0 0x7f8630022850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8638978640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f86300123a0 0x7f8630022850 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8638978640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8628017050 0x7f8628022370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.166+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x7f8628017050 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.167+0000 7f8638177640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8628017050 0x7f8628022370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.167+0000 7f8638177640 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8628017050 0x7f8628022370 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.167+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.167+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x55f522387c10 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x55f522387c10 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x7f8628017050 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x7f8628017050 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.168+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x7f86300123a0 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x55f52238de20 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x55f522387c10 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x7f8628017050 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x55f522387c10 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.169+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.170+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55f5223907a0 con 0x7f86300123a0 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.170+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.170+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f52238de20 2024-04-23T19:10:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.170+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.170+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f52238de20 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.170+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.170+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f8628017050 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f52238de20 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f52238de20 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f522387c10 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.171+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f522387c10 2024-04-23T19:10:31.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.172+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.172+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f8628017050 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.172+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.172+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f52238de20 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.172+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.172+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f86300123a0 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.173+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.173+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f522387c10 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.173+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.173+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f8628017050 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.173+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.173+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f52238de20 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f8628017050 2024-04-23T19:10:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f86300123a0 2024-04-23T19:10:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f52238de20 2024-04-23T19:10:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.174+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f8628017050 2024-04-23T19:10:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.175+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.175+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x55f522387c10 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.175+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.175+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f862805a060 con 0x7f86300123a0 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.175+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.175+0000 7f863cd2d800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238ca30 con 0x55f522387c10 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238d2d0 con 0x55f522387c10 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238d2d0 con 0x7f8628017050 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 zone default found 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 4 Realm: () 2024-04-23T19:10:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 zonegroup default 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.176+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f522390c20 con 0x7f8628017050 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52238cfd0 con 0x55f52238de20 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f522392b30 con 0x55f52238de20 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 20 started sync module instance, tier type = 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.177+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522391f40 con 0x7f86300123a0 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.178+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.178+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94511329518432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522391ba0 con 0x7f86300123a0 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.178+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522393a90 con 0x55f522387c10 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.179+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94511329518432] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.180+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.180+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94511329523168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522393a00 con 0x55f522387c10 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.180+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522394950 con 0x55f52238de20 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.181+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94511329523168] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.181+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.181+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94511329526944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522394d40 con 0x55f52238de20 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.181+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522395740 con 0x7f86300123a0 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.182+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94511329526944] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.182+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.182+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94511329525376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522395ae0 con 0x7f86300123a0 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.182+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522396580 con 0x55f522387c10 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.183+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.183+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94511329534064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522396540 con 0x55f522387c10 2024-04-23T19:10:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.183+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522397500 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.183+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94511329525376] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.184+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94511329534064] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.184+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.184+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94511329538064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522397500 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.184+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522397dd0 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.185+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94511329538064] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.185+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.185+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94511329530448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522397500 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.185+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223983a0 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94511329530448] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94511329540304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223983a0 con 0x7f8628017050 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 20 add_watcher() i=0 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 20 add_watcher() i=1 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 20 add_watcher() i=2 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 20 add_watcher() i=3 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 20 add_watcher() i=4 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 20 add_watcher() i=5 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.186+0000 7f863cd2d800 20 add_watcher() i=6 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.187+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94511329540304] v30'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.187+0000 7f863cd2d800 20 add_watcher() i=7 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.187+0000 7f863cd2d800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.187+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f52239b7a0 con 0x55f522387c10 2024-04-23T19:10:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.187+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.188+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94511329542336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522398cc0 con 0x55f522387c10 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.188+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94511329542336] v30'121 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.189+0000 7f84eefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.189+0000 7f863cd2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.189+0000 7f863cd2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.189+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f52239b410 con 0x55f522387c10 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.190+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.190+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a47f0 con 0x7f8628017050 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.191+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.191+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a4ca0 con 0x55f52238de20 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.192+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.192+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a4d30 con 0x55f522387c10 2024-04-23T19:10:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.193+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'122 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.193+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a52e0 con 0x7f86300123a0 2024-04-23T19:10:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.194+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'35 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.194+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5790 con 0x55f52238de20 2024-04-23T19:10:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.195+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'23 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.195+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5790 con 0x55f522387c10 2024-04-23T19:10:31.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.196+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.196+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5790 con 0x7f8628017050 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.197+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'23 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.197+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5790 con 0x7f86300123a0 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.198+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.198+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5960 con 0x55f52238de20 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.199+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'21 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.199+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x7f86300123a0 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.200+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'36 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.200+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x7f8628017050 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.201+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.201+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x7f86300123a0 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.202+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.202+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x7f8628017050 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.203+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'29 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.203+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f522387c10 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.204+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'123 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.204+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f52238de20 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.205+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'30 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.205+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x7f86300123a0 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.206+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'37 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.206+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f52238de20 2024-04-23T19:10:31.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.207+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.207+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f522387c10 2024-04-23T19:10:31.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.208+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.208+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f522387c10 2024-04-23T19:10:31.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.209+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'124 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.209+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f52238de20 2024-04-23T19:10:31.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.211+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'31 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.211+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x7f86300123a0 2024-04-23T19:10:31.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.211+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.211+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f52238de20 2024-04-23T19:10:31.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.212+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.212+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f52238de20 2024-04-23T19:10:31.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.213+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.213+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f52238de20 2024-04-23T19:10:31.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.215+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.215+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x55f52238de20 2024-04-23T19:10:31.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.216+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'24 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.216+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5d50 con 0x7f8628017050 2024-04-23T19:10:31.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.217+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.217+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5f20 con 0x55f52238de20 2024-04-23T19:10:31.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.218+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.218+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5960 con 0x7f8628017050 2024-04-23T19:10:31.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.219+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'24 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.219+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5790 con 0x55f52238de20 2024-04-23T19:10:31.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.220+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'32 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.220+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5790 con 0x7f8628017050 2024-04-23T19:10:31.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.221+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'30 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.221+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223a5790 con 0x7f86300123a0 2024-04-23T19:10:31.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.222+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.222+0000 7f84edfe3640 20 reqs_thread_entry: start 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 20 Filter name: none 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f845f7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d0cd0 con 0x7f8628017050 2024-04-23T19:10:31.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.223+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d35e0 con 0x7f8628017050 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.224+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(95 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 155+0+15 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.224+0000 7f863cd2d800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-23T19:10:31.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.224+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d15c0 con 0x7f8628017050 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v30'4 uv4 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f863cd2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 94511329779920 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3bf0 con 0x55f52238de20 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== watch-notify(notify (1) cookie 94511329526944 notify 128849018883 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55f5223a60f0 con 0x55f52238de20 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f850ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018883 cookie 94511329526944 notifier 4527 bl.length()=170 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(97 notify.2 [notify cookie 94511329779920 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f850ffff640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f84fc003d60 con 0x55f52238de20 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== watch-notify(notify_complete (2) cookie 94511329779920 notify 128849018883 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f52237c890 con 0x55f52238de20 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f863cd2d800 10 removing email index: 2024-04-23T19:10:31.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f863cd2d800 10 removing user buckets index 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.225+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:99 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3b70 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.226+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(99 fud.buckets [delete] v30'4 uv4 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.226+0000 7f863cd2d800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.226+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94511329786736 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d30e0 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== watch-notify(notify (1) cookie 94511329530448 notify 128849018889 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55f5223a5d50 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f850f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018889 cookie 94511329530448 notifier 4527 bl.length()=160 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f850f7fe640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f846c0032c0 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(100 notify.6 [notify cookie 94511329786736 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(101 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== watch-notify(notify_complete (2) cookie 94511329786736 notify 128849018889 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f52239b2a0 con 0x7f8628017050 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f863cd2d800 10 removing user index: fud 2024-04-23T19:10:31.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.227+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d30e0 con 0x7f8628017050 2024-04-23T19:10:31.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(102 fud [call,call,delete] v30'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f863cd2d800 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-23T19:10:31.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f863cd2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-23T19:10:31.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.3 7:c609908c:::notify.5:head [notify cookie 94511329786736 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d20d0 con 0x7f8628017050 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== watch-notify(notify (1) cookie 94511329538064 notify 128849018890 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55f52239b2a0 con 0x7f8628017050 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f850ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018890 cookie 94511329538064 notifier 4527 bl.length()=152 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f850ffff640 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:104 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f84fc003d60 con 0x7f8628017050 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.228+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(103 notify.5 [notify cookie 94511329786736 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.229+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(104 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.229+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== watch-notify(notify_complete (2) cookie 94511329786736 notify 128849018890 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f52239b580 con 0x7f8628017050 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.230+0000 7f84eefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.230+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94511329542336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3b70 con 0x55f522387c10 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.231+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94511329542336] v30'125 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.231+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94511329518432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3b70 con 0x7f86300123a0 2024-04-23T19:10:31.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.232+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94511329518432] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.232+0000 7f863cd2d800 20 remove_watcher() i=0 2024-04-23T19:10:31.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.232+0000 7f863cd2d800 2 removed watcher, disabling cache 2024-04-23T19:10:31.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.232+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94511329523168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3b70 con 0x55f522387c10 2024-04-23T19:10:31.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.233+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94511329523168] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.233+0000 7f863cd2d800 20 remove_watcher() i=1 2024-04-23T19:10:31.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.233+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94511329526944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3b70 con 0x55f52238de20 2024-04-23T19:10:31.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.234+0000 7f8639179640 1 -- 172.21.15.63:0/3334736802 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94511329526944] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8630029070 con 0x55f52238de20 2024-04-23T19:10:31.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.234+0000 7f863cd2d800 20 remove_watcher() i=2 2024-04-23T19:10:31.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.234+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94511329525376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223944a0 con 0x7f86300123a0 2024-04-23T19:10:31.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.235+0000 7f8638978640 1 -- 172.21.15.63:0/3334736802 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94511329525376] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8628047070 con 0x7f86300123a0 2024-04-23T19:10:31.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.235+0000 7f863cd2d800 20 remove_watcher() i=3 2024-04-23T19:10:31.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.235+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94511329534064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3b70 con 0x55f522387c10 2024-04-23T19:10:31.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.236+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94511329534064] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x55f522387c10 2024-04-23T19:10:31.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.236+0000 7f863cd2d800 20 remove_watcher() i=4 2024-04-23T19:10:31.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.236+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94511329538064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f522393e80 con 0x7f8628017050 2024-04-23T19:10:31.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.237+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94511329538064] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.237+0000 7f863cd2d800 20 remove_watcher() i=5 2024-04-23T19:10:31.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.237+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94511329530448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223944a0 con 0x7f8628017050 2024-04-23T19:10:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.238+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94511329530448] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.238+0000 7f863cd2d800 20 remove_watcher() i=6 2024-04-23T19:10:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.238+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94511329540304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55f5223d3b70 con 0x7f8628017050 2024-04-23T19:10:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94511329540304] v30'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f862c05a040 con 0x7f8628017050 2024-04-23T19:10:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 20 remove_watcher() i=7 2024-04-23T19:10:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8628017050 msgr2=0x7f8628022370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8628017050 0x7f8628022370 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522387c10 msgr2=0x55f52238a0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522387c10 0x55f52238a0d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f86300123a0 msgr2=0x7f8630022850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f86300123a0 0x7f8630022850 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f52238de20 msgr2=0x55f522390210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f52238de20 0x55f522390210 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8504036e60 msgr2=0x7f8504039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8504036e60 0x7f8504039320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8628022e90 tx=0x7f8628047000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 msgr2=0x55f52237c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f52237c0c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8630015ac0 tx=0x7f863004cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f8638177640 1 -- 172.21.15.63:0/3334736802 reap_dead start 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 shutdown_connections 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3334736802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522374bf0 0x55f52237c0c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 >> 172.21.15.63:0/3334736802 conn(0x55f522370ec0 msgr2=0x55f52236e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 shutdown_connections 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/3334736802 wait complete. 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8510036e60 msgr2=0x7f8510039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8510036e60 0x7f8510039320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f862804b320 tx=0x7f862804af90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522367840 msgr2=0x55f522367c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.239+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522367840 0x55f522367c60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55f522251bf0 tx=0x7f86300178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 shutdown_connections 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8510036e60 0x7f8510039320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/1754207432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522367840 0x55f522367c60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 >> 172.21.15.63:0/1754207432 conn(0x55f52235b8e0 msgr2=0x55f52235bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 shutdown_connections 2024-04-23T19:10:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/1754207432 wait complete. 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f522356ab0 msgr2=0x55f522358ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f522356ab0 0x55f522358ea0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522255730 msgr2=0x55f522353470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522255730 0x55f522353470 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8520036e60 msgr2=0x7f8520039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8520036e60 0x7f8520039320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f862c054000 tx=0x7f862c04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 msgr2=0x55f522251540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f522251540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f862800c010 tx=0x7f8628000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 shutdown_connections 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f522356ab0 0x55f522358ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f522255730 0x55f522353470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8520036e60 0x7f8520039320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 --2- 172.21.15.63:0/3077625608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f522251c20 0x55f522251540 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 >> 172.21.15.63:0/3077625608 conn(0x55f52224b630 msgr2=0x55f52233c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 shutdown_connections 2024-04-23T19:10:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.240+0000 7f863cd2d800 1 -- 172.21.15.63:0/3077625608 wait complete. 2024-04-23T19:10:31.249 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-23T19:10:31.249 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-23T19:10:31.249 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-23T19:10:31.304 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:31.304 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3107616800 1 Processor -- start 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3107616800 1 -- start start 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3107616800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563acf663500 0x563acf66c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3107616800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563acf5157f0 con 0x563acf6619c0 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3107616800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x563acf504e20 con 0x563acf663500 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3102483640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563acf663500 0x563acf66c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3102c84640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x563acf6619c0 0x563acf661de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38824/0 (socket says 172.21.15.63:38824) 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3102c84640 1 -- 172.21.15.63:0/2861281596 learned_addr learned my addr 172.21.15.63:0/2861281596 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3102483640 1 -- 172.21.15.63:0/2861281596 >> v1:172.21.15.16:6789/0 conn(0x563acf6619c0 legacy=0x563acf661de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.305+0000 7f3102483640 1 -- 172.21.15.63:0/2861281596 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acf523b50 con 0x563acf663500 2024-04-23T19:10:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.307+0000 7f3102483640 1 --2- 172.21.15.63:0/2861281596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf663500 0x563acf66c000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f30f800a470 tx=0x7f30f801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecfcaf3fced50da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.307+0000 7f3101c82640 1 -- 172.21.15.63:0/2861281596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f8057070 con 0x563acf663500 2024-04-23T19:10:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3101c82640 1 -- 172.21.15.63:0/2861281596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f804adc0 con 0x563acf663500 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3101c82640 1 -- 172.21.15.63:0/2861281596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f8052590 con 0x563acf663500 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3107616800 1 -- 172.21.15.63:0/2861281596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf663500 msgr2=0x563acf66c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3107616800 1 --2- 172.21.15.63:0/2861281596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf663500 0x563acf66c000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f30f800a470 tx=0x7f30f801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3107616800 1 -- 172.21.15.63:0/2861281596 shutdown_connections 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3107616800 1 --2- 172.21.15.63:0/2861281596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf663500 0x563acf66c000 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3107616800 1 -- 172.21.15.63:0/2861281596 >> 172.21.15.63:0/2861281596 conn(0x563acf65d5a0 msgr2=0x563acf65f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3107616800 1 -- 172.21.15.63:0/2861281596 shutdown_connections 2024-04-23T19:10:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.308+0000 7f3107616800 1 -- 172.21.15.63:0/2861281596 wait complete. 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3107616800 1 Processor -- start 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3107616800 1 -- start start 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3107616800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf7568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3107616800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563acf5ae690 con 0x563acf6619c0 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3102c84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf7568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3102c84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf7568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54830/0 (socket says 172.21.15.63:54830) 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3102c84640 1 -- 172.21.15.63:0/4133133696 learned_addr learned my addr 172.21.15.63:0/4133133696 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.321+0000 7f3102c84640 1 -- 172.21.15.63:0/4133133696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acf756eb0 con 0x563acf6619c0 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f3102c84640 1 --2- 172.21.15.63:0/4133133696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf7568e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f30f4001de0 tx=0x7f30f400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc16b949ff66421b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f2ff0e09640 1 -- 172.21.15.63:0/4133133696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f4012020 con 0x563acf6619c0 2024-04-23T19:10:31.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f2ff0e09640 1 -- 172.21.15.63:0/4133133696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f400e5a0 con 0x563acf6619c0 2024-04-23T19:10:31.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f2ff0e09640 1 -- 172.21.15.63:0/4133133696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f400e8c0 con 0x563acf6619c0 2024-04-23T19:10:31.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f3107616800 1 -- 172.21.15.63:0/4133133696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 msgr2=0x563acf7568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f3107616800 1 --2- 172.21.15.63:0/4133133696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf7568e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f30f4001de0 tx=0x7f30f400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f3107616800 1 -- 172.21.15.63:0/4133133696 shutdown_connections 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f3107616800 1 --2- 172.21.15.63:0/4133133696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf7568e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.322+0000 7f3107616800 1 -- 172.21.15.63:0/4133133696 >> 172.21.15.63:0/4133133696 conn(0x563acf65d5a0 msgr2=0x563acf752200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3107616800 1 -- 172.21.15.63:0/4133133696 shutdown_connections 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3107616800 1 -- 172.21.15.63:0/4133133696 wait complete. 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3107616800 1 Processor -- start 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3107616800 1 -- start start 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3107616800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf75e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3107616800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563acf66e820 con 0x563acf6619c0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3102c84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf75e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3102c84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf75e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54836/0 (socket says 172.21.15.63:54836) 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3102c84640 1 -- 172.21.15.63:0/1518590241 learned_addr learned my addr 172.21.15.63:0/1518590241 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.323+0000 7f3102c84640 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acf75e970 con 0x563acf6619c0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.324+0000 7f3102c84640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf75e430 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f30f4001de0 tx=0x7f30f4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.324+0000 7f2fef606640 1 -- 172.21.15.63:0/1518590241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f4012030 con 0x563acf6619c0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.324+0000 7f2fef606640 1 -- 172.21.15.63:0/1518590241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f401a7a0 con 0x563acf6619c0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.324+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563acf75ec00 con 0x563acf6619c0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.324+0000 7f2fef606640 1 -- 172.21.15.63:0/1518590241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f401aac0 con 0x563acf6619c0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.324+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acf7618b0 con 0x563acf6619c0 2024-04-23T19:10:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f2fef606640 1 -- 172.21.15.63:0/1518590241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f30f4015020 con 0x563acf6619c0 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f2fef606640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fe8036eb0 0x7f2fe8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f2fef606640 1 -- 172.21.15.63:0/1518590241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f30f4020070 con 0x563acf6619c0 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f3102483640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fe8036eb0 0x7f2fe8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf663500 0x563acf765670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf765bb0 con 0x563acf663500 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f3103485640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf663500 0x563acf765670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f3102483640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fe8036eb0 0x7f2fe8039370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30f8054000 tx=0x7f30f804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.325+0000 7f3103485640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf663500 0x563acf765670 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.326+0000 7f3103485640 1 -- 172.21.15.63:0/1518590241 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f30fc00a4a0 con 0x563acf663500 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.326+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf762c40 con 0x563acf663500 2024-04-23T19:10:31.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.326+0000 7f3103485640 1 -- 172.21.15.63:0/1518590241 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f30fc00a4a0 con 0x563acf663500 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.326+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7679e0 con 0x563acf663500 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.326+0000 7f3103485640 1 -- 172.21.15.63:0/1518590241 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f30fc00a4a0 con 0x563acf663500 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.326+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf766d20 con 0x563acf663500 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.327+0000 7f3103485640 1 -- 172.21.15.63:0/1518590241 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f30fc00a4a0 con 0x563acf663500 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.327+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563acf768d90 0x563acf76b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.327+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf76b700 con 0x563acf768d90 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.327+0000 7f3102c84640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563acf768d90 0x563acf76b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.327+0000 7f3102c84640 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563acf768d90 0x563acf76b180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3102c84640 1 -- 172.21.15.63:0/1518590241 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f30f4024980 con 0x563acf768d90 2024-04-23T19:10:31.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3107616800 1 Processor -- start 2024-04-23T19:10:31.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3107616800 1 -- start start 2024-04-23T19:10:31.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3107616800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf779b20 0x563acf779f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3107616800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563acf554e30 con 0x563acf779b20 2024-04-23T19:10:31.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3103485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf779b20 0x563acf779f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3103485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf779b20 0x563acf779f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54838/0 (socket says 172.21.15.63:54838) 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.328+0000 7f3103485640 1 -- 172.21.15.63:0/1668562457 learned_addr learned my addr 172.21.15.63:0/1668562457 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.329+0000 7f3103485640 1 -- 172.21.15.63:0/1668562457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acf77a480 con 0x563acf779b20 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.329+0000 7f3103485640 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf779b20 0x563acf779f40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x563acf75f830 tx=0x7f30fc0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.329+0000 7f2fece01640 1 -- 172.21.15.63:0/1668562457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc013440 con 0x563acf779b20 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.329+0000 7f2fece01640 1 -- 172.21.15.63:0/1668562457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30fc0135e0 con 0x563acf779b20 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.329+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563acf77aef0 con 0x563acf779b20 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.329+0000 7f2fece01640 1 -- 172.21.15.63:0/1668562457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc0138c0 con 0x563acf779b20 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.329+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acf77b3f0 con 0x563acf779b20 2024-04-23T19:10:31.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.330+0000 7f2fece01640 1 -- 172.21.15.63:0/1668562457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f30fc011040 con 0x563acf779b20 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.330+0000 7f2fece01640 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fdc036e60 0x7f2fdc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.330+0000 7f2fece01640 1 -- 172.21.15.63:0/1668562457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f30fc01d070 con 0x563acf779b20 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.330+0000 7f3102c84640 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fdc036e60 0x7f2fdc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.330+0000 7f3107616800 1 Processor -- start 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.330+0000 7f3107616800 1 -- start start 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.330+0000 7f3102c84640 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fdc036e60 0x7f2fdc039320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f30f401a6f0 tx=0x7f30f4024e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f3107616800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf787260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f3107616800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563acf757230 con 0x563acf786e40 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f3103485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf787260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f3103485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf787260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54842/0 (socket says 172.21.15.63:54842) 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f3103485640 1 -- 172.21.15.63:0/742056034 learned_addr learned my addr 172.21.15.63:0/742056034 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f3103485640 1 -- 172.21.15.63:0/742056034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acf787830 con 0x563acf786e40 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f3103485640 1 --2- 172.21.15.63:0/742056034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf787260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f30fc04ccb0 tx=0x7f30fc012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfceb328d77d79a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f2fe67fc640 1 -- 172.21.15.63:0/742056034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc012df0 con 0x563acf786e40 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f2fe67fc640 1 -- 172.21.15.63:0/742056034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30fc001310 con 0x563acf786e40 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.331+0000 7f2fe67fc640 1 -- 172.21.15.63:0/742056034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc001630 con 0x563acf786e40 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 -- 172.21.15.63:0/742056034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 msgr2=0x563acf787260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 --2- 172.21.15.63:0/742056034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf787260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f30fc04ccb0 tx=0x7f30fc012910 comp rx=0 tx=0).stop 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 -- 172.21.15.63:0/742056034 shutdown_connections 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 --2- 172.21.15.63:0/742056034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf787260 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 -- 172.21.15.63:0/742056034 >> 172.21.15.63:0/742056034 conn(0x563acf7831a0 msgr2=0x563acf781200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 -- 172.21.15.63:0/742056034 shutdown_connections 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 -- 172.21.15.63:0/742056034 wait complete. 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 Processor -- start 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 -- start start 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf78e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3107616800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563acf787fa0 con 0x563acf786e40 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3103485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf78e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3103485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf78e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54848/0 (socket says 172.21.15.63:54848) 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.332+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 learned_addr learned my addr 172.21.15.63:0/3987642281 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.333+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acf78e960 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.333+0000 7f3103485640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf78e420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f30fc015ac0 tx=0x7f30fc04cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.333+0000 7f2fe4ff9640 1 -- 172.21.15.63:0/3987642281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc021c60 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.333+0000 7f2fe4ff9640 1 -- 172.21.15.63:0/3987642281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30fc021e40 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.333+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563acf78ebf0 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.333+0000 7f2fe4ff9640 1 -- 172.21.15.63:0/3987642281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc02a350 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.333+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acf7918a0 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.334+0000 7f2fe4ff9640 1 -- 172.21.15.63:0/3987642281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f30fc040040 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.334+0000 7f2fe4ff9640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fcc036e60 0x7f2fcc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.334+0000 7f2fe4ff9640 1 -- 172.21.15.63:0/3987642281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f30fc033030 con 0x563acf786e40 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.334+0000 7f3102c84640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fcc036e60 0x7f2fcc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.334+0000 7f3102c84640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fcc036e60 0x7f2fcc039320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30f4024fd0 tx=0x7f30f4049000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.334+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf799f00 0x563acf79c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79cb10 con 0x563acf799f00 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3102483640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf799f00 0x563acf79c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3102483640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf799f00 0x563acf79c3c0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 20 realm 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79cf40 con 0x563acf799f00 2024-04-23T19:10:31.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.335+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79d810 con 0x563acf799f00 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79dbb0 con 0x563acf799f00 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf799f00 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 20 searching for the correct realm 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563acf7a0170 0x563acf7a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a2b90 con 0x563acf7a0170 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.336+0000 7f3103485640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563acf7a0170 0x563acf7a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.337+0000 7f3103485640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563acf7a0170 0x563acf7a2560 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.337+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.337+0000 7f3103485640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f30fc0123a0 0x7f30fc022830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.337+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a2b90 con 0x7f30fc0123a0 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.337+0000 7f3102c84640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f30fc0123a0 0x7f30fc022830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.338+0000 7f3102c84640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f30fc0123a0 0x7f30fc022830 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.338+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.338+0000 7f3102c84640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f30f4017050 0x7f30f4017430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.338+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a2b90 con 0x7f30f4017050 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.338+0000 7f3102483640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f30f4017050 0x7f30f4017430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.338+0000 7f3102483640 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f30f4017050 0x7f30f4017430 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.339+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.339+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf799f00 2024-04-23T19:10:31.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.339+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.339+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf799f00 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.339+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.339+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x7f30f4017050 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.339+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x7f30f4017050 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x7f30fc0123a0 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf7a0170 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf799f00 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.340+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.341+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x7f30f4017050 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.341+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.341+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf799f00 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.341+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.341+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x7f30fc0123a0 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.341+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.341+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf7a0170 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x563acf79e400 con 0x563acf7a0170 2024-04-23T19:10:31.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30f4017050 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf7a0170 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.342+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf7a0170 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf799f00 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf799f00 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30f4017050 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.343+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf7a0170 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.344+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.344+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30fc0123a0 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.344+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.344+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf799f00 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.344+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.344+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30f4017050 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf7a0170 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30f4017050 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30fc0123a0 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.345+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf7a0170 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30f4017050 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x563acf799f00 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7f30fc066040 con 0x7f30fc0123a0 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.346+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79ec50 con 0x563acf799f00 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79f040 con 0x563acf799f00 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf79f040 con 0x7f30f4017050 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 zone default found 2024-04-23T19:10:31.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 4 Realm: () 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 zonegroup default 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.347+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a3010 con 0x7f30f4017050 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a49f0 con 0x563acf7a0170 2024-04-23T19:10:31.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a4d90 con 0x563acf7a0170 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 20 started sync module instance, tier type = 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.348+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a3e90 con 0x7f30fc0123a0 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.349+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.349+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94810588985824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a5130 con 0x7f30fc0123a0 2024-04-23T19:10:31.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.349+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a5980 con 0x563acf799f00 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.350+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94810588985824] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.350+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.350+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94810588992720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a58f0 con 0x563acf799f00 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.350+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a6870 con 0x563acf7a0170 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.351+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94810588992720] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.352+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.352+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94810588994976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a6870 con 0x563acf7a0170 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.352+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a7270 con 0x7f30fc0123a0 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.353+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94810588994976] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.353+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.353+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94810588996544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a67e0 con 0x7f30fc0123a0 2024-04-23T19:10:31.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.353+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a8130 con 0x563acf799f00 2024-04-23T19:10:31.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.354+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.354+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94810589003360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a89d0 con 0x563acf799f00 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.354+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a94c0 con 0x7f30f4017050 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.354+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94810588996544] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.355+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.355+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94810589007760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a9480 con 0x7f30f4017050 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.355+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94810589003360] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.355+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a98b0 con 0x7f30f4017050 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.356+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94810589007760] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.356+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.356+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94810589005088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a9850 con 0x7f30f4017050 2024-04-23T19:10:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.356+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7aa370 con 0x7f30f4017050 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(58 notify.7 [create] v30'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94810589005088] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94810589011520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7aa7e0 con 0x7f30f4017050 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 20 add_watcher() i=0 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 20 add_watcher() i=1 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 20 add_watcher() i=2 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 20 add_watcher() i=3 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 20 add_watcher() i=4 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 20 add_watcher() i=5 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.357+0000 7f3107616800 20 add_watcher() i=6 2024-04-23T19:10:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.358+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94810589011520] v30'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.358+0000 7f3107616800 20 add_watcher() i=7 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.358+0000 7f3107616800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.358+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7aa7e0 con 0x563acf799f00 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.358+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.358+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94810589012784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7aa780 con 0x563acf799f00 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.359+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94810589012784] v30'126 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.359+0000 7f2fb97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.359+0000 7f3107616800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.359+0000 7f3107616800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f3107616800 5 note: GC not initialized 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f2f33fff640 20 reqs_thread_entry: start 2024-04-23T19:10:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f3107616800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f3107616800 20 Filter name: none 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f2f31ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f3107616800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.360+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7e0df0 con 0x7f30f4017050 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 246+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-23T19:10:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7df4e0 con 0x7f30f4017050 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.361+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~419 in=419b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7e3830 con 0x7f30f4017050 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.362+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~419] v30'5 uv5 ondisk = 0) ==== 288+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.362+0000 7f3107616800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-23T19:10:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.362+0000 7f3107616800 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.362+0000 7f3107616800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.362+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 94810589241872 in=622b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7e43a0 con 0x7f30fc0123a0 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== watch-notify(notify (1) cookie 94810588996544 notify 128849018898 ret 0) ==== 652+0+0 (crc 0 0 0) 0x7f30f4046040 con 0x7f30fc0123a0 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f2fe67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018898 cookie 94810588996544 notifier 4534 bl.length()=610 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f2fe67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f2fe67fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(65 notify.3 [notify cookie 94810589241872 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f2fe67fc640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f2fc8003ea0 con 0x7f30fc0123a0 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== watch-notify(notify_complete (2) cookie 94810589241872 notify 128849018898 ret 0) ==== 42+0+48 (crc 0 0 0) 0x563acf7618b0 con 0x7f30fc0123a0 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.363+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7e1200 con 0x7f30f4017050 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v30'6 uv6 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f3107616800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f3107616800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f3107616800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94810589249104 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x563acf7e3220 con 0x563acf7a0170 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== watch-notify(notify (1) cookie 94810588994976 notify 128849018884 ret 0) ==== 234+0+0 (crc 0 0 0) 0x7f30fc053030 con 0x563acf7a0170 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f2fe5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018884 cookie 94810588994976 notifier 4534 bl.length()=192 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f2fe5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f2fe5ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:10:31.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.364+0000 7f2fe5ffb640 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f2f38003400 con 0x563acf7a0170 2024-04-23T19:10:31.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.365+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 notify.2 [notify cookie 94810589249104 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f30fc029070 con 0x563acf7a0170 2024-04-23T19:10:31.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.365+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== watch-notify(notify_complete (2) cookie 94810589249104 notify 128849018884 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f30fc06d020 con 0x563acf7a0170 2024-04-23T19:10:31.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.365+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30fc06f040 con 0x563acf7a0170 2024-04-23T19:10:31.387 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"testx$tenanteduser","display_name":"tenanted-user","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"testx$tenanteduser","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-23T19:10:31.362003Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:31.361989Z","tags":[],"group_ids":[]} 2024-04-23T19:10:31.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.366+0000 7f2fb97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.366+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94810589012784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7e00c0 con 0x563acf799f00 2024-04-23T19:10:31.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.367+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94810589012784] v30'127 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.367+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94810588985824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7aa330 con 0x7f30fc0123a0 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.368+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94810588985824] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.368+0000 7f3107616800 20 remove_watcher() i=0 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.368+0000 7f3107616800 2 removed watcher, disabling cache 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.368+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94810588992720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7aa330 con 0x563acf799f00 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.369+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94810588992720] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.369+0000 7f3107616800 20 remove_watcher() i=1 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.369+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94810588994976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a54d0 con 0x563acf7a0170 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.370+0000 7f3103485640 1 -- 172.21.15.63:0/3987642281 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94810588994976] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30fc06f040 con 0x563acf7a0170 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.370+0000 7f3107616800 20 remove_watcher() i=2 2024-04-23T19:10:31.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.370+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94810588996544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7aa330 con 0x7f30fc0123a0 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.371+0000 7f3102c84640 1 -- 172.21.15.63:0/3987642281 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94810588996544] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f4024980 con 0x7f30fc0123a0 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.371+0000 7f3107616800 20 remove_watcher() i=3 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.371+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94810589003360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a54d0 con 0x563acf799f00 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.372+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94810589003360] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x563acf799f00 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.372+0000 7f3107616800 20 remove_watcher() i=4 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.372+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94810589007760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a63c0 con 0x7f30f4017050 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.373+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94810589007760] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.373+0000 7f3107616800 20 remove_watcher() i=5 2024-04-23T19:10:31.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.373+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94810589005088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a7e60 con 0x7f30f4017050 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.374+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94810589005088] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.374+0000 7f3107616800 20 remove_watcher() i=6 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.374+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94810589011520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x563acf7a54d0 con 0x7f30f4017050 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94810589011520] v30'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f30f805a040 con 0x7f30f4017050 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 20 remove_watcher() i=7 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f30f4017050 msgr2=0x7f30f4017430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f30f4017050 0x7f30f4017430 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf799f00 msgr2=0x563acf79c3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf799f00 0x563acf79c3c0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f30fc0123a0 msgr2=0x7f30fc022830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f30fc0123a0 0x7f30fc022830 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563acf7a0170 msgr2=0x563acf7a2560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563acf7a0170 0x563acf7a2560 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fcc036e60 msgr2=0x7f2fcc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fcc036e60 0x7f2fcc039320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30f4024fd0 tx=0x7f30f4049000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3102483640 1 -- 172.21.15.63:0/3987642281 reap_dead start 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 msgr2=0x563acf78e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf78e420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f30fc015ac0 tx=0x7f30fc04cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 shutdown_connections 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/3987642281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf786e40 0x563acf78e420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 >> 172.21.15.63:0/3987642281 conn(0x563acf7831a0 msgr2=0x563acf780800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 shutdown_connections 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/3987642281 wait complete. 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fdc036e60 msgr2=0x7f2fdc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fdc036e60 0x7f2fdc039320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f30f401a6f0 tx=0x7f30f4024e80 comp rx=0 tx=0).stop 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf779b20 msgr2=0x563acf779f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf779b20 0x563acf779f40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x563acf75f830 tx=0x7f30fc0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 shutdown_connections 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fdc036e60 0x7f2fdc039320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 --2- 172.21.15.63:0/1668562457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf779b20 0x563acf779f40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 >> 172.21.15.63:0/1668562457 conn(0x563acf76dbc0 msgr2=0x563acf76dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.375+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 shutdown_connections 2024-04-23T19:10:31.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1668562457 wait complete. 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563acf768d90 msgr2=0x563acf76b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563acf768d90 0x563acf76b180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf663500 msgr2=0x563acf765670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf663500 0x563acf765670 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fe8036eb0 msgr2=0x7f2fe8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fe8036eb0 0x7f2fe8039370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30f8054000 tx=0x7f30f804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 msgr2=0x563acf75e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf75e430 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f30f4001de0 tx=0x7f30f4000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 shutdown_connections 2024-04-23T19:10:31.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563acf768d90 0x563acf76b180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563acf663500 0x563acf765670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2fe8036eb0 0x7f2fe8039370 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 --2- 172.21.15.63:0/1518590241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563acf6619c0 0x563acf75e430 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 >> 172.21.15.63:0/1518590241 conn(0x563acf65d5a0 msgr2=0x563acf750a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 shutdown_connections 2024-04-23T19:10:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.376+0000 7f3107616800 1 -- 172.21.15.63:0/1518590241 wait complete. 2024-04-23T19:10:31.401 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-23T19:10:31.362003Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:31.361989Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:31.401 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-23T19:10:31.402 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-23T19:10:31.402 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-04-23T19:10:31.450 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:31.450 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f4519800 1 Processor -- start 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f4519800 1 -- start start 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f4519800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55c86172b6c0 0x55c86172dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f4519800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c8615d97f0 con 0x55c861727bb0 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f4519800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55c8615c8e20 con 0x55c86172b6c0 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f181e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55c86172b6c0 0x55c86172dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f201f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55c861727bb0 0x55c861729fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38834/0 (socket says 172.21.15.63:38834) 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f201f640 1 -- 172.21.15.63:0/4228242808 learned_addr learned my addr 172.21.15.63:0/4228242808 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f181e640 1 -- 172.21.15.63:0/4228242808 >> v1:172.21.15.16:6789/0 conn(0x55c861727bb0 legacy=0x55c861729fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.451+0000 7f62f181e640 1 -- 172.21.15.63:0/4228242808 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8615e7b50 con 0x55c86172b6c0 2024-04-23T19:10:31.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.453+0000 7f62f181e640 1 --2- 172.21.15.63:0/4228242808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c86172dab0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f62e800a470 tx=0x7f62e801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4269cfd357f056b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.453+0000 7f62f101d640 1 -- 172.21.15.63:0/4228242808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8057070 con 0x55c86172b6c0 2024-04-23T19:10:31.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.453+0000 7f62f101d640 1 -- 172.21.15.63:0/4228242808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8052570 con 0x55c86172b6c0 2024-04-23T19:10:31.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.453+0000 7f62f101d640 1 -- 172.21.15.63:0/4228242808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8052870 con 0x55c86172b6c0 2024-04-23T19:10:31.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.454+0000 7f62f4519800 1 -- 172.21.15.63:0/4228242808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 msgr2=0x55c86172dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.454+0000 7f62f4519800 1 --2- 172.21.15.63:0/4228242808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c86172dab0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f62e800a470 tx=0x7f62e801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.454+0000 7f62f4519800 1 -- 172.21.15.63:0/4228242808 shutdown_connections 2024-04-23T19:10:31.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.454+0000 7f62f4519800 1 --2- 172.21.15.63:0/4228242808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c86172dab0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.454+0000 7f62f4519800 1 -- 172.21.15.63:0/4228242808 >> 172.21.15.63:0/4228242808 conn(0x55c861721620 msgr2=0x55c861723a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.454+0000 7f62f4519800 1 -- 172.21.15.63:0/4228242808 shutdown_connections 2024-04-23T19:10:31.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.454+0000 7f62f4519800 1 -- 172.21.15.63:0/4228242808 wait complete. 2024-04-23T19:10:31.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f4519800 1 Processor -- start 2024-04-23T19:10:31.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f4519800 1 -- start start 2024-04-23T19:10:31.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f4519800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861727bb0 0x55c8618183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f4519800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c861672710 con 0x55c861727bb0 2024-04-23T19:10:31.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f201f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861727bb0 0x55c8618183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f201f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861727bb0 0x55c8618183a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54864/0 (socket says 172.21.15.63:54864) 2024-04-23T19:10:31.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f201f640 1 -- 172.21.15.63:0/3434869506 learned_addr learned my addr 172.21.15.63:0/3434869506 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.467+0000 7f62f201f640 1 -- 172.21.15.63:0/3434869506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c861818970 con 0x55c861727bb0 2024-04-23T19:10:31.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f62f201f640 1 --2- 172.21.15.63:0/3434869506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861727bb0 0x55c8618183a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f62e4001de0 tx=0x7f62e400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e95d3e2a2bde0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f61e0608640 1 -- 172.21.15.63:0/3434869506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e4012020 con 0x55c861727bb0 2024-04-23T19:10:31.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f61e0608640 1 -- 172.21.15.63:0/3434869506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e400e5a0 con 0x55c861727bb0 2024-04-23T19:10:31.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f61e0608640 1 -- 172.21.15.63:0/3434869506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e400e8c0 con 0x55c861727bb0 2024-04-23T19:10:31.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f62f4519800 1 -- 172.21.15.63:0/3434869506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861727bb0 msgr2=0x55c8618183a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f62f4519800 1 --2- 172.21.15.63:0/3434869506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861727bb0 0x55c8618183a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f62e4001de0 tx=0x7f62e400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f62f4519800 1 -- 172.21.15.63:0/3434869506 shutdown_connections 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f62f4519800 1 --2- 172.21.15.63:0/3434869506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861727bb0 0x55c8618183a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.468+0000 7f62f4519800 1 -- 172.21.15.63:0/3434869506 >> 172.21.15.63:0/3434869506 conn(0x55c861721620 msgr2=0x55c861813cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f4519800 1 -- 172.21.15.63:0/3434869506 shutdown_connections 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f4519800 1 -- 172.21.15.63:0/3434869506 wait complete. 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f4519800 1 Processor -- start 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f4519800 1 -- start start 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f4519800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c861820190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f4519800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c86172e2f0 con 0x55c86172b6c0 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f201f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c861820190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f201f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c861820190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54880/0 (socket says 172.21.15.63:54880) 2024-04-23T19:10:31.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f201f640 1 -- 172.21.15.63:0/1576643938 learned_addr learned my addr 172.21.15.63:0/1576643938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.469+0000 7f62f201f640 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8618206d0 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.470+0000 7f62f201f640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c861820190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f62e4001de0 tx=0x7f62e4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.470+0000 7f61dee05640 1 -- 172.21.15.63:0/1576643938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e4012030 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.470+0000 7f61dee05640 1 -- 172.21.15.63:0/1576643938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e401a700 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.470+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c861820960 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.470+0000 7f61dee05640 1 -- 172.21.15.63:0/1576643938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e401aa20 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.470+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c861823610 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f61dee05640 1 -- 172.21.15.63:0/1576643938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f62e4015020 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f61dee05640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61d8036e60 0x7f61d8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f61dee05640 1 -- 172.21.15.63:0/1576643938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f62e4020070 con 0x55c86172b6c0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f62f181e640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61d8036e60 0x7f61d8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c8618253c0 0x55c8618277b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c861827cf0 con 0x55c8618253c0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f62f2820640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c8618253c0 0x55c8618277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f62f181e640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61d8036e60 0x7f61d8039320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62e8054000 tx=0x7f62e804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.471+0000 7f62f2820640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c8618253c0 0x55c8618277b0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.472+0000 7f62f2820640 1 -- 172.21.15.63:0/1576643938 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62ec00a4a0 con 0x55c8618253c0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.472+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618249a0 con 0x55c8618253c0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.472+0000 7f62f2820640 1 -- 172.21.15.63:0/1576643938 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f62ec00a4a0 con 0x55c8618253c0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.472+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c861829ad0 con 0x55c8618253c0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.472+0000 7f62f2820640 1 -- 172.21.15.63:0/1576643938 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f62ec00a4a0 con 0x55c8618253c0 2024-04-23T19:10:31.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.472+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c861828e10 con 0x55c8618253c0 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.473+0000 7f62f2820640 1 -- 172.21.15.63:0/1576643938 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f62ec00a4a0 con 0x55c8618253c0 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.473+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c86182ae80 0x55c86182d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.473+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c86182d7f0 con 0x55c86182ae80 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.473+0000 7f62f201f640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c86182ae80 0x55c86182d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.473+0000 7f62f201f640 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c86182ae80 0x55c86182d270 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f201f640 1 -- 172.21.15.63:0/1576643938 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f62e4024980 con 0x55c86182ae80 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f4519800 1 Processor -- start 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f4519800 1 -- start start 2024-04-23T19:10:31.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f4519800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86183bba0 0x55c86183bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f4519800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c861618e30 con 0x55c86183bba0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f2820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86183bba0 0x55c86183bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f2820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86183bba0 0x55c86183bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54892/0 (socket says 172.21.15.63:54892) 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.474+0000 7f62f2820640 1 -- 172.21.15.63:0/2661580178 learned_addr learned my addr 172.21.15.63:0/2661580178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.475+0000 7f62f2820640 1 -- 172.21.15.63:0/2661580178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86183c500 con 0x55c86183bba0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.475+0000 7f62f2820640 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86183bba0 0x55c86183bfc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55c861828090 tx=0x7f62ec0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.475+0000 7f61d7fff640 1 -- 172.21.15.63:0/2661580178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec013440 con 0x55c86183bba0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.475+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c86183cf70 con 0x55c86183bba0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.475+0000 7f61d7fff640 1 -- 172.21.15.63:0/2661580178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec0135e0 con 0x55c86183bba0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.475+0000 7f61d7fff640 1 -- 172.21.15.63:0/2661580178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec0138c0 con 0x55c86183bba0 2024-04-23T19:10:31.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.475+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86183d470 con 0x55c86183bba0 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.476+0000 7f61d7fff640 1 -- 172.21.15.63:0/2661580178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f62ec011040 con 0x55c86183bba0 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.476+0000 7f61d7fff640 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61cc036e60 0x7f61cc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.476+0000 7f61d7fff640 1 -- 172.21.15.63:0/2661580178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f62ec01d070 con 0x55c86183bba0 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.476+0000 7f62f201f640 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61cc036e60 0x7f61cc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.476+0000 7f62f4519800 1 Processor -- start 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.476+0000 7f62f201f640 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61cc036e60 0x7f61cc039320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f62e40015f0 tx=0x7f62e404b440 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.476+0000 7f62f4519800 1 -- start start 2024-04-23T19:10:31.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f62f4519800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c8618492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f62f4519800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c861818cf0 con 0x55c861848ea0 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f62f2820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c8618492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f62f2820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c8618492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54904/0 (socket says 172.21.15.63:54904) 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f62f2820640 1 -- 172.21.15.63:0/2984037346 learned_addr learned my addr 172.21.15.63:0/2984037346 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f62f2820640 1 -- 172.21.15.63:0/2984037346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c861849890 con 0x55c861848ea0 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f62f2820640 1 --2- 172.21.15.63:0/2984037346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c8618492c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f62ec04cd80 tx=0x7f62ec04ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184c083eee32f9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f61d5ffb640 1 -- 172.21.15.63:0/2984037346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec012cc0 con 0x55c861848ea0 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f61d5ffb640 1 -- 172.21.15.63:0/2984037346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec02ac50 con 0x55c861848ea0 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.477+0000 7f61d5ffb640 1 -- 172.21.15.63:0/2984037346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec0014f0 con 0x55c861848ea0 2024-04-23T19:10:31.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 -- 172.21.15.63:0/2984037346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 msgr2=0x55c8618492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 --2- 172.21.15.63:0/2984037346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c8618492c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f62ec04cd80 tx=0x7f62ec04ce70 comp rx=0 tx=0).stop 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 -- 172.21.15.63:0/2984037346 shutdown_connections 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 --2- 172.21.15.63:0/2984037346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c8618492c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 -- 172.21.15.63:0/2984037346 >> 172.21.15.63:0/2984037346 conn(0x55c861845200 msgr2=0x55c861843260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 -- 172.21.15.63:0/2984037346 shutdown_connections 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 -- 172.21.15.63:0/2984037346 wait complete. 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 Processor -- start 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 -- start start 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c861850430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f4519800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c86184a000 con 0x55c861848ea0 2024-04-23T19:10:31.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f2820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c861850430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f2820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c861850430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54916/0 (socket says 172.21.15.63:54916) 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.478+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 learned_addr learned my addr 172.21.15.63:0/2002017430 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.479+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c861850970 con 0x55c861848ea0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.479+0000 7f62f2820640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c861850430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f62ec048040 tx=0x7f62ec04cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.479+0000 7f61c3fff640 1 -- 172.21.15.63:0/2002017430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec02ae00 con 0x55c861848ea0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.479+0000 7f61c3fff640 1 -- 172.21.15.63:0/2002017430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec0123a0 con 0x55c861848ea0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.479+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c861850c00 con 0x55c861848ea0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.479+0000 7f61c3fff640 1 -- 172.21.15.63:0/2002017430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62ec012680 con 0x55c861848ea0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.479+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8618538b0 con 0x55c861848ea0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.480+0000 7f61c3fff640 1 -- 172.21.15.63:0/2002017430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f62ec041030 con 0x55c861848ea0 2024-04-23T19:10:31.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.480+0000 7f61c3fff640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61bc036e60 0x7f61bc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.480+0000 7f61c3fff640 1 -- 172.21.15.63:0/2002017430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f62ec032070 con 0x55c861848ea0 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.480+0000 7f62f201f640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61bc036e60 0x7f61bc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.480+0000 7f62f201f640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61bc036e60 0x7f61bc039320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62e4024c90 tx=0x7f62e4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c86185c120 0x55c86185e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c86185ed30 con 0x55c86185c120 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f181e640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c86185c120 0x55c86185e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f181e640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c86185c120 0x55c86185e5e0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 20 realm 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c86185f100 con 0x55c86185c120 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.481+0000 7f62f4519800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c86185f9d0 con 0x55c86185c120 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c86185fd70 con 0x55c86185c120 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618605c0 con 0x55c86185c120 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 20 searching for the correct realm 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c861862330 0x55c861864720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.482+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.483+0000 7f62f2820640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c861862330 0x55c861864720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.483+0000 7f62f2820640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c861862330 0x55c861864720 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.483+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.483+0000 7f62f2820640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f62ec04ea10 0x7f62ec02c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.483+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62ec04ea10 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.483+0000 7f62f201f640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f62ec04ea10 0x7f62ec02c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.484+0000 7f62f201f640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f62ec04ea10 0x7f62ec02c460 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.484+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.484+0000 7f62f201f640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f62e4017050 0x7f62e40165c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.484+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.484+0000 7f62f181e640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f62e4017050 0x7f62e40165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f62e4017050 0x7f62e40165c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.485+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.486+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.486+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.486+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62ec04ea10 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.486+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.486+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.486+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.486+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62ec04ea10 2024-04-23T19:10:31.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.487+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.488+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.488+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.488+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.488+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.488+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.489+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.489+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.489+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.489+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.489+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.490+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.490+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.490+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.490+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.490+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.490+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.491+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.491+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62ec04ea10 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.491+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.491+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.491+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.491+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.492+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.492+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.492+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.492+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.492+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.493+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62ec04ea10 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.493+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.493+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c861862330 2024-04-23T19:10:31.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.493+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.493+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62e4017050 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.493+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.493+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x55c86185c120 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55c861864cb0 con 0x7f62ec04ea10 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:31.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c861860f40 con 0x55c86185c120 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.494+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618617e0 con 0x55c86185c120 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618617e0 con 0x7f62e4017050 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 zone default found 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 4 Realm: () 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 zonegroup default 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c861865130 con 0x7f62e4017050 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c861861500 con 0x55c861862330 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.495+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618670c0 con 0x55c861862330 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.496+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.496+0000 7f62f4519800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.496+0000 7f62f4519800 20 started sync module instance, tier type = 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.496+0000 7f62f4519800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:31.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.496+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c861866450 con 0x7f62ec04ea10 2024-04-23T19:10:31.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.497+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.497+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94319118018672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618660b0 con 0x7f62ec04ea10 2024-04-23T19:10:31.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.497+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c861867ff0 con 0x55c86185c120 2024-04-23T19:10:31.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.498+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.498+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94319118018672] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.498+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94319118023968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618685f0 con 0x55c86185c120 2024-04-23T19:10:31.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.498+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c861868e40 con 0x55c861862330 2024-04-23T19:10:31.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.499+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94319118023968] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.499+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.499+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94319118025504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c861868d30 con 0x55c861862330 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.499+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c861869850 con 0x7f62ec04ea10 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.501+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.501+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94319118025504] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.501+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94319118029600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186a0f0 con 0x7f62ec04ea10 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.501+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186a990 con 0x55c86185c120 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.503+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.503+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94319118034144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186aed0 con 0x55c86185c120 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.503+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186b9f0 con 0x7f62e4017050 2024-04-23T19:10:31.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.503+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94319118029600] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.504+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.504+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94319118038208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186b9b0 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.504+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186b9b0 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.504+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94319118034144] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.505+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94319118038208] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.505+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.505+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94319118031936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186bf50 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.505+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186c7a0 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94319118031936] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94319118040944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186c840 con 0x7f62e4017050 2024-04-23T19:10:31.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 20 add_watcher() i=0 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 20 add_watcher() i=1 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 20 add_watcher() i=2 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 20 add_watcher() i=3 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 20 add_watcher() i=4 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 20 add_watcher() i=5 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.506+0000 7f62f4519800 20 add_watcher() i=6 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.507+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94319118040944] v30'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.507+0000 7f62f4519800 20 add_watcher() i=7 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.507+0000 7f62f4519800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:31.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.507+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fc40 con 0x55c86185c120 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.508+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.508+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94319118042464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186d0f0 con 0x55c86185c120 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94319118042464] v30'128 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f61a8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 5 note: GC not initialized 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f61237fe640 20 reqs_thread_entry: start 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 20 Filter name: none 2024-04-23T19:10:31.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f61217fa640 20 reqs_thread_entry: start 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.510+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618a53e0 con 0x7f62e4017050 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f4519800 20 rados_obj.operate() r=0 bl.length=419 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f4519800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f4519800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f4519800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f4519800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.511+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618a4e30 con 0x7f62ec04ea10 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618a8a10 con 0x55c86185c120 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(64 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv5 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:31.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.512+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618ad170 con 0x7f62ec04ea10 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.513+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(65 foo.buckets [call] v30'24 uv24 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.513+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 8.2 8:40a7511a:root::.bucket.meta.testx%3amyfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [create,call version.inc in=38b,writefull 0~412 in=412b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b0a80 con 0x7f62ec04ea10 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [create,call,writefull 0~412,setxattr (203)] v30'3 uv3 ondisk = 0) ==== 338+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x7 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify cookie 94319118313648 in=859b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b0550 con 0x55c86185c120 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== watch-notify(notify (1) cookie 94319118034144 notify 128849018891 ret 0) ==== 889+0+0 (crc 0 0 0) 0x55c8618a8e00 con 0x55c86185c120 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f61d5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018891 cookie 94319118034144 notifier 4541 bl.length()=847 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f61d5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x7 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f61d5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f61d5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f61d5ffb640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f61b8003ea0 con 0x55c86185c120 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(67 notify.4 [notify cookie 94319118313648 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== watch-notify(notify_complete (2) cookie 94319118313648 notify 128849018891 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c8618a7f50 con 0x55c86185c120 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 20 handle_bi_update(): bucket=testx:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_sources= new_sources= 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 20 handle_bi_update(): bucket=testx:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential sources added=[] removed=[] 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 20 handle_bi_update(): bucket=testx:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_dests= new_dests= 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 20 handle_bi_update(): bucket=testx:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential dests added=[] removed=[] 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.515+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b0650 con 0x55c86185c120 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.517+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v30'1 uv1 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.517+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b0650 con 0x55c86185c120 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v30'1 uv1 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f4519800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f4519800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f4519800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 94319118320032 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b05c0 con 0x55c86185c120 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== watch-notify(notify (1) cookie 94319118034144 notify 128849018892 ret 0) ==== 390+0+0 (crc 0 0 0) 0x55c8618b09f0 con 0x55c86185c120 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f61d57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018892 cookie 94319118034144 notifier 4541 bl.length()=348 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f61d57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f61d57fa640 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f61d57fa640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f6128003400 con 0x55c86185c120 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(71 notify.4 [notify cookie 94319118320032 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== watch-notify(notify_complete (2) cookie 94319118320032 notify 128849018892 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c8618af8c0 con 0x55c86185c120 2024-04-23T19:10:31.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.518+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b0650 con 0x7f62ec04ea10 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(73 myfoo [call,delete] v30'25 uv25 ondisk = 0) ==== 191+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f4519800 10 removing default.rgw.meta+root+myfoo from cache 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f4519800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94319118320032 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b05c0 con 0x7f62e4017050 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== watch-notify(notify (1) cookie 94319118031936 notify 128849018891 ret 0) ==== 191+0+0 (crc 0 0 0) 0x55c8618af8c0 con 0x7f62e4017050 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f61d5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018891 cookie 94319118031936 notifier 4541 bl.length()=149 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.6 [notify cookie 94319118320032 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f61d5ffb640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f61b8003ea0 con 0x7f62e4017050 2024-04-23T19:10:31.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== watch-notify(notify_complete (2) cookie 94319118320032 notify 128849018891 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c8618a7f50 con 0x7f62e4017050 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.520+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b0650 con 0x55c86185c120 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.521+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(76 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call,delete] v30'6 uv6 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.521+0000 7f62f4519800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 from cache 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.521+0000 7f62f4519800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.521+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.3 7:c609908c:::notify.5:head [notify cookie 94319118291120 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55c8618b0650 con 0x7f62e4017050 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.522+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== watch-notify(notify (1) cookie 94319118038208 notify 128849018892 ret 0) ==== 248+0+0 (crc 0 0 0) 0x55c8618a7f50 con 0x7f62e4017050 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.522+0000 7f61d5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018892 cookie 94319118038208 notifier 4541 bl.length()=206 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.522+0000 7f61d5ffb640 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f61b8003ea0 con 0x7f62e4017050 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.522+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(77 notify.5 [notify cookie 94319118291120 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.522+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(78 notify.5 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.522+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== watch-notify(notify_complete (2) cookie 94319118291120 notify 128849018892 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c8615e7b50 con 0x7f62e4017050 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.523+0000 7f61a8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.523+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94319118042464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x55c86185c120 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.524+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94319118042464] v30'129 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.524+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94319118018672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x7f62ec04ea10 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.525+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94319118018672] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.525+0000 7f62f4519800 20 remove_watcher() i=0 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.525+0000 7f62f4519800 2 removed watcher, disabling cache 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.525+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94319118023968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x55c86185c120 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.526+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94319118023968] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.526+0000 7f62f4519800 20 remove_watcher() i=1 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.526+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94319118025504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x55c861862330 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.527+0000 7f62f2820640 1 -- 172.21.15.63:0/2002017430 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94319118025504] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ec029070 con 0x55c861862330 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.527+0000 7f62f4519800 20 remove_watcher() i=2 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.527+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94319118029600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x7f62ec04ea10 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.528+0000 7f62f201f640 1 -- 172.21.15.63:0/2002017430 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94319118029600] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e4024980 con 0x7f62ec04ea10 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.528+0000 7f62f4519800 20 remove_watcher() i=3 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.528+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94319118034144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x55c86185c120 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.529+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94319118034144] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x55c86185c120 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.529+0000 7f62f4519800 20 remove_watcher() i=4 2024-04-23T19:10:31.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.529+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94319118038208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x7f62e4017050 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.530+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94319118038208] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.530+0000 7f62f4519800 20 remove_watcher() i=5 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.530+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94319118031936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x7f62e4017050 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.531+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94319118031936] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.531+0000 7f62f4519800 20 remove_watcher() i=6 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.531+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94319118040944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55c86186fb10 con 0x7f62e4017050 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94319118040944] v30'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62e805a040 con 0x7f62e4017050 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 20 remove_watcher() i=7 2024-04-23T19:10:31.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f62e4017050 msgr2=0x7f62e40165c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f62e4017050 0x7f62e40165c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c86185c120 msgr2=0x55c86185e5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c86185c120 0x55c86185e5e0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f62ec04ea10 msgr2=0x7f62ec02c460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f62ec04ea10 0x7f62ec02c460 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c861862330 msgr2=0x55c861864720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c861862330 0x55c861864720 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61bc036e60 msgr2=0x7f61bc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61bc036e60 0x7f61bc039320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62e4024c90 tx=0x7f62e4030000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 msgr2=0x55c861850430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f181e640 1 -- 172.21.15.63:0/2002017430 reap_dead start 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.532+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c861850430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f62ec048040 tx=0x7f62ec04cbd0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 shutdown_connections 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 --2- 172.21.15.63:0/2002017430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c861848ea0 0x55c861850430 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 >> 172.21.15.63:0/2002017430 conn(0x55c861845200 msgr2=0x55c861842860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 shutdown_connections 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2002017430 wait complete. 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61cc036e60 msgr2=0x7f61cc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61cc036e60 0x7f61cc039320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f62e40015f0 tx=0x7f62e404b440 comp rx=0 tx=0).stop 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86183bba0 msgr2=0x55c86183bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86183bba0 0x55c86183bfc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55c861828090 tx=0x7f62ec0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 shutdown_connections 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61cc036e60 0x7f61cc039320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 --2- 172.21.15.63:0/2661580178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86183bba0 0x55c86183bfc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 >> 172.21.15.63:0/2661580178 conn(0x55c86182fc40 msgr2=0x55c861830040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 shutdown_connections 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/2661580178 wait complete. 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c86182ae80 msgr2=0x55c86182d270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c86182ae80 0x55c86182d270 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c8618253c0 msgr2=0x55c8618277b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.533+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c8618253c0 0x55c8618277b0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61d8036e60 msgr2=0x7f61d8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61d8036e60 0x7f61d8039320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62e8054000 tx=0x7f62e804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 msgr2=0x55c861820190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c861820190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f62e4001de0 tx=0x7f62e4000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 shutdown_connections 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c86182ae80 0x55c86182d270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c8618253c0 0x55c8618277b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f61d8036e60 0x7f61d8039320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 --2- 172.21.15.63:0/1576643938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c86172b6c0 0x55c861820190 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 >> 172.21.15.63:0/1576643938 conn(0x55c861721620 msgr2=0x55c8618124c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 shutdown_connections 2024-04-23T19:10:31.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.534+0000 7f62f4519800 1 -- 172.21.15.63:0/1576643938 wait complete. 2024-04-23T19:10:31.556 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-04-23T19:10:31.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-04-23T19:10:31.556 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-04-23T19:10:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.599+0000 7fb995533800 1 Processor -- start 2024-04-23T19:10:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb995533800 1 -- start start 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb995533800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55d0c44d8c10 0x55d0c44db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb995533800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d0c438a7f0 con 0x55d0c44db540 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb995533800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55d0c4379e20 con 0x55d0c44d8c10 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb9912f2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55d0c44d8c10 0x55d0c44db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb990af1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55d0c44db540 0x55d0c44dd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38840/0 (socket says 172.21.15.63:38840) 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb990af1640 1 -- 172.21.15.63:0/3738283825 learned_addr learned my addr 172.21.15.63:0/3738283825 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb9912f2640 1 -- 172.21.15.63:0/3738283825 >> v1:172.21.15.16:6789/0 conn(0x55d0c44db540 legacy=0x55d0c44dd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.600+0000 7fb9912f2640 1 -- 172.21.15.63:0/3738283825 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c4398b50 con 0x55d0c44d8c10 2024-04-23T19:10:31.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.602+0000 7fb9912f2640 1 --2- 172.21.15.63:0/3738283825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c44db000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb98400a470 tx=0x7fb98401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1463622ac1ec92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.602+0000 7fb983fff640 1 -- 172.21.15.63:0/3738283825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb984057070 con 0x55d0c44d8c10 2024-04-23T19:10:31.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.602+0000 7fb983fff640 1 -- 172.21.15.63:0/3738283825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb984052570 con 0x55d0c44d8c10 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.602+0000 7fb983fff640 1 -- 172.21.15.63:0/3738283825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb984052870 con 0x55d0c44d8c10 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.603+0000 7fb995533800 1 -- 172.21.15.63:0/3738283825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 msgr2=0x55d0c44db000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.603+0000 7fb995533800 1 --2- 172.21.15.63:0/3738283825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c44db000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb98400a470 tx=0x7fb98401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.603+0000 7fb995533800 1 -- 172.21.15.63:0/3738283825 shutdown_connections 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.603+0000 7fb995533800 1 --2- 172.21.15.63:0/3738283825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c44db000 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.603+0000 7fb995533800 1 -- 172.21.15.63:0/3738283825 >> 172.21.15.63:0/3738283825 conn(0x55d0c44d2600 msgr2=0x55d0c44d4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.603+0000 7fb995533800 1 -- 172.21.15.63:0/3738283825 shutdown_connections 2024-04-23T19:10:31.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.603+0000 7fb995533800 1 -- 172.21.15.63:0/3738283825 wait complete. 2024-04-23T19:10:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb995533800 1 Processor -- start 2024-04-23T19:10:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb995533800 1 -- start start 2024-04-23T19:10:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb995533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb995533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0c44236f0 con 0x55d0c44d8c10 2024-04-23T19:10:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb9912f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb9912f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54938/0 (socket says 172.21.15.63:54938) 2024-04-23T19:10:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb9912f2640 1 -- 172.21.15.63:0/3771773615 learned_addr learned my addr 172.21.15.63:0/3771773615 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.616+0000 7fb9912f2640 1 -- 172.21.15.63:0/3771773615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c45c9950 con 0x55d0c44d8c10 2024-04-23T19:10:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb9912f2640 1 --2- 172.21.15.63:0/3771773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45c9380 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb98401ced0 tx=0x7fb98404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb57425e4e32582c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb87f606640 1 -- 172.21.15.63:0/3771773615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb984057040 con 0x55d0c44d8c10 2024-04-23T19:10:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb87f606640 1 -- 172.21.15.63:0/3771773615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9840529a0 con 0x55d0c44d8c10 2024-04-23T19:10:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb87f606640 1 -- 172.21.15.63:0/3771773615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb984052cc0 con 0x55d0c44d8c10 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb995533800 1 -- 172.21.15.63:0/3771773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 msgr2=0x55d0c45c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb995533800 1 --2- 172.21.15.63:0/3771773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45c9380 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb98401ced0 tx=0x7fb98404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb995533800 1 -- 172.21.15.63:0/3771773615 shutdown_connections 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb995533800 1 --2- 172.21.15.63:0/3771773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45c9380 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.617+0000 7fb995533800 1 -- 172.21.15.63:0/3771773615 >> 172.21.15.63:0/3771773615 conn(0x55d0c44d2600 msgr2=0x55d0c45c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb995533800 1 -- 172.21.15.63:0/3771773615 shutdown_connections 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb995533800 1 -- 172.21.15.63:0/3771773615 wait complete. 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb995533800 1 Processor -- start 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb995533800 1 -- start start 2024-04-23T19:10:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb995533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb995533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0c44e1360 con 0x55d0c44d8c10 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb9912f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb9912f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45d0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54948/0 (socket says 172.21.15.63:54948) 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb9912f2640 1 -- 172.21.15.63:0/113273297 learned_addr learned my addr 172.21.15.63:0/113273297 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.618+0000 7fb9912f2640 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c45d1290 con 0x55d0c44d8c10 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.619+0000 7fb9912f2640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45d0d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb98401ced0 tx=0x7fb984003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.619+0000 7fb87de03640 1 -- 172.21.15.63:0/113273297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb984073070 con 0x55d0c44d8c10 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.619+0000 7fb87de03640 1 -- 172.21.15.63:0/113273297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98405d840 con 0x55d0c44d8c10 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.619+0000 7fb87de03640 1 -- 172.21.15.63:0/113273297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb984057070 con 0x55d0c44d8c10 2024-04-23T19:10:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.619+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d0c45d1520 con 0x55d0c44d8c10 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.619+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c45d41d0 con 0x55d0c44d8c10 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb87de03640 1 -- 172.21.15.63:0/113273297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb984076070 con 0x55d0c44d8c10 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb87de03640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb878036e60 0x7fb878039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb87de03640 1 -- 172.21.15.63:0/113273297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fb98408e690 con 0x55d0c44d8c10 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb990af1640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb878036e60 0x7fb878039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c44db540 0x55d0c45d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c45d8570 con 0x55d0c44db540 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb991af3640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c44db540 0x55d0c45d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb990af1640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb878036e60 0x7fb878039320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb988000f60 tx=0x7fb9880012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.620+0000 7fb991af3640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c44db540 0x55d0c45d7f90 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.621+0000 7fb991af3640 1 -- 172.21.15.63:0/113273297 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb98c00a4a0 con 0x55d0c44db540 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.621+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c45d5560 con 0x55d0c44db540 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.621+0000 7fb991af3640 1 -- 172.21.15.63:0/113273297 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb98c00a4a0 con 0x55d0c44db540 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.621+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c45da3b0 con 0x55d0c44db540 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.621+0000 7fb991af3640 1 -- 172.21.15.63:0/113273297 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fb98c00a4a0 con 0x55d0c44db540 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.621+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c45d96f0 con 0x55d0c44db540 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.622+0000 7fb991af3640 1 -- 172.21.15.63:0/113273297 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb98c00a4a0 con 0x55d0c44db540 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.622+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d0c45db760 0x55d0c45ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.622+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c45de0d0 con 0x55d0c45db760 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.622+0000 7fb9912f2640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d0c45db760 0x55d0c45ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.622+0000 7fb9912f2640 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d0c45db760 0x55d0c45ddb50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb9912f2640 1 -- 172.21.15.63:0/113273297 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fb984075030 con 0x55d0c45db760 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb995533800 1 Processor -- start 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb995533800 1 -- start start 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb995533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c45ec4f0 0x55d0c45ec910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb995533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0c43c9e30 con 0x55d0c45ec4f0 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb991af3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c45ec4f0 0x55d0c45ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb991af3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c45ec4f0 0x55d0c45ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54958/0 (socket says 172.21.15.63:54958) 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.623+0000 7fb991af3640 1 -- 172.21.15.63:0/1212400660 learned_addr learned my addr 172.21.15.63:0/1212400660 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.624+0000 7fb991af3640 1 -- 172.21.15.63:0/1212400660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c45ece50 con 0x55d0c45ec4f0 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.624+0000 7fb991af3640 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c45ec4f0 0x55d0c45ec910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55d0c45d2150 tx=0x7fb98c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.624+0000 7fb876ffd640 1 -- 172.21.15.63:0/1212400660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c013440 con 0x55d0c45ec4f0 2024-04-23T19:10:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.624+0000 7fb876ffd640 1 -- 172.21.15.63:0/1212400660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c0135e0 con 0x55d0c45ec4f0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.624+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d0c45ed8c0 con 0x55d0c45ec4f0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.624+0000 7fb876ffd640 1 -- 172.21.15.63:0/1212400660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c0138c0 con 0x55d0c45ec4f0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.624+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c45eddc0 con 0x55d0c45ec4f0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb876ffd640 1 -- 172.21.15.63:0/1212400660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb98c011040 con 0x55d0c45ec4f0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb876ffd640 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb86c036e60 0x7fb86c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb876ffd640 1 -- 172.21.15.63:0/1212400660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fb98c01d070 con 0x55d0c45ec4f0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb9912f2640 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb86c036e60 0x7fb86c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb995533800 1 Processor -- start 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb995533800 1 -- start start 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb9912f2640 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb86c036e60 0x7fb86c039320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb984066c20 tx=0x7fb98408ca30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb995533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c44d8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.625+0000 7fb995533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0c45c9c10 con 0x55d0c44d8200 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb991af3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c44d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb991af3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c44d8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54966/0 (socket says 172.21.15.63:54966) 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb991af3640 1 -- 172.21.15.63:0/3233899379 learned_addr learned my addr 172.21.15.63:0/3233899379 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb991af3640 1 -- 172.21.15.63:0/3233899379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c44d6a00 con 0x55d0c44d8200 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb991af3640 1 --2- 172.21.15.63:0/3233899379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c44d8620 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb98c04ccb0 tx=0x7fb98c001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e04901a03105c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb874ff9640 1 -- 172.21.15.63:0/3233899379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c012df0 con 0x55d0c44d8200 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb874ff9640 1 -- 172.21.15.63:0/3233899379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c001310 con 0x55d0c44d8200 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.626+0000 7fb874ff9640 1 -- 172.21.15.63:0/3233899379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c001630 con 0x55d0c44d8200 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 -- 172.21.15.63:0/3233899379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 msgr2=0x55d0c44d8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 --2- 172.21.15.63:0/3233899379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c44d8620 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb98c04ccb0 tx=0x7fb98c001f90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 -- 172.21.15.63:0/3233899379 shutdown_connections 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 --2- 172.21.15.63:0/3233899379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c44d8620 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 -- 172.21.15.63:0/3233899379 >> 172.21.15.63:0/3233899379 conn(0x55d0c45f5b70 msgr2=0x55d0c45f3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 -- 172.21.15.63:0/3233899379 shutdown_connections 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 -- 172.21.15.63:0/3233899379 wait complete. 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 Processor -- start 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 -- start start 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c4602fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb995533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0c44d6ed0 con 0x55d0c44d8200 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb991af3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c4602fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb991af3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c4602fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54974/0 (socket says 172.21.15.63:54974) 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.627+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 learned_addr learned my addr 172.21.15.63:0/366726418 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.628+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c46034e0 con 0x55d0c44d8200 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.628+0000 7fb991af3640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c4602fa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb98c001fd0 tx=0x7fb98c04cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.628+0000 7fb862ffd640 1 -- 172.21.15.63:0/366726418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c02ae30 con 0x55d0c44d8200 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.628+0000 7fb862ffd640 1 -- 172.21.15.63:0/366726418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c021c60 con 0x55d0c44d8200 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.628+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d0c4603770 con 0x55d0c44d8200 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.628+0000 7fb862ffd640 1 -- 172.21.15.63:0/366726418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c02a350 con 0x55d0c44d8200 2024-04-23T19:10:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.628+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c4606420 con 0x55d0c44d8200 2024-04-23T19:10:31.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.629+0000 7fb862ffd640 1 -- 172.21.15.63:0/366726418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb98c040040 con 0x55d0c44d8200 2024-04-23T19:10:31.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.629+0000 7fb862ffd640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb85c036e60 0x7fb85c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.629+0000 7fb862ffd640 1 -- 172.21.15.63:0/366726418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fb98c032030 con 0x55d0c44d8200 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.629+0000 7fb9912f2640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb85c036e60 0x7fb85c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.629+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.629+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c460ec40 0x55d0c4611100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.629+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4611850 con 0x55d0c460ec40 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb990af1640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c460ec40 0x55d0c4611100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb990af1640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c460ec40 0x55d0c4611100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb9912f2640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb85c036e60 0x7fb85c039320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d0c45de4b0 tx=0x7fb984088040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb988009050 con 0x55d0c460ec40 2024-04-23T19:10:31.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 20 realm 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c460e090 con 0x55d0c460ec40 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb988009050 con 0x55d0c460ec40 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.630+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4612330 con 0x55d0c460ec40 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb988009050 con 0x55d0c460ec40 2024-04-23T19:10:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4612c00 con 0x55d0c460ec40 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb988009050 con 0x55d0c460ec40 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4612fa0 con 0x55d0c460ec40 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fb988009050 con 0x55d0c460ec40 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 20 searching for the correct realm 2024-04-23T19:10:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d0c4614e80 0x55d0c4617270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x55d0c4614e80 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.631+0000 7fb991af3640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d0c4614e80 0x55d0c4617270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.632+0000 7fb991af3640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d0c4614e80 0x55d0c4617270 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.632+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.632+0000 7fb991af3640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fb98c046070 0x7fb98c037430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.632+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x7fb98c046070 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.632+0000 7fb9912f2640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fb98c046070 0x7fb98c037430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.632+0000 7fb9912f2640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fb98c046070 0x7fb98c037430 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.633+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.633+0000 7fb9912f2640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fb98409b040 0x7fb98408ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.633+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x7fb98409b040 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.633+0000 7fb990af1640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fb98409b040 0x7fb98408ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.633+0000 7fb990af1640 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fb98409b040 0x7fb98408ea70 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb988009050 con 0x7fb98409b040 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x55d0c460ec40 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb988009050 con 0x55d0c460ec40 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x55d0c460ec40 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x7fb98409b040 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.634+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x7fb98409b040 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x7fb98c046070 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x55d0c4614e80 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x55d0c460ec40 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.635+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x7fb98409b040 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.636+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.636+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x55d0c460ec40 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.636+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.636+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617920 con 0x7fb98c046070 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.636+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.637+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c4614e80 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.637+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.637+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c4614e80 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.637+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.637+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98409b040 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c4614e80 2024-04-23T19:10:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c4614e80 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c460ec40 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c460ec40 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.638+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98409b040 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.639+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.639+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c4614e80 2024-04-23T19:10:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.639+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.639+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98c046070 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.639+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.639+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c460ec40 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98409b040 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c4614e80 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98409b040 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.640+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98c046070 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.641+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.641+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c4614e80 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.641+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.641+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98409b040 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.641+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.641+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x55d0c460ec40 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fb9840a9060 con 0x7fb98c046070 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4613950 con 0x55d0c460ec40 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c46141f0 con 0x55d0c460ec40 2024-04-23T19:10:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.642+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4613590 con 0x7fb98409b040 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 20 zone default found 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 4 Realm: () 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 20 zonegroup default 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4617d80 con 0x7fb98409b040 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.643+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4619c10 con 0x55d0c4614e80 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb995533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4619fb0 con 0x55d0c4614e80 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb995533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb995533800 20 started sync module instance, tier type = 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb995533800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.644+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4619050 con 0x7fb98c046070 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.645+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.645+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94355136287856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461a800 con 0x7fb98c046070 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.645+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461b050 con 0x55d0c460ec40 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.646+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94355136287856] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.646+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.646+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94355136291664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461b0b0 con 0x55d0c460ec40 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.646+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461bba0 con 0x55d0c4614e80 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.647+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94355136291664] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.647+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.647+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94355136296560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461bb60 con 0x55d0c4614e80 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.648+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461c600 con 0x7fb98c046070 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.648+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94355136296560] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.649+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.649+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94355136299312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461ca20 con 0x7fb98c046070 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.649+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461d470 con 0x55d0c460ec40 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.649+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94355136299312] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.650+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.650+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94355136302944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461d860 con 0x55d0c460ec40 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.650+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461e280 con 0x7fb98409b040 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.651+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94355136302944] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.651+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.651+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94355136306544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461e240 con 0x7fb98409b040 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.651+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461eca0 con 0x7fb98409b040 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.652+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(56 notify.6 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.652+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94355136309120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461f150 con 0x7fb98409b040 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.652+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461fbd0 con 0x7fb98409b040 2024-04-23T19:10:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.652+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94355136306544] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94355136309120] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94355136313024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461eca0 con 0x7fb98409b040 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 20 add_watcher() i=0 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 20 add_watcher() i=1 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 20 add_watcher() i=2 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 20 add_watcher() i=3 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 20 add_watcher() i=4 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 20 add_watcher() i=5 2024-04-23T19:10:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.653+0000 7fb995533800 20 add_watcher() i=6 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.654+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94355136313024] v30'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.654+0000 7fb995533800 20 add_watcher() i=7 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.654+0000 7fb995533800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.654+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4622cd0 con 0x55d0c460ec40 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.654+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.654+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94355136293792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4620150 con 0x55d0c460ec40 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.655+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94355136293792] v30'130 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.655+0000 7fb8477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.655+0000 7fb995533800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.655+0000 7fb995533800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.655+0000 7fb995533800 5 note: GC not initialized 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb8467e4640 20 reqs_thread_entry: start 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb995533800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb995533800 20 Filter name: none 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb7b7fff640 20 reqs_thread_entry: start 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb995533800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4655a10 con 0x55d0c460ec40 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 281+0+332 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.656+0000 7fb995533800 20 rados_obj.operate() r=0 bl.length=193 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.657+0000 7fb8477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.657+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94355136293792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c4620150 con 0x55d0c460ec40 2024-04-23T19:10:31.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.658+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94355136293792] v30'131 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.658+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94355136287856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461aba0 con 0x7fb98c046070 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.659+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94355136287856] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.659+0000 7fb995533800 20 remove_watcher() i=0 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.659+0000 7fb995533800 2 removed watcher, disabling cache 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.659+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94355136291664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461aba0 con 0x55d0c460ec40 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.660+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94355136291664] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.660+0000 7fb995533800 20 remove_watcher() i=1 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.660+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94355136296560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461a350 con 0x55d0c4614e80 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.661+0000 7fb991af3640 1 -- 172.21.15.63:0/366726418 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94355136296560] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98c029070 con 0x55d0c4614e80 2024-04-23T19:10:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.661+0000 7fb995533800 20 remove_watcher() i=2 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.661+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94355136299312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461aba0 con 0x7fb98c046070 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.662+0000 7fb9912f2640 1 -- 172.21.15.63:0/366726418 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94355136299312] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb984097030 con 0x7fb98c046070 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.662+0000 7fb995533800 20 remove_watcher() i=3 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.662+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94355136302944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461b670 con 0x55d0c460ec40 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.663+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94355136302944] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x55d0c460ec40 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.663+0000 7fb995533800 20 remove_watcher() i=4 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.663+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94355136306544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461c130 con 0x7fb98409b040 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.664+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94355136306544] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.664+0000 7fb995533800 20 remove_watcher() i=5 2024-04-23T19:10:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.664+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94355136309120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461aba0 con 0x7fb98409b040 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.665+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94355136309120] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.665+0000 7fb995533800 20 remove_watcher() i=6 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.665+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94355136313024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55d0c461aba0 con 0x7fb98409b040 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94355136313024] v30'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb98802a040 con 0x7fb98409b040 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 20 remove_watcher() i=7 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fb98409b040 msgr2=0x7fb98408ea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fb98409b040 0x7fb98408ea70 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c460ec40 msgr2=0x55d0c4611100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c460ec40 0x55d0c4611100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fb98c046070 msgr2=0x7fb98c037430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fb98c046070 0x7fb98c037430 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d0c4614e80 msgr2=0x55d0c4617270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.666+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55d0c4614e80 0x55d0c4617270 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb85c036e60 msgr2=0x7fb85c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb85c036e60 0x7fb85c039320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d0c45de4b0 tx=0x7fb984088040 comp rx=0 tx=0).stop 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 msgr2=0x55d0c4602fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb990af1640 1 -- 172.21.15.63:0/366726418 reap_dead start 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c4602fa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb98c001fd0 tx=0x7fb98c04cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 shutdown_connections 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/366726418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8200 0x55d0c4602fa0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 >> 172.21.15.63:0/366726418 conn(0x55d0c45f5b70 msgr2=0x55d0c45f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 shutdown_connections 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/366726418 wait complete. 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb86c036e60 msgr2=0x7fb86c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb86c036e60 0x7fb86c039320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb984066c20 tx=0x7fb98408ca30 comp rx=0 tx=0).stop 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c45ec4f0 msgr2=0x55d0c45ec910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c45ec4f0 0x55d0c45ec910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55d0c45d2150 tx=0x7fb98c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 shutdown_connections 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb86c036e60 0x7fb86c039320 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/1212400660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c45ec4f0 0x55d0c45ec910 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 >> 172.21.15.63:0/1212400660 conn(0x55d0c45e0590 msgr2=0x55d0c45e0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 shutdown_connections 2024-04-23T19:10:31.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/1212400660 wait complete. 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d0c45db760 msgr2=0x55d0c45ddb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d0c45db760 0x55d0c45ddb50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c44db540 msgr2=0x55d0c45d7f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.667+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c44db540 0x55d0c45d7f90 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb878036e60 msgr2=0x7fb878039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb878036e60 0x7fb878039320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb988000f60 tx=0x7fb9880012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 msgr2=0x55d0c45d0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45d0d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb98401ced0 tx=0x7fb984003020 comp rx=0 tx=0).stop 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 shutdown_connections 2024-04-23T19:10:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55d0c45db760 0x55d0c45ddb50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55d0c44db540 0x55d0c45d7f90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb878036e60 0x7fb878039320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 --2- 172.21.15.63:0/113273297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0c44d8c10 0x55d0c45d0d50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 >> 172.21.15.63:0/113273297 conn(0x55d0c44d2600 msgr2=0x55d0c45c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 shutdown_connections 2024-04-23T19:10:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.668+0000 7fb995533800 1 -- 172.21.15.63:0/113273297 wait complete. 2024-04-23T19:10:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:{"key":"bucket:testx/myfoo","ver":{"tag":"lLXMzISW9iDPRatdQlKK5F7F","ver":2},"mtime":"2024-04-23T19:10:31.517457Z","data":{"bucket":{"name":"myfoo","marker":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","bucket_id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","tenant":"testx","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"testx$tenanteduser","creation_time":"2024-04-23T19:10:29.008577Z","linked":true,"has_bucket_info":false}} 2024-04-23T19:10:31.690 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'lLXMzISW9iDPRatdQlKK5F7F', 'ver': 2}, 'mtime': '2024-04-23T19:10:31.517457Z', 'data': {'bucket': {'name': 'myfoo', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'bucket_id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-04-23T19:10:29.008577Z', 'linked': True, 'has_bucket_info': False}} 2024-04-23T19:10:31.690 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-23T19:10:31.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-23T19:10:31.691 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-04-23T19:10:31.776 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:31.777 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f0337b2a800 1 Processor -- start 2024-04-23T19:10:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f0337b2a800 1 -- start start 2024-04-23T19:10:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f0337b2a800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559f0406b6a0 0x559f0406bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f0337b2a800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559f03f1d7f0 con 0x559f0406c000 2024-04-23T19:10:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f0337b2a800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x559f03f0ce20 con 0x559f0406b6a0 2024-04-23T19:10:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f03337f2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559f0406b6a0 0x559f0406bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f0332ff1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x559f0406c000 0x559f04072950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38844/0 (socket says 172.21.15.63:38844) 2024-04-23T19:10:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f0332ff1640 1 -- 172.21.15.63:0/3671299524 learned_addr learned my addr 172.21.15.63:0/3671299524 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f03337f2640 1 -- 172.21.15.63:0/3671299524 >> v1:172.21.15.16:6789/0 conn(0x559f0406c000 legacy=0x559f04072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.777+0000 7f03337f2640 1 -- 172.21.15.63:0/3671299524 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f03f2bb50 con 0x559f0406b6a0 2024-04-23T19:10:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.779+0000 7f03337f2640 1 --2- 172.21.15.63:0/3671299524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0406bac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f032400a470 tx=0x7f032401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a977b05d3da861 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.779+0000 7f03327f0640 1 -- 172.21.15.63:0/3671299524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324057070 con 0x559f0406b6a0 2024-04-23T19:10:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.779+0000 7f03327f0640 1 -- 172.21.15.63:0/3671299524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0324052570 con 0x559f0406b6a0 2024-04-23T19:10:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.779+0000 7f03327f0640 1 -- 172.21.15.63:0/3671299524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324052870 con 0x559f0406b6a0 2024-04-23T19:10:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.780+0000 7f0337b2a800 1 -- 172.21.15.63:0/3671299524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 msgr2=0x559f0406bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.780+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3671299524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0406bac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f032400a470 tx=0x7f032401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.780+0000 7f0337b2a800 1 -- 172.21.15.63:0/3671299524 shutdown_connections 2024-04-23T19:10:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.780+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3671299524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0406bac0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.780+0000 7f0337b2a800 1 -- 172.21.15.63:0/3671299524 >> 172.21.15.63:0/3671299524 conn(0x559f04064db0 msgr2=0x559f040671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.780+0000 7f0337b2a800 1 -- 172.21.15.63:0/3671299524 shutdown_connections 2024-04-23T19:10:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.780+0000 7f0337b2a800 1 -- 172.21.15.63:0/3671299524 wait complete. 2024-04-23T19:10:31.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.793+0000 7f0337b2a800 1 Processor -- start 2024-04-23T19:10:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.793+0000 7f0337b2a800 1 -- start start 2024-04-23T19:10:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.793+0000 7f0337b2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.793+0000 7f0337b2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f03fb66f0 con 0x559f0406b6a0 2024-04-23T19:10:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.793+0000 7f03337f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.793+0000 7f03337f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0415e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:54998/0 (socket says 172.21.15.63:54998) 2024-04-23T19:10:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.793+0000 7f03337f2640 1 -- 172.21.15.63:0/1324582889 learned_addr learned my addr 172.21.15.63:0/1324582889 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.794+0000 7f03337f2640 1 -- 172.21.15.63:0/1324582889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f0415eb70 con 0x559f0406b6a0 2024-04-23T19:10:31.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.794+0000 7f03337f2640 1 --2- 172.21.15.63:0/1324582889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0415e5a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0324055040 tx=0x7f032404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa127f19c07caa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.794+0000 7f022160a640 1 -- 172.21.15.63:0/1324582889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324057040 con 0x559f0406b6a0 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.794+0000 7f022160a640 1 -- 172.21.15.63:0/1324582889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0324052990 con 0x559f0406b6a0 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.794+0000 7f022160a640 1 -- 172.21.15.63:0/1324582889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324052c70 con 0x559f0406b6a0 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 -- 172.21.15.63:0/1324582889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 msgr2=0x559f0415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 --2- 172.21.15.63:0/1324582889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0415e5a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0324055040 tx=0x7f032404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 -- 172.21.15.63:0/1324582889 shutdown_connections 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 --2- 172.21.15.63:0/1324582889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406b6a0 0x559f0415e5a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 -- 172.21.15.63:0/1324582889 >> 172.21.15.63:0/1324582889 conn(0x559f04064db0 msgr2=0x559f04159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 -- 172.21.15.63:0/1324582889 shutdown_connections 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 -- 172.21.15.63:0/1324582889 wait complete. 2024-04-23T19:10:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 Processor -- start 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 -- start start 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406c000 0x559f04166200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.795+0000 7f0337b2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f040743c0 con 0x559f0406c000 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f03337f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406c000 0x559f04166200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f03337f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406c000 0x559f04166200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55014/0 (socket says 172.21.15.63:55014) 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f03337f2640 1 -- 172.21.15.63:0/82459987 learned_addr learned my addr 172.21.15.63:0/82459987 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f03337f2640 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f04166740 con 0x559f0406c000 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f03337f2640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406c000 0x559f04166200 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f032400a470 tx=0x7f0324003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f021fe07640 1 -- 172.21.15.63:0/82459987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324073070 con 0x559f0406c000 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f021fe07640 1 -- 172.21.15.63:0/82459987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032405d650 con 0x559f0406c000 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f021fe07640 1 -- 172.21.15.63:0/82459987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0324057070 con 0x559f0406c000 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f041669d0 con 0x559f0406c000 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.796+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f04169680 con 0x559f0406c000 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.797+0000 7f021fe07640 1 -- 172.21.15.63:0/82459987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0324076070 con 0x559f0406c000 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.797+0000 7f021fe07640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0218036e60 0x7f0218039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.797+0000 7f021fe07640 1 -- 172.21.15.63:0/82459987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f032408e5c0 con 0x559f0406c000 2024-04-23T19:10:31.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.797+0000 7f0332ff1640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0218036e60 0x7f0218039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.797+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f0416b430 0x559f0416d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.797+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f0416dd60 con 0x559f0416b430 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.798+0000 7f0333ff3640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f0416b430 0x559f0416d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.798+0000 7f0332ff1640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0218036e60 0x7f0218039320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0328000f60 tx=0x7f03280012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.798+0000 7f0333ff3640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f0416b430 0x559f0416d820 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.798+0000 7f0333ff3640 1 -- 172.21.15.63:0/82459987 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f032c00a4a0 con 0x559f0416b430 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.798+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f0416aa10 con 0x559f0416b430 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f0333ff3640 1 -- 172.21.15.63:0/82459987 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f032c00a4a0 con 0x559f0416b430 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f0416fba0 con 0x559f0416b430 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f0333ff3640 1 -- 172.21.15.63:0/82459987 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f032c00a4a0 con 0x559f0416b430 2024-04-23T19:10:31.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f0416eee0 con 0x559f0416b430 2024-04-23T19:10:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f0333ff3640 1 -- 172.21.15.63:0/82459987 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f032c00a4a0 con 0x559f0416b430 2024-04-23T19:10:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f04170f50 0x559f04173340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041738c0 con 0x559f04170f50 2024-04-23T19:10:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.799+0000 7f03337f2640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f04170f50 0x559f04173340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.800+0000 7f03337f2640 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f04170f50 0x559f04173340 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.800+0000 7f03337f2640 1 -- 172.21.15.63:0/82459987 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0324075030 con 0x559f04170f50 2024-04-23T19:10:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.800+0000 7f0337b2a800 1 Processor -- start 2024-04-23T19:10:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.800+0000 7f0337b2a800 1 -- start start 2024-04-23T19:10:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0337b2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f04181ce0 0x559f04182100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0337b2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f03f5ce30 con 0x559f04181ce0 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0333ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f04181ce0 0x559f04182100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0333ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f04181ce0 0x559f04182100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55028/0 (socket says 172.21.15.63:55028) 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0333ff3640 1 -- 172.21.15.63:0/3294503214 learned_addr learned my addr 172.21.15.63:0/3294503214 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0333ff3640 1 -- 172.21.15.63:0/3294503214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f04182640 con 0x559f04181ce0 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0333ff3640 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f04181ce0 0x559f04182100 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x559f04167600 tx=0x7f032c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f021d602640 1 -- 172.21.15.63:0/3294503214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032c013440 con 0x559f04181ce0 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f021d602640 1 -- 172.21.15.63:0/3294503214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c0135e0 con 0x559f04181ce0 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f021d602640 1 -- 172.21.15.63:0/3294503214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032c0138c0 con 0x559f04181ce0 2024-04-23T19:10:31.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.801+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f041830b0 con 0x559f04181ce0 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.802+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f041835b0 con 0x559f04181ce0 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.802+0000 7f021d602640 1 -- 172.21.15.63:0/3294503214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f032c011040 con 0x559f04181ce0 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.802+0000 7f021d602640 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f020c036e60 0x7f020c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.802+0000 7f021d602640 1 -- 172.21.15.63:0/3294503214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f032c01d070 con 0x559f04181ce0 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.802+0000 7f03337f2640 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f020c036e60 0x7f020c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0337b2a800 1 Processor -- start 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0337b2a800 1 -- start start 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f03337f2640 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f020c036e60 0x7f020c039320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f032408c9d0 tx=0x7f032408c8a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0337b2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f0418f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0337b2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f0415ee30 con 0x559f0418f090 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0333ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f0418f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0333ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f0418f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55044/0 (socket says 172.21.15.63:55044) 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0333ff3640 1 -- 172.21.15.63:0/472932892 learned_addr learned my addr 172.21.15.63:0/472932892 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0333ff3640 1 -- 172.21.15.63:0/472932892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f0418fa40 con 0x559f0418f090 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.803+0000 7f0333ff3640 1 --2- 172.21.15.63:0/472932892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f0418f470 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f032c04ccb0 tx=0x7f032c001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d21caa356b53ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0216ffd640 1 -- 172.21.15.63:0/472932892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032c012df0 con 0x559f0418f090 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0216ffd640 1 -- 172.21.15.63:0/472932892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c001310 con 0x559f0418f090 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0216ffd640 1 -- 172.21.15.63:0/472932892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032c001630 con 0x559f0418f090 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 -- 172.21.15.63:0/472932892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 msgr2=0x559f0418f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 --2- 172.21.15.63:0/472932892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f0418f470 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f032c04ccb0 tx=0x7f032c001f90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 -- 172.21.15.63:0/472932892 shutdown_connections 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 --2- 172.21.15.63:0/472932892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f0418f470 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 -- 172.21.15.63:0/472932892 >> 172.21.15.63:0/472932892 conn(0x559f0418b360 msgr2=0x559f041893c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 -- 172.21.15.63:0/472932892 shutdown_connections 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 -- 172.21.15.63:0/472932892 wait complete. 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.804+0000 7f0337b2a800 1 Processor -- start 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0337b2a800 1 -- start start 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0337b2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f041965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0337b2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f04190210 con 0x559f0418f090 2024-04-23T19:10:31.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0333ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f041965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0333ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f041965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55050/0 (socket says 172.21.15.63:55050) 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 learned_addr learned my addr 172.21.15.63:0/3083444980 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f04196b00 con 0x559f0418f090 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0333ff3640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f041965c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f032c001fd0 tx=0x7f032c04cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f02157fa640 1 -- 172.21.15.63:0/3083444980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032c02ae30 con 0x559f0418f090 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f02157fa640 1 -- 172.21.15.63:0/3083444980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c021c60 con 0x559f0418f090 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f04196d90 con 0x559f0418f090 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.805+0000 7f02157fa640 1 -- 172.21.15.63:0/3083444980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032c02a350 con 0x559f0418f090 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.806+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f04199a40 con 0x559f0418f090 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.806+0000 7f02157fa640 1 -- 172.21.15.63:0/3083444980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f032c040040 con 0x559f0418f090 2024-04-23T19:10:31.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.806+0000 7f02157fa640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f01fc036e60 0x7f01fc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.806+0000 7f02157fa640 1 -- 172.21.15.63:0/3083444980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f032c032030 con 0x559f0418f090 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.806+0000 7f03337f2640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f01fc036e60 0x7f01fc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.807+0000 7f03337f2640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f01fc036e60 0x7f01fc039320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f032408e010 tx=0x7f032408a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.807+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.807+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f041a2250 0x559f041a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.807+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041a4e60 con 0x559f041a2250 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.807+0000 7f0332ff1640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f041a2250 0x559f041a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.807+0000 7f0332ff1640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f041a2250 0x559f041a4710 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0328009050 con 0x559f041a2250 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 realm 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041a16a0 con 0x559f041a2250 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0328009050 con 0x559f041a2250 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041a5950 con 0x559f041a2250 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0328009050 con 0x559f041a2250 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041a6220 con 0x559f041a2250 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0328009050 con 0x559f041a2250 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.808+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041a65c0 con 0x559f041a2250 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.809+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0328009050 con 0x559f041a2250 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.809+0000 7f0337b2a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.809+0000 7f0337b2a800 20 searching for the correct realm 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.809+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f041a84a0 0x559f041aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.809+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.809+0000 7f0333ff3640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f041a84a0 0x559f041aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.809+0000 7f0333ff3640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f041a84a0 0x559f041aa890 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f0333ff3640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f032c046070 0x7f032c0390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032c046070 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f03337f2640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f032c046070 0x7f032c0390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f03337f2640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f032c046070 0x7f032c0390c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f03337f2640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f032409a040 0x7f032408e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.810+0000 7f0332ff1640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f032409a040 0x7f032408e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.811+0000 7f0332ff1640 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f032409a040 0x7f032408e8e0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.811+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0328009050 con 0x7f032409a040 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.811+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.811+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0328009050 con 0x559f041a2250 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.811+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032c046070 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.812+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032c046070 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.813+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.814+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.815+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.816+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.816+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032c046070 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.816+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.816+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.816+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.816+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032c046070 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.817+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a84a0 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032409a040 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x559f041a2250 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x559f041aaf40 con 0x7f032c046070 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0337b2a800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0337b2a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0337b2a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0337b2a800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0337b2a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.818+0000 7f0337b2a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041a6f40 con 0x559f041a2250 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041a77e0 con 0x559f041a2250 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041ab3c0 con 0x7f032409a040 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 zone default found 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 4 Realm: () 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:31.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 zonegroup default 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.819+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041ab870 con 0x7f032409a040 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041ad250 con 0x559f041a84a0 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041ad6a0 con 0x559f041a84a0 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 20 started sync module instance, tier type = 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.820+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041ac6e0 con 0x7f032c046070 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.821+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.821+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94141457025744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041ac340 con 0x7f032c046070 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.821+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041ae5d0 con 0x559f041a2250 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.822+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94141457025744] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.822+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.822+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94141457031456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041ae9f0 con 0x559f041a2250 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.822+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041af3b0 con 0x559f041a84a0 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.823+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.823+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94141457035008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041af830 con 0x559f041a84a0 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.824+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b0340 con 0x7f032c046070 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.824+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94141457031456] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.824+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94141457035008] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.825+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.825+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94141457036064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041ae540 con 0x7f032c046070 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.825+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b0d50 con 0x559f041a2250 2024-04-23T19:10:31.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.826+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.826+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94141457036064] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.826+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94141457041776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b1240 con 0x559f041a2250 2024-04-23T19:10:31.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.826+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b1a90 con 0x7f032409a040 2024-04-23T19:10:31.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.827+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94141457041776] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.827+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.827+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94141457043312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b1900 con 0x7f032409a040 2024-04-23T19:10:31.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.827+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b2420 con 0x7f032409a040 2024-04-23T19:10:31.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.828+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(56 notify.6 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.828+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94141457047280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b2890 con 0x7f032409a040 2024-04-23T19:10:31.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.828+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b3360 con 0x7f032409a040 2024-04-23T19:10:31.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.828+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94141457043312] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94141457047280] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94141457051216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b23e0 con 0x7f032409a040 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 20 add_watcher() i=0 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 20 add_watcher() i=1 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 20 add_watcher() i=2 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 20 add_watcher() i=3 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 20 add_watcher() i=4 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 20 add_watcher() i=5 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.829+0000 7f0337b2a800 20 add_watcher() i=6 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.830+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94141457051216] v30'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.830+0000 7f0337b2a800 20 add_watcher() i=7 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.830+0000 7f0337b2a800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.830+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041b6460 con 0x559f041a2250 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.830+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.830+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94141457044368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b38e0 con 0x559f041a2250 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.831+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94141457044368] v30'132 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.831+0000 7f01e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.831+0000 7f0337b2a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.831+0000 7f0337b2a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.831+0000 7f0337b2a800 5 note: GC not initialized 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.831+0000 7f01e8fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 20 Filter name: none 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f01627fc640 20 reqs_thread_entry: start 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041e9760 con 0x559f041a2250 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041eb6e0 con 0x559f041a2250 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) ==== 197+0+241 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 20 rados_obj.operate() r=0 bl.length=193 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.832+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.2 8:40a7511a:root::.bucket.meta.testx%3amyfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041ece00 con 0x7f032c046070 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~412 out=412b,getxattrs out=298b] v0'0 uv3 ondisk = 0) ==== 338+0+774 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 20 rados_obj.operate() r=0 bl.length=412 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.833+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041f16e0 con 0x559f041a2250 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.834+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v30'2 uv2 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.834+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [create,call version.inc in=38b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041f4ff0 con 0x559f041a2250 2024-04-23T19:10:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.835+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(66 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [create,call,writefull 0~379,setxattr (123)] v30'7 uv7 ondisk = 0) ==== 332+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.835+0000 7f0337b2a800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x7 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.835+0000 7f0337b2a800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.835+0000 7f0337b2a800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.835+0000 7f0337b2a800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.835+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94141457313072 in=740b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041f49b0 con 0x7f032409a040 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== watch-notify(notify (1) cookie 94141457043312 notify 128849018893 ret 0) ==== 770+0+0 (crc 0 0 0) 0x559f041f5390 con 0x7f032409a040 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0216ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018893 cookie 94141457043312 notifier 4555 bl.length()=728 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0216ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x7 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0216ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0216ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(67 notify.5 [notify cookie 94141457313072 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0216ffd640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f01f8003ea0 con 0x7f032409a040 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== watch-notify(notify_complete (2) cookie 94141457313072 notify 128849018893 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559f041f1110 con 0x7f032409a040 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0337b2a800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_sources= new_sources= 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0337b2a800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential sources added=[] removed=[] 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0337b2a800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_dests= new_dests= 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0337b2a800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential dests added=[] removed=[] 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.836+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041f4a50 con 0x7f032c046070 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.837+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(69 foo.buckets [call] v30'26 uv26 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.838+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041f4a50 con 0x7f032c046070 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.838+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v30'27 uv27 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0337b2a800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0337b2a800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0337b2a800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94141457319328 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041f55f0 con 0x7f032409a040 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== watch-notify(notify (1) cookie 94141457047280 notify 128849018894 ret 0) ==== 356+0+0 (crc 0 0 0) 0x559f03f1d7f0 con 0x7f032409a040 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f02167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018894 cookie 94141457047280 notifier 4555 bl.length()=314 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f02167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f02167fc640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(71 notify.6 [notify cookie 94141457319328 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f02167fc640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f0168003400 con 0x7f032409a040 2024-04-23T19:10:31.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== watch-notify(notify_complete (2) cookie 94141457319328 notify 128849018894 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f032802e9d0 con 0x7f032409a040 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.839+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041f4a50 con 0x559f041a2250 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(73 testx/myfoo [call,delete] v30'2 uv2 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0337b2a800 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0337b2a800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify cookie 94141457319328 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041f1d20 con 0x559f041a2250 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify (1) cookie 94141457041776 notify 128849018893 ret 0) ==== 197+0+0 (crc 0 0 0) 0x7f032802e9d0 con 0x559f041a2250 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0216ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018893 cookie 94141457041776 notifier 4555 bl.length()=155 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0216ffd640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f01f8003ea0 con 0x559f041a2250 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.840+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(74 notify.4 [notify cookie 94141457319328 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.841+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== watch-notify(notify_complete (2) cookie 94141457319328 notify 128849018893 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f0328053040 con 0x559f041a2250 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.841+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(75 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.841+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:76 8.2 8:40a7511a:root::.bucket.meta.testx%3amyfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041f4a50 con 0x7f032c046070 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(76 .bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call,delete] v30'4 uv4 ondisk = 0) ==== 254+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f0337b2a800 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 from cache 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f0337b2a800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [notify cookie 94141457290432 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x559f041f4a50 con 0x559f041a2250 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== watch-notify(notify (1) cookie 94141457041776 notify 128849018894 ret 0) ==== 254+0+0 (crc 0 0 0) 0x7f0328053040 con 0x559f041a2250 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f02167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018894 cookie 94141457041776 notifier 4555 bl.length()=212 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f02167fc640 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f0168003470 con 0x559f041a2250 2024-04-23T19:10:31.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(77 notify.4 [notify cookie 94141457290432 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.842+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(78 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.843+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== watch-notify(notify_complete (2) cookie 94141457290432 notify 128849018894 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f032803f480 con 0x559f041a2250 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.843+0000 7f01e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.843+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94141457044368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041f45a0 con 0x559f041a2250 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.844+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94141457044368] v30'133 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.844+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94141457025744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b1390 con 0x7f032c046070 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.845+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94141457025744] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.846+0000 7f0337b2a800 20 remove_watcher() i=0 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.846+0000 7f0337b2a800 2 removed watcher, disabling cache 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.846+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94141457031456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041acad0 con 0x559f041a2250 2024-04-23T19:10:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.847+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94141457031456] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.847+0000 7f0337b2a800 20 remove_watcher() i=1 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.847+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94141457035008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041acad0 con 0x559f041a84a0 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.848+0000 7f0333ff3640 1 -- 172.21.15.63:0/3083444980 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94141457035008] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032c029070 con 0x559f041a84a0 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.848+0000 7f0337b2a800 20 remove_watcher() i=2 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.848+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94141457036064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041aef00 con 0x7f032c046070 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.849+0000 7f03337f2640 1 -- 172.21.15.63:0/3083444980 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94141457036064] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0324097070 con 0x7f032c046070 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.849+0000 7f0337b2a800 20 remove_watcher() i=3 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.849+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94141457041776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041acad0 con 0x559f041a2250 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.850+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94141457041776] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x559f041a2250 2024-04-23T19:10:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.850+0000 7f0337b2a800 20 remove_watcher() i=4 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.850+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94141457043312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b0970 con 0x7f032409a040 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.851+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94141457043312] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.851+0000 7f0337b2a800 20 remove_watcher() i=5 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.851+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94141457047280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041acad0 con 0x7f032409a040 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.852+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94141457047280] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.852+0000 7f0337b2a800 20 remove_watcher() i=6 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.852+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94141457051216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x559f041b0970 con 0x7f032409a040 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94141457051216] v30'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f032802a040 con 0x7f032409a040 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 20 remove_watcher() i=7 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f032409a040 msgr2=0x7f032408e8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f032409a040 0x7f032408e8e0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f041a2250 msgr2=0x559f041a4710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f041a2250 0x559f041a4710 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f032c046070 msgr2=0x7f032c0390c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f032c046070 0x7f032c0390c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f041a84a0 msgr2=0x559f041aa890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f041a84a0 0x559f041aa890 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f01fc036e60 msgr2=0x7f01fc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f01fc036e60 0x7f01fc039320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f032408e010 tx=0x7f032408a000 comp rx=0 tx=0).stop 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 msgr2=0x559f041965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3083444980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0418f090 0x559f041965c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f032c001fd0 tx=0x7f032c04cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 reap_dead start 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0332ff1640 1 -- 172.21.15.63:0/3083444980 reap_dead start 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 shutdown_connections 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 >> 172.21.15.63:0/3083444980 conn(0x559f0418b360 msgr2=0x559f041889c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 shutdown_connections 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3083444980 wait complete. 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f020c036e60 msgr2=0x7f020c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f020c036e60 0x7f020c039320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f032408c9d0 tx=0x7f032408c8a0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f04181ce0 msgr2=0x559f04182100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f04181ce0 0x559f04182100 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x559f04167600 tx=0x7f032c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 shutdown_connections 2024-04-23T19:10:31.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f020c036e60 0x7f020c039320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 --2- 172.21.15.63:0/3294503214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f04181ce0 0x559f04182100 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 >> 172.21.15.63:0/3294503214 conn(0x559f04175d80 msgr2=0x559f04176180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 shutdown_connections 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.853+0000 7f0337b2a800 1 -- 172.21.15.63:0/3294503214 wait complete. 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f04170f50 msgr2=0x559f04173340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f04170f50 0x559f04173340 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f0416b430 msgr2=0x559f0416d820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f0416b430 0x559f0416d820 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0218036e60 msgr2=0x7f0218039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0218036e60 0x7f0218039320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0328000f60 tx=0x7f03280012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406c000 msgr2=0x559f04166200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406c000 0x559f04166200 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f032400a470 tx=0x7f0324003020 comp rx=0 tx=0).stop 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 shutdown_connections 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f04170f50 0x559f04173340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f0416b430 0x559f0416d820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0218036e60 0x7f0218039320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 --2- 172.21.15.63:0/82459987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f0406c000 0x559f04166200 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 >> 172.21.15.63:0/82459987 conn(0x559f04064db0 msgr2=0x559f041586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 shutdown_connections 2024-04-23T19:10:31.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.854+0000 7f0337b2a800 1 -- 172.21.15.63:0/82459987 wait complete. 2024-04-23T19:10:31.882 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-23T19:10:31.883 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-23T19:10:31.883 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-04-23T19:10:31.929 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:31.929 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61cf44800 1 Processor -- start 2024-04-23T19:10:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61cf44800 1 -- start start 2024-04-23T19:10:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61cf44800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x560a3726c490 0x560a3726e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61cf44800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560a3711b7f0 con 0x560a37268000 2024-04-23T19:10:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61cf44800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x560a3710ae20 con 0x560a3726c490 2024-04-23T19:10:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61a242640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x560a3726c490 0x560a3726e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61aa43640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x560a37268000 0x560a37268420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38846/0 (socket says 172.21.15.63:38846) 2024-04-23T19:10:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61aa43640 1 -- 172.21.15.63:0/1914832143 learned_addr learned my addr 172.21.15.63:0/1914832143 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61a242640 1 -- 172.21.15.63:0/1914832143 >> v1:172.21.15.16:6789/0 conn(0x560a37268000 legacy=0x560a37268420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.930+0000 7ff61a242640 1 -- 172.21.15.63:0/1914832143 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a37129b50 con 0x560a3726c490 2024-04-23T19:10:31.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.932+0000 7ff61a242640 1 --2- 172.21.15.63:0/1914832143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3726c490 0x560a3726e880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff60400a470 tx=0x7ff60401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7703b84dce64f2df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.932+0000 7ff619a41640 1 -- 172.21.15.63:0/1914832143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff604057070 con 0x560a3726c490 2024-04-23T19:10:31.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.932+0000 7ff619a41640 1 -- 172.21.15.63:0/1914832143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff604052570 con 0x560a3726c490 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.932+0000 7ff619a41640 1 -- 172.21.15.63:0/1914832143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff604052870 con 0x560a3726c490 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.933+0000 7ff61cf44800 1 -- 172.21.15.63:0/1914832143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3726c490 msgr2=0x560a3726e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.933+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1914832143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3726c490 0x560a3726e880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff60400a470 tx=0x7ff60401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.933+0000 7ff61cf44800 1 -- 172.21.15.63:0/1914832143 shutdown_connections 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.933+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1914832143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3726c490 0x560a3726e880 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.933+0000 7ff61cf44800 1 -- 172.21.15.63:0/1914832143 >> 172.21.15.63:0/1914832143 conn(0x560a37263600 msgr2=0x560a37265a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.933+0000 7ff61cf44800 1 -- 172.21.15.63:0/1914832143 shutdown_connections 2024-04-23T19:10:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.933+0000 7ff61cf44800 1 -- 172.21.15.63:0/1914832143 wait complete. 2024-04-23T19:10:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.945+0000 7ff61cf44800 1 Processor -- start 2024-04-23T19:10:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61cf44800 1 -- start start 2024-04-23T19:10:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61cf44800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37359270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61cf44800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a371b46f0 con 0x560a37268000 2024-04-23T19:10:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61aa43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37359270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61aa43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37359270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55064/0 (socket says 172.21.15.63:55064) 2024-04-23T19:10:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61aa43640 1 -- 172.21.15.63:0/2253273840 learned_addr learned my addr 172.21.15.63:0/2253273840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61aa43640 1 -- 172.21.15.63:0/2253273840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a37359840 con 0x560a37268000 2024-04-23T19:10:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.946+0000 7ff61aa43640 1 --2- 172.21.15.63:0/2253273840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37359270 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0017a0 tx=0x7ff60c001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415c95fbc58456d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff508e09640 1 -- 172.21.15.63:0/2253273840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff60c016070 con 0x560a37268000 2024-04-23T19:10:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff508e09640 1 -- 172.21.15.63:0/2253273840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c004520 con 0x560a37268000 2024-04-23T19:10:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff508e09640 1 -- 172.21.15.63:0/2253273840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff60c004820 con 0x560a37268000 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff61cf44800 1 -- 172.21.15.63:0/2253273840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 msgr2=0x560a37359270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff61cf44800 1 --2- 172.21.15.63:0/2253273840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37359270 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0017a0 tx=0x7ff60c001c60 comp rx=0 tx=0).stop 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff61cf44800 1 -- 172.21.15.63:0/2253273840 shutdown_connections 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff61cf44800 1 --2- 172.21.15.63:0/2253273840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37359270 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff61cf44800 1 -- 172.21.15.63:0/2253273840 >> 172.21.15.63:0/2253273840 conn(0x560a37263600 msgr2=0x560a37354b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff61cf44800 1 -- 172.21.15.63:0/2253273840 shutdown_connections 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.947+0000 7ff61cf44800 1 -- 172.21.15.63:0/2253273840 wait complete. 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61cf44800 1 Processor -- start 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61cf44800 1 -- start start 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61cf44800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37361f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61cf44800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a3726f150 con 0x560a37268000 2024-04-23T19:10:31.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61aa43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37361f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61aa43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37361f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55078/0 (socket says 172.21.15.63:55078) 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61aa43640 1 -- 172.21.15.63:0/3387875559 learned_addr learned my addr 172.21.15.63:0/3387875559 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.948+0000 7ff61aa43640 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a37362470 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.949+0000 7ff61aa43640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37361f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff60c000f80 tx=0x7ff60c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.949+0000 7ff507606640 1 -- 172.21.15.63:0/3387875559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff60c016040 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.949+0000 7ff507606640 1 -- 172.21.15.63:0/3387875559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c011530 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.949+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560a37362700 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.949+0000 7ff507606640 1 -- 172.21.15.63:0/3387875559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff60c011890 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.949+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a373653b0 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff507606640 1 -- 172.21.15.63:0/3387875559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff60c01c3f0 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff507606640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff500036e60 0x7ff500039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff507606640 1 -- 172.21.15.63:0/3387875559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff60c04bf70 con 0x560a37268000 2024-04-23T19:10:31.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff61a242640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff500036e60 0x7ff500039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3726c490 0x560a37369170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a37369700 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff61b244640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3726c490 0x560a37369170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff61a242640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff500036e60 0x7ff500039320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff604054000 tx=0x7ff60404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.950+0000 7ff61b244640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3726c490 0x560a37369170 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.951+0000 7ff61b244640 1 -- 172.21.15.63:0/3387875559 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff6140083e0 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.951+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a37366720 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.951+0000 7ff61b244640 1 -- 172.21.15.63:0/3387875559 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff6140083e0 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.951+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a3736b520 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.951+0000 7ff61b244640 1 -- 172.21.15.63:0/3387875559 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff6140083e0 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.951+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a3736a860 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.952+0000 7ff61b244640 1 -- 172.21.15.63:0/3387875559 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff6140083e0 con 0x560a3726c490 2024-04-23T19:10:31.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.952+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x560a3736c8d0 0x560a3736ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.952+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a3736f240 con 0x560a3736c8d0 2024-04-23T19:10:31.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.952+0000 7ff61aa43640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x560a3736c8d0 0x560a3736ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.952+0000 7ff61aa43640 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x560a3736c8d0 0x560a3736ecc0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.953+0000 7ff61aa43640 1 -- 172.21.15.63:0/3387875559 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff60c019050 con 0x560a3736c8d0 2024-04-23T19:10:31.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.953+0000 7ff61cf44800 1 Processor -- start 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.953+0000 7ff61cf44800 1 -- start start 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.953+0000 7ff61cf44800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37357a00 0x560a37355e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.953+0000 7ff61cf44800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a3715ae30 con 0x560a37357a00 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff61b244640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37357a00 0x560a37355e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff61b244640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37357a00 0x560a37355e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55084/0 (socket says 172.21.15.63:55084) 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff61b244640 1 -- 172.21.15.63:0/2250685995 learned_addr learned my addr 172.21.15.63:0/2250685995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff61b244640 1 -- 172.21.15.63:0/2250685995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a373563b0 con 0x560a37357a00 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff61b244640 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37357a00 0x560a37355e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x560a37363330 tx=0x7ff61400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff504e01640 1 -- 172.21.15.63:0/2250685995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff61401c070 con 0x560a37357a00 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff504e01640 1 -- 172.21.15.63:0/2250685995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61400bdc0 con 0x560a37357a00 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560a37356640 con 0x560a37357a00 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff504e01640 1 -- 172.21.15.63:0/2250685995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff614018540 con 0x560a37357a00 2024-04-23T19:10:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.954+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a37356b40 con 0x560a37357a00 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.955+0000 7ff504e01640 1 -- 172.21.15.63:0/2250685995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff61401f070 con 0x560a37357a00 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.955+0000 7ff504e01640 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4f4036e60 0x7ff4f4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.955+0000 7ff504e01640 1 -- 172.21.15.63:0/2250685995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff61404e910 con 0x560a37357a00 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.955+0000 7ff61aa43640 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4f4036e60 0x7ff4f4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61cf44800 1 Processor -- start 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61cf44800 1 -- start start 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61aa43640 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4f4036e60 0x7ff4f4039320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff60c019860 tx=0x7ff60c023f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61cf44800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a3738d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61cf44800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a37359bc0 con 0x560a3738cbf0 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61b244640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a3738d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61b244640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a3738d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55100/0 (socket says 172.21.15.63:55100) 2024-04-23T19:10:31.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61b244640 1 -- 172.21.15.63:0/263252912 learned_addr learned my addr 172.21.15.63:0/263252912 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61b244640 1 -- 172.21.15.63:0/263252912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3738d5e0 con 0x560a3738cbf0 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff61b244640 1 --2- 172.21.15.63:0/263252912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a3738d010 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff6140174e0 tx=0x7ff6140175d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4712aa371880a866 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.956+0000 7ff4fe7fc640 1 -- 172.21.15.63:0/263252912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff61404a070 con 0x560a3738cbf0 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff4fe7fc640 1 -- 172.21.15.63:0/263252912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff614029c70 con 0x560a3738cbf0 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff4fe7fc640 1 -- 172.21.15.63:0/263252912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff614026040 con 0x560a3738cbf0 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 -- 172.21.15.63:0/263252912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 msgr2=0x560a3738d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 --2- 172.21.15.63:0/263252912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a3738d010 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff6140174e0 tx=0x7ff6140175d0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 -- 172.21.15.63:0/263252912 shutdown_connections 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 --2- 172.21.15.63:0/263252912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a3738d010 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 -- 172.21.15.63:0/263252912 >> 172.21.15.63:0/263252912 conn(0x560a37388f10 msgr2=0x560a37386f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 -- 172.21.15.63:0/263252912 shutdown_connections 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 -- 172.21.15.63:0/263252912 wait complete. 2024-04-23T19:10:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 Processor -- start 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.957+0000 7ff61cf44800 1 -- start start 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61cf44800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a373941e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61cf44800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a3738dd60 con 0x560a3738cbf0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61b244640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a373941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61b244640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a373941e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55102/0 (socket says 172.21.15.63:55102) 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 learned_addr learned my addr 172.21.15.63:0/1315364054 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a37394720 con 0x560a3738cbf0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61b244640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a373941e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff6140174e0 tx=0x7ff61404ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff4fcff9640 1 -- 172.21.15.63:0/1315364054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff614026030 con 0x560a3738cbf0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff4fcff9640 1 -- 172.21.15.63:0/1315364054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff614029e20 con 0x560a3738cbf0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560a373949b0 con 0x560a3738cbf0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff4fcff9640 1 -- 172.21.15.63:0/1315364054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff61404ce60 con 0x560a3738cbf0 2024-04-23T19:10:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.958+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a37397660 con 0x560a3738cbf0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.959+0000 7ff4fcff9640 1 -- 172.21.15.63:0/1315364054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff614018730 con 0x560a3738cbf0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.959+0000 7ff4fcff9640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4e4036e60 0x7ff4e4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.959+0000 7ff4fcff9640 1 -- 172.21.15.63:0/1315364054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7ff614036c80 con 0x560a3738cbf0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.959+0000 7ff61aa43640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4e4036e60 0x7ff4e4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.960+0000 7ff61aa43640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4e4036e60 0x7ff4e4039320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff60c019a90 tx=0x7ff60c048000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.960+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.960+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3739fcb0 0x560a373a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.960+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a28c0 con 0x560a3739fcb0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.960+0000 7ff61a242640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3739fcb0 0x560a373a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.960+0000 7ff61a242640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3739fcb0 0x560a373a2170 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 realm 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a2c90 con 0x560a3739fcb0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a3560 con 0x560a3739fcb0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a3900 con 0x560a3739fcb0 2024-04-23T19:10:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.961+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a4150 con 0x560a3739fcb0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61cf44800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61cf44800 20 searching for the correct realm 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x560a373a5ec0 0x560a373a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61b244640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x560a373a5ec0 0x560a373a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61b244640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x560a373a5ec0 0x560a373a82b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61b244640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff614021070 0x7ff61402b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.962+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff614021070 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.963+0000 7ff61aa43640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff614021070 0x7ff61402b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.963+0000 7ff61aa43640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff614021070 0x7ff61402b450 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.963+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.963+0000 7ff61aa43640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff60c02e040 0x7ff60c043a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.963+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.963+0000 7ff61a242640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff60c02e040 0x7ff60c043a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.964+0000 7ff61a242640 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff60c02e040 0x7ff60c043a10 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.964+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.964+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.964+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.964+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.964+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.964+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff614021070 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.965+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.966+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.966+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.966+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.966+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.966+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.966+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff614021070 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.967+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.968+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff614021070 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.969+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.970+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.970+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.970+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.970+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.970+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.970+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff614021070 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a373a5ec0 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff60c02e040 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x560a3739fcb0 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.971+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8840 con 0x7ff614021070 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:31.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a4aa0 con 0x560a3739fcb0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a5340 con 0x560a3739fcb0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.972+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a5340 con 0x7ff60c02e040 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 20 zone default found 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 4 Realm: () 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 20 zonegroup default 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a8cc0 con 0x7ff60c02e040 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373a5040 con 0x560a373a5ec0 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.973+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61cf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373aacd0 con 0x560a373a5ec0 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61cf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61cf44800 20 started sync module instance, tier type = 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61cf44800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.974+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373a9f90 con 0x7ff614021070 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.975+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.975+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94601876249312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ab130 con 0x7ff614021070 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.975+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ab980 con 0x560a3739fcb0 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.976+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94601876249312] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.976+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.976+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94601876255952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373abda0 con 0x560a3739fcb0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.976+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ac5f0 con 0x560a373a5ec0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.977+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94601876255952] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.977+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.977+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94601876257488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ac4e0 con 0x560a373a5ec0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.977+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ad060 con 0x7ff614021070 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.978+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94601876257488] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.978+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.978+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94601876261712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ad450 con 0x7ff614021070 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.978+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ade90 con 0x560a3739fcb0 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.979+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94601876261712] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:31.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.979+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.980+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94601876265312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ae280 con 0x560a3739fcb0 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.980+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373aeca0 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.981+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.981+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94601876265312] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.981+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94601876268944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373aec60 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.981+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373abf30 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.982+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94601876268944] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.982+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.982+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94601876271248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373af790 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.982+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373b0150 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94601876271248] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94601876258608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373b0150 con 0x7ff60c02e040 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 20 add_watcher() i=0 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 20 add_watcher() i=1 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 20 add_watcher() i=2 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 20 add_watcher() i=3 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 20 add_watcher() i=4 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 20 add_watcher() i=5 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.983+0000 7ff61cf44800 20 add_watcher() i=6 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.984+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94601876258608] v30'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.984+0000 7ff61cf44800 20 add_watcher() i=7 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.984+0000 7ff61cf44800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.984+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373b34b0 con 0x560a3739fcb0 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.984+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.984+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94601876287152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373b02c0 con 0x560a3739fcb0 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.985+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94601876287152] v30'134 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.985+0000 7ff4d17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.985+0000 7ff61cf44800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.985+0000 7ff61cf44800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.985+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bbd80 con 0x560a3739fcb0 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.987+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.987+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bc3d0 con 0x7ff60c02e040 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.988+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.988+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bc830 con 0x560a373a5ec0 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.989+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.989+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bc760 con 0x560a3739fcb0 2024-04-23T19:10:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.990+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'135 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.990+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bcd30 con 0x7ff614021070 2024-04-23T19:10:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.991+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'38 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.991+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bd580 con 0x560a373a5ec0 2024-04-23T19:10:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.992+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'25 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.992+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bd580 con 0x560a3739fcb0 2024-04-23T19:10:32.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.993+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.993+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bd970 con 0x7ff60c02e040 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.994+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'25 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.994+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bd970 con 0x7ff614021070 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.995+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.995+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bd970 con 0x560a373a5ec0 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.996+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'22 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.996+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bda00 con 0x7ff614021070 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.997+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'39 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.997+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff60c02e040 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.998+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.998+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff614021070 2024-04-23T19:10:32.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.999+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:31.999+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff60c02e040 2024-04-23T19:10:32.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.000+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'31 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.000+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a3739fcb0 2024-04-23T19:10:32.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.003+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'136 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.003+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.004+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'33 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.004+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff614021070 2024-04-23T19:10:32.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.005+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'40 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.005+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.006+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.006+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a3739fcb0 2024-04-23T19:10:32.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.007+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.007+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a3739fcb0 2024-04-23T19:10:32.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.008+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'137 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.008+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.009+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'34 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.009+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff614021070 2024-04-23T19:10:32.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.010+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.010+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.011+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.011+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.012+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.012+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.013+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.013+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.014+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'26 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.014+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff60c02e040 2024-04-23T19:10:32.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.015+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.015+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.016+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.016+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff60c02e040 2024-04-23T19:10:32.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.017+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'26 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.017+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x560a373a5ec0 2024-04-23T19:10:32.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.018+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'35 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.018+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff60c02e040 2024-04-23T19:10:32.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.019+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'32 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.019+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373bdf90 con 0x7ff614021070 2024-04-23T19:10:32.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.020+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.020+0000 7ff44bfff640 20 reqs_thread_entry: start 2024-04-23T19:10:32.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:32.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 20 Filter name: none 2024-04-23T19:10:32.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff449ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:32.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-23T19:10:32.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8eb0 con 0x7ff60c02e040 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(94 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 20 rados_obj.operate() r=0 bl.length=419 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.021+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373eb390 con 0x560a3739fcb0 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.022+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(95 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 170+0+15 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.022+0000 7ff61cf44800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-23T19:10:32.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.022+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373eb390 con 0x7ff60c02e040 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v30'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61cf44800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 94601876506912 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373ebc60 con 0x560a373a5ec0 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== watch-notify(notify (1) cookie 94601876257488 notify 128849018885 ret 0) ==== 212+0+0 (crc 0 0 0) 0x560a373bc760 con 0x560a373a5ec0 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff4fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018885 cookie 94601876257488 notifier 4562 bl.length()=170 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(97 notify.2 [notify cookie 94601876506912 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff4fe7fc640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7ff4e0003d60 con 0x560a373a5ec0 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== watch-notify(notify_complete (2) cookie 94601876506912 notify 128849018885 ret 0) ==== 42+0+48 (crc 0 0 0) 0x560a37394720 con 0x560a373a5ec0 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61cf44800 10 removing email index: 2024-04-23T19:10:32.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61cf44800 10 removing user buckets index 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.023+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8920 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.024+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(99 testx$tenanteduser.buckets [delete] v30'3 uv3 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.024+0000 7ff61cf44800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.024+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [notify cookie 94601876506912 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373ebc10 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.024+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== watch-notify(notify (1) cookie 94601876265312 notify 128849018895 ret 0) ==== 217+0+0 (crc 0 0 0) 0x560a373bd2a0 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.024+0000 7ff4fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018895 cookie 94601876265312 notifier 4562 bl.length()=175 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.025+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(100 notify.4 [notify cookie 94601876506912 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.025+0000 7ff4fe7fc640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7ff4e0003d60 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.025+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(101 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.025+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== watch-notify(notify_complete (2) cookie 94601876506912 notify 128849018895 ret 0) ==== 42+0+48 (crc 0 0 0) 0x560a373bcb00 con 0x560a3739fcb0 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.025+0000 7ff61cf44800 10 removing user index: testx$tenanteduser 2024-04-23T19:10:32.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.025+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8920 con 0x7ff60c02e040 2024-04-23T19:10:32.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(102 testx$tenanteduser [call,call,delete] v30'8 uv8 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff61cf44800 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-04-23T19:10:32.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff61cf44800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-23T19:10:32.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:103 7.e 7:7759931f:::notify.3:head [notify cookie 94601876506912 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x560a373ea780 con 0x7ff614021070 2024-04-23T19:10:32.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== watch-notify(notify (1) cookie 94601876261712 notify 128849018899 ret 0) ==== 209+0+0 (crc 0 0 0) 0x560a373bd470 con 0x7ff614021070 2024-04-23T19:10:32.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff4fdffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018899 cookie 94601876261712 notifier 4562 bl.length()=167 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(103 notify.3 [notify cookie 94601876506912 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.026+0000 7ff4fdffb640 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7ff4500032c0 con 0x7ff614021070 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.027+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== watch-notify(notify_complete (2) cookie 94601876506912 notify 128849018899 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff60c056060 con 0x7ff614021070 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.027+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(104 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.028+0000 7ff4d17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.028+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94601876287152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8920 con 0x560a3739fcb0 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.029+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94601876287152] v30'138 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.029+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94601876249312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8920 con 0x7ff614021070 2024-04-23T19:10:32.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.030+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94601876249312] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.030+0000 7ff61cf44800 20 remove_watcher() i=0 2024-04-23T19:10:32.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.030+0000 7ff61cf44800 2 removed watcher, disabling cache 2024-04-23T19:10:32.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.030+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94601876255952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8920 con 0x560a3739fcb0 2024-04-23T19:10:32.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.031+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94601876255952] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.031+0000 7ff61cf44800 20 remove_watcher() i=1 2024-04-23T19:10:32.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.031+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94601876257488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ab4d0 con 0x560a373a5ec0 2024-04-23T19:10:32.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.032+0000 7ff61b244640 1 -- 172.21.15.63:0/1315364054 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94601876257488] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff614056040 con 0x560a373a5ec0 2024-04-23T19:10:32.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.032+0000 7ff61cf44800 20 remove_watcher() i=2 2024-04-23T19:10:32.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.032+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94601876261712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8920 con 0x7ff614021070 2024-04-23T19:10:32.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.033+0000 7ff61aa43640 1 -- 172.21.15.63:0/1315364054 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94601876261712] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60c048070 con 0x7ff614021070 2024-04-23T19:10:32.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.033+0000 7ff61cf44800 20 remove_watcher() i=3 2024-04-23T19:10:32.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.033+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94601876265312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ab4d0 con 0x560a3739fcb0 2024-04-23T19:10:32.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.034+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94601876265312] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x560a3739fcb0 2024-04-23T19:10:32.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.034+0000 7ff61cf44800 20 remove_watcher() i=4 2024-04-23T19:10:32.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.034+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94601876268944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373acb50 con 0x7ff60c02e040 2024-04-23T19:10:32.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.035+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94601876268944] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.035+0000 7ff61cf44800 20 remove_watcher() i=5 2024-04-23T19:10:32.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.035+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94601876271248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373e8920 con 0x7ff60c02e040 2024-04-23T19:10:32.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.036+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94601876271248] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.036+0000 7ff61cf44800 20 remove_watcher() i=6 2024-04-23T19:10:32.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.036+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94601876258608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x560a373ab4d0 con 0x7ff60c02e040 2024-04-23T19:10:32.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94601876258608] v30'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff60405a040 con 0x7ff60c02e040 2024-04-23T19:10:32.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 20 remove_watcher() i=7 2024-04-23T19:10:32.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff60c02e040 msgr2=0x7ff60c043a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff60c02e040 0x7ff60c043a10 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3739fcb0 msgr2=0x560a373a2170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3739fcb0 0x560a373a2170 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff614021070 msgr2=0x7ff61402b450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff614021070 0x7ff61402b450 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x560a373a5ec0 msgr2=0x560a373a82b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x560a373a5ec0 0x560a373a82b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4e4036e60 msgr2=0x7ff4e4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4e4036e60 0x7ff4e4039320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff60c019a90 tx=0x7ff60c048000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 msgr2=0x560a373941e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/1315364054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a3738cbf0 0x560a373941e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff6140174e0 tx=0x7ff61404ccb0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 reap_dead start 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61a242640 1 -- 172.21.15.63:0/1315364054 reap_dead start 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 shutdown_connections 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 >> 172.21.15.63:0/1315364054 conn(0x560a37388f10 msgr2=0x560a37386570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 shutdown_connections 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/1315364054 wait complete. 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4f4036e60 msgr2=0x7ff4f4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4f4036e60 0x7ff4f4039320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff60c019860 tx=0x7ff60c023f00 comp rx=0 tx=0).stop 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37357a00 msgr2=0x560a37355e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37357a00 0x560a37355e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x560a37363330 tx=0x7ff61400ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 shutdown_connections 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff4f4036e60 0x7ff4f4039320 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 --2- 172.21.15.63:0/2250685995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37357a00 0x560a37355e70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.037+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 >> 172.21.15.63:0/2250685995 conn(0x560a37371720 msgr2=0x560a37371b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 shutdown_connections 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/2250685995 wait complete. 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x560a3736c8d0 msgr2=0x560a3736ecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x560a3736c8d0 0x560a3736ecc0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3726c490 msgr2=0x560a37369170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3726c490 0x560a37369170 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff500036e60 msgr2=0x7ff500039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff500036e60 0x7ff500039320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff604054000 tx=0x7ff60404e000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 msgr2=0x560a37361f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37361f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff60c000f80 tx=0x7ff60c001250 comp rx=0 tx=0).stop 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 shutdown_connections 2024-04-23T19:10:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x560a3736c8d0 0x560a3736ecc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x560a3726c490 0x560a37369170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff500036e60 0x7ff500039320 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 --2- 172.21.15.63:0/3387875559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a37268000 0x560a37361f30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 >> 172.21.15.63:0/3387875559 conn(0x560a37263600 msgr2=0x560a37357e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 shutdown_connections 2024-04-23T19:10:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.038+0000 7ff61cf44800 1 -- 172.21.15.63:0/3387875559 wait complete. 2024-04-23T19:10:32.053 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-23T19:10:32.057 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-04-23T19:10:32.059 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:32.059 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-23T19:10:32.060 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-23T19:10:32.060 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-04-23T19:10:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f478b31f800 1 Processor -- start 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f478b31f800 1 -- start start 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f478b31f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5646d6055a60 0x5646d6055e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f478b31f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5646d5f097f0 con 0x5646d60563c0 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f478b31f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5646d5ef8e20 con 0x5646d6055a60 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f4786fc3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5646d6055a60 0x5646d6055e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f47867c2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5646d60563c0 0x5646d6056870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38862/0 (socket says 172.21.15.63:38862) 2024-04-23T19:10:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.106+0000 7f47867c2640 1 -- 172.21.15.63:0/2175199829 learned_addr learned my addr 172.21.15.63:0/2175199829 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.107+0000 7f4786fc3640 1 -- 172.21.15.63:0/2175199829 >> v1:172.21.15.16:6789/0 conn(0x5646d60563c0 legacy=0x5646d6056870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.107+0000 7f4786fc3640 1 -- 172.21.15.63:0/2175199829 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d5f17b50 con 0x5646d6055a60 2024-04-23T19:10:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f4786fc3640 1 --2- 172.21.15.63:0/2175199829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6055e80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f477800a470 tx=0x7f477801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301e3f44631f0bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f4785fc1640 1 -- 172.21.15.63:0/2175199829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4778057070 con 0x5646d6055a60 2024-04-23T19:10:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f4785fc1640 1 -- 172.21.15.63:0/2175199829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477804adc0 con 0x5646d6055a60 2024-04-23T19:10:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f4785fc1640 1 -- 172.21.15.63:0/2175199829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4778052590 con 0x5646d6055a60 2024-04-23T19:10:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f478b31f800 1 -- 172.21.15.63:0/2175199829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 msgr2=0x5646d6055e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f478b31f800 1 --2- 172.21.15.63:0/2175199829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6055e80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f477800a470 tx=0x7f477801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f478b31f800 1 -- 172.21.15.63:0/2175199829 shutdown_connections 2024-04-23T19:10:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f478b31f800 1 --2- 172.21.15.63:0/2175199829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6055e80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.109+0000 7f478b31f800 1 -- 172.21.15.63:0/2175199829 >> 172.21.15.63:0/2175199829 conn(0x5646d6051600 msgr2=0x5646d6053a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.110+0000 7f478b31f800 1 -- 172.21.15.63:0/2175199829 shutdown_connections 2024-04-23T19:10:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.110+0000 7f478b31f800 1 -- 172.21.15.63:0/2175199829 wait complete. 2024-04-23T19:10:32.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.122+0000 7f478b31f800 1 Processor -- start 2024-04-23T19:10:32.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.122+0000 7f478b31f800 1 -- start start 2024-04-23T19:10:32.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.122+0000 7f478b31f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d614a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.122+0000 7f478b31f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646d5fa26f0 con 0x5646d6055a60 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4786fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d614a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4786fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d614a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55124/0 (socket says 172.21.15.63:55124) 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4786fc3640 1 -- 172.21.15.63:0/110921130 learned_addr learned my addr 172.21.15.63:0/110921130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4786fc3640 1 -- 172.21.15.63:0/110921130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d614abd0 con 0x5646d6055a60 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4786fc3640 1 --2- 172.21.15.63:0/110921130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d614a600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f477801ced0 tx=0x7f477804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211224d5d8188b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4674e09640 1 -- 172.21.15.63:0/110921130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4778057040 con 0x5646d6055a60 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4674e09640 1 -- 172.21.15.63:0/110921130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4778063070 con 0x5646d6055a60 2024-04-23T19:10:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.123+0000 7f4674e09640 1 -- 172.21.15.63:0/110921130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4778052800 con 0x5646d6055a60 2024-04-23T19:10:32.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 -- 172.21.15.63:0/110921130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 msgr2=0x5646d614a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 --2- 172.21.15.63:0/110921130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d614a600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f477801ced0 tx=0x7f477804c000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 -- 172.21.15.63:0/110921130 shutdown_connections 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 --2- 172.21.15.63:0/110921130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d614a600 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 -- 172.21.15.63:0/110921130 >> 172.21.15.63:0/110921130 conn(0x5646d6051600 msgr2=0x5646d6145f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 -- 172.21.15.63:0/110921130 shutdown_connections 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 -- 172.21.15.63:0/110921130 wait complete. 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 Processor -- start 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 -- start start 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.124+0000 7f478b31f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646d60571c0 con 0x5646d6055a60 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4786fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4786fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55138/0 (socket says 172.21.15.63:55138) 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4786fc3640 1 -- 172.21.15.63:0/3421827628 learned_addr learned my addr 172.21.15.63:0/3421827628 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4786fc3640 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d61525c0 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4786fc3640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6152080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f477801ced0 tx=0x7f4778002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4673606640 1 -- 172.21.15.63:0/3421827628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f477806d070 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4673606640 1 -- 172.21.15.63:0/3421827628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4778052ca0 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646d6152850 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f4673606640 1 -- 172.21.15.63:0/3421827628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4778057050 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.125+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d61554b0 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.126+0000 7f4673606640 1 -- 172.21.15.63:0/3421827628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f477805d470 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f4673606640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f466c036e60 0x7f466c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f4673606640 1 -- 172.21.15.63:0/3421827628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f477808df00 con 0x5646d6055a60 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f47867c2640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f466c036e60 0x7f466c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d60563c0 0x5646d6159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6159840 con 0x5646d60563c0 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f47877c4640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d60563c0 0x5646d6159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f47867c2640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f466c036e60 0x7f466c039320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f477c000f60 tx=0x7f477c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f47877c4640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d60563c0 0x5646d6159270 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f47877c4640 1 -- 172.21.15.63:0/3421827628 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f478000a4a0 con 0x5646d60563c0 2024-04-23T19:10:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.127+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6156840 con 0x5646d60563c0 2024-04-23T19:10:32.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.128+0000 7f47877c4640 1 -- 172.21.15.63:0/3421827628 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f478000a4a0 con 0x5646d60563c0 2024-04-23T19:10:32.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.128+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d615b690 con 0x5646d60563c0 2024-04-23T19:10:32.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.128+0000 7f47877c4640 1 -- 172.21.15.63:0/3421827628 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f478000a4a0 con 0x5646d60563c0 2024-04-23T19:10:32.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.128+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d615a9d0 con 0x5646d60563c0 2024-04-23T19:10:32.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.128+0000 7f47877c4640 1 -- 172.21.15.63:0/3421827628 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f478000a4a0 con 0x5646d60563c0 2024-04-23T19:10:32.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.129+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5646d615ca40 0x5646d615ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.129+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d615f3b0 con 0x5646d615ca40 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.129+0000 7f4786fc3640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5646d615ca40 0x5646d615ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.129+0000 7f4786fc3640 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5646d615ca40 0x5646d615ee30 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.129+0000 7f4786fc3640 1 -- 172.21.15.63:0/3421827628 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f477805a040 con 0x5646d615ca40 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f478b31f800 1 Processor -- start 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f478b31f800 1 -- start start 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f478b31f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d616d7b0 0x5646d616dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f478b31f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646d5f48e30 con 0x5646d616d7b0 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f47877c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d616d7b0 0x5646d616dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f47877c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d616d7b0 0x5646d616dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55144/0 (socket says 172.21.15.63:55144) 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f47877c4640 1 -- 172.21.15.63:0/2804535751 learned_addr learned my addr 172.21.15.63:0/2804535751 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.130+0000 7f47877c4640 1 -- 172.21.15.63:0/2804535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d616e110 con 0x5646d616d7b0 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.131+0000 7f47877c4640 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d616d7b0 0x5646d616dbd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5646d6159c00 tx=0x7f47800178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.131+0000 7f4670e01640 1 -- 172.21.15.63:0/2804535751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4780013440 con 0x5646d616d7b0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.131+0000 7f4670e01640 1 -- 172.21.15.63:0/2804535751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47800135e0 con 0x5646d616d7b0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.131+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646d616eb80 con 0x5646d616d7b0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.131+0000 7f4670e01640 1 -- 172.21.15.63:0/2804535751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47800138c0 con 0x5646d616d7b0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.131+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d616f080 con 0x5646d616d7b0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.131+0000 7f4670e01640 1 -- 172.21.15.63:0/2804535751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4780011040 con 0x5646d616d7b0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f4670e01640 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4660036e60 0x7f4660039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f4670e01640 1 -- 172.21.15.63:0/2804535751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f478001d070 con 0x5646d616d7b0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f4786fc3640 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4660036e60 0x7f4660039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f478b31f800 1 Processor -- start 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f478b31f800 1 -- start start 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f4786fc3640 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4660036e60 0x7f4660039320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f477805cb80 tx=0x7f477807c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f478b31f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d617af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f478b31f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646d614ae90 con 0x5646d617ab40 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f47877c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d617af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f47877c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d617af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55152/0 (socket says 172.21.15.63:55152) 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.132+0000 7f47877c4640 1 -- 172.21.15.63:0/3071300912 learned_addr learned my addr 172.21.15.63:0/3071300912 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f47877c4640 1 -- 172.21.15.63:0/3071300912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d617b530 con 0x5646d617ab40 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f47877c4640 1 --2- 172.21.15.63:0/3071300912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d617af60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47800013e0 tx=0x7f4780012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbf54a421a122d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f466a7fc640 1 -- 172.21.15.63:0/3071300912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4780043070 con 0x5646d617ab40 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f466a7fc640 1 -- 172.21.15.63:0/3071300912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f478002ac50 con 0x5646d617ab40 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f466a7fc640 1 -- 172.21.15.63:0/3071300912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47800014f0 con 0x5646d617ab40 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f478b31f800 1 -- 172.21.15.63:0/3071300912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 msgr2=0x5646d617af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f478b31f800 1 --2- 172.21.15.63:0/3071300912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d617af60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47800013e0 tx=0x7f4780012c90 comp rx=0 tx=0).stop 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f478b31f800 1 -- 172.21.15.63:0/3071300912 shutdown_connections 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f478b31f800 1 --2- 172.21.15.63:0/3071300912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d617af60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f478b31f800 1 -- 172.21.15.63:0/3071300912 >> 172.21.15.63:0/3071300912 conn(0x5646d6176e10 msgr2=0x5646d6174e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f478b31f800 1 -- 172.21.15.63:0/3071300912 shutdown_connections 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.133+0000 7f478b31f800 1 -- 172.21.15.63:0/3071300912 wait complete. 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f478b31f800 1 Processor -- start 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f478b31f800 1 -- start start 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f478b31f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d6182040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f478b31f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646d617bcc0 con 0x5646d617ab40 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f47877c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d6182040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f47877c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d6182040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55156/0 (socket says 172.21.15.63:55156) 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 learned_addr learned my addr 172.21.15.63:0/1414007921 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d6182580 con 0x5646d617ab40 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.134+0000 7f47877c4640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d6182040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4780001da0 tx=0x7f478004cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.135+0000 7f4668ff9640 1 -- 172.21.15.63:0/1414007921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4780043040 con 0x5646d617ab40 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.135+0000 7f4668ff9640 1 -- 172.21.15.63:0/1414007921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4780030030 con 0x5646d617ab40 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.135+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646d6182870 con 0x5646d617ab40 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.135+0000 7f4668ff9640 1 -- 172.21.15.63:0/1414007921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47800123a0 con 0x5646d617ab40 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.135+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d61854c0 con 0x5646d617ab40 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.135+0000 7f4668ff9640 1 -- 172.21.15.63:0/1414007921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4780012610 con 0x5646d617ab40 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f4668ff9640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4650036e60 0x7f4650039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f4668ff9640 1 -- 172.21.15.63:0/1414007921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f478000e040 con 0x5646d617ab40 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f4786fc3640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4650036e60 0x7f4650039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d618dee0 0x5646d61903a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6190af0 con 0x5646d618dee0 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f47867c2640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d618dee0 0x5646d61903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.136+0000 7f4786fc3640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4650036e60 0x7f4650039320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f477805d040 tx=0x7f4778085000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f47867c2640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d618dee0 0x5646d61903a0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f477c009050 con 0x5646d618dee0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 20 realm 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d618d330 con 0x5646d618dee0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f477c009050 con 0x5646d618dee0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d61915e0 con 0x5646d618dee0 2024-04-23T19:10:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f477c009050 con 0x5646d618dee0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.137+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6191eb0 con 0x5646d618dee0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f477c009050 con 0x5646d618dee0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f478b31f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6192250 con 0x5646d618dee0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f477c009050 con 0x5646d618dee0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f478b31f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f478b31f800 20 searching for the correct realm 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5646d6194130 0x5646d6196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f47877c4640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5646d6194130 0x5646d6196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.138+0000 7f47877c4640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5646d6194130 0x5646d6196520 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.139+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.139+0000 7f47877c4640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4780017f80 0x7f478004e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.139+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f4780017f80 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.139+0000 7f4786fc3640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4780017f80 0x7f478004e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.139+0000 7f4786fc3640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4780017f80 0x7f478004e9b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.139+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.139+0000 7f4786fc3640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f477808c630 0x7f477808cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.140+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.140+0000 7f47867c2640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f477808c630 0x7f477808cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.140+0000 7f47867c2640 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f477808c630 0x7f477808cab0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.140+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c009050 con 0x7f477808c630 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.140+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c009050 con 0x5646d618dee0 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.141+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f4780017f80 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.142+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f4780017f80 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.143+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.144+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f4780017f80 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.145+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.146+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f4780017f80 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d6194130 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f477808c630 2024-04-23T19:10:32.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x5646d618dee0 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.147+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5646d6196bd0 con 0x7f4780017f80 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6192c00 con 0x5646d618dee0 2024-04-23T19:10:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d61934a0 con 0x5646d618dee0 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.148+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6197050 con 0x7f477808c630 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 zone default found 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 4 Realm: () 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 zonegroup default 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6197500 con 0x7f477808c630 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6198ee0 con 0x5646d6194130 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.149+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d6199380 con 0x5646d6194130 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.150+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.150+0000 7f478b31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.150+0000 7f478b31f800 20 started sync module instance, tier type = 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.150+0000 7f478b31f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.150+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d6198370 con 0x7f4780017f80 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.151+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.151+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94862239696736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d6199c90 con 0x7f4780017f80 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.151+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619a4e0 con 0x5646d618dee0 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.152+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.152+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94862239696736] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.152+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94862239703088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619aa80 con 0x5646d618dee0 2024-04-23T19:10:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.152+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61997e0 con 0x5646d6194130 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.153+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94862239703088] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.153+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.153+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94862239704272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619b520 con 0x5646d6194130 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.153+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619bee0 con 0x7f4780017f80 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.154+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.154+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94862239709744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619c360 con 0x7f4780017f80 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.154+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619ce80 con 0x5646d618dee0 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.155+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94862239704272] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.155+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.156+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94862239713616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619ce40 con 0x5646d618dee0 2024-04-23T19:10:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.156+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619d8d0 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.156+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94862239709744] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.156+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94862239713616] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.157+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.157+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94862239707248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619d8d0 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.157+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619e170 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.158+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94862239707248] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.158+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.158+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94862239716320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619d8d0 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.158+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619e790 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94862239716320] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94862239718592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619e790 con 0x7f477808c630 2024-04-23T19:10:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 20 add_watcher() i=0 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 20 add_watcher() i=1 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 20 add_watcher() i=2 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 20 add_watcher() i=3 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 20 add_watcher() i=4 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 20 add_watcher() i=5 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.159+0000 7f478b31f800 20 add_watcher() i=6 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.160+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94862239718592] v30'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.160+0000 7f478b31f800 20 add_watcher() i=7 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.160+0000 7f478b31f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.160+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d61a1ac0 con 0x5646d618dee0 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.160+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.160+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94862239720528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619ed20 con 0x5646d618dee0 2024-04-23T19:10:32.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.161+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94862239720528] v30'139 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.161+0000 7f463d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.161+0000 7f478b31f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.161+0000 7f478b31f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.162+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61a1200 con 0x5646d618dee0 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.163+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v30'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.163+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61aab80 con 0x7f477808c630 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.164+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v30'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.164+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ab030 con 0x5646d6194130 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.165+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v30'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.165+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ab030 con 0x5646d618dee0 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.166+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v30'140 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.166+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ab200 con 0x7f4780017f80 2024-04-23T19:10:32.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.167+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v30'41 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.167+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61a9360 con 0x5646d6194130 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.167+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v30'27 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.167+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61a9360 con 0x5646d618dee0 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.168+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v30'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.168+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61a9360 con 0x7f477808c630 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.169+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v30'27 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.169+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61a9360 con 0x7f4780017f80 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.170+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v30'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.170+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x5646d6194130 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.171+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v30'23 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.171+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x7f4780017f80 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.172+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v30'42 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.172+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x7f477808c630 2024-04-23T19:10:32.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.173+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v30'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.173+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x7f4780017f80 2024-04-23T19:10:32.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.174+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v30'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.174+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x7f477808c630 2024-04-23T19:10:32.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.175+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v30'33 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.176+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x5646d618dee0 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.176+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v30'141 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.176+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x5646d6194130 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.177+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v30'36 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.177+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x7f4780017f80 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.178+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v30'43 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.179+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61abb50 con 0x5646d6194130 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.179+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v30'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.179+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d618dee0 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.180+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v30'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.180+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d618dee0 2024-04-23T19:10:32.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.181+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v30'142 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.181+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d6194130 2024-04-23T19:10:32.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.182+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v30'37 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.182+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x7f4780017f80 2024-04-23T19:10:32.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.183+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v30'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.183+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d6194130 2024-04-23T19:10:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.184+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v30'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.184+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d6194130 2024-04-23T19:10:32.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.186+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v30'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.186+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d6194130 2024-04-23T19:10:32.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.186+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v30'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.186+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d6194130 2024-04-23T19:10:32.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.187+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v30'28 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.187+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x7f477808c630 2024-04-23T19:10:32.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.188+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v30'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.188+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d6194130 2024-04-23T19:10:32.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.189+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v30'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.189+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x7f477808c630 2024-04-23T19:10:32.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.191+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v30'28 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.191+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x5646d6194130 2024-04-23T19:10:32.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.192+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v30'38 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.192+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x7f477808c630 2024-04-23T19:10:32.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.193+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v30'34 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.193+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61ac140 con 0x7f4780017f80 2024-04-23T19:10:32.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.194+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v30'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.194+0000 7f45b7fff640 20 reqs_thread_entry: start 2024-04-23T19:10:32.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:32.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 20 Filter name: none 2024-04-23T19:10:32.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f45b5ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:32.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d61a89a0 con 0x7f4780017f80 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d61d8720 con 0x5646d618dee0 2024-04-23T19:10:32.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.195+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(95 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 20 get_obj_state: octx=0x7f477c00e510 obj=myfoo:four state=0x5646d61d9a08 s->prefetch_data=0 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:96 3.33s0 3:cf48e634:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5646d61da830 con 0x7f477808c630 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(96 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1_four [getxattrs out=849b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 234+0+865 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 10 manifest: total_size = 4 2024-04-23T19:10:32.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9274500934538331564 2024-04-23T19:10:32.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 20 get_obj_state: octx=0x7f477c00e510 obj=myfoo:four state=0x5646d61d9a08 s->prefetch_data=0 2024-04-23T19:10:32.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 20 prepare_atomic_modification: state is not atomic. state=0x5646d61d9a08 2024-04-23T19:10:32.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 2024-04-23T19:10:32.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.196+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 4.22 4:4491d676:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61df0c0 con 0x5646d618dee0 2024-04-23T19:10:32.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.197+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(97 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 [stat,call,call] v30'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.197+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:98 3.33s0 3:cf48e634:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61df0c0 con 0x7f477808c630 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.198+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(98 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1_four [call] v30'2 uv2 ondisk = 0) ==== 192+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.198+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 4.22 4:4491d676:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61df580 con 0x5646d618dee0 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.200+0000 7f463d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.200+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94862239720528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d61df580 con 0x5646d618dee0 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.200+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(99 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 [stat,call,call] v30'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.200+0000 7f466a7fc640 20 handle_completion(): completion ok for obj=four 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.201+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(100 data_loggenerations_metadata [watch unwatch cookie 94862239720528] v30'143 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.201+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94862239696736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619e450 con 0x7f4780017f80 2024-04-23T19:10:32.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.202+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(101 notify.0 [watch unwatch cookie 94862239696736] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.202+0000 7f478b31f800 20 remove_watcher() i=0 2024-04-23T19:10:32.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.202+0000 7f478b31f800 2 removed watcher, disabling cache 2024-04-23T19:10:32.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.202+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:102 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94862239703088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619e450 con 0x5646d618dee0 2024-04-23T19:10:32.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.203+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(102 notify.1 [watch unwatch cookie 94862239703088] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.203+0000 7f478b31f800 20 remove_watcher() i=1 2024-04-23T19:10:32.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.203+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:103 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94862239704272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619a030 con 0x5646d6194130 2024-04-23T19:10:32.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.204+0000 7f47877c4640 1 -- 172.21.15.63:0/1414007921 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(103 notify.2 [watch unwatch cookie 94862239704272] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4780029070 con 0x5646d6194130 2024-04-23T19:10:32.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.204+0000 7f478b31f800 20 remove_watcher() i=2 2024-04-23T19:10:32.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.204+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94862239709744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619e450 con 0x7f4780017f80 2024-04-23T19:10:32.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.205+0000 7f4786fc3640 1 -- 172.21.15.63:0/1414007921 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(104 notify.3 [watch unwatch cookie 94862239709744] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477805a040 con 0x7f4780017f80 2024-04-23T19:10:32.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.205+0000 7f478b31f800 20 remove_watcher() i=3 2024-04-23T19:10:32.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.205+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94862239713616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619a030 con 0x5646d618dee0 2024-04-23T19:10:32.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.206+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(105 notify.4 [watch unwatch cookie 94862239713616] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x5646d618dee0 2024-04-23T19:10:32.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.206+0000 7f478b31f800 20 remove_watcher() i=4 2024-04-23T19:10:32.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.206+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94862239707248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619ba30 con 0x7f477808c630 2024-04-23T19:10:32.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.207+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(106 notify.5 [watch unwatch cookie 94862239707248] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.207+0000 7f478b31f800 20 remove_watcher() i=5 2024-04-23T19:10:32.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.207+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:107 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94862239716320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619e450 con 0x7f477808c630 2024-04-23T19:10:32.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.208+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(107 notify.6 [watch unwatch cookie 94862239716320] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.208+0000 7f478b31f800 20 remove_watcher() i=6 2024-04-23T19:10:32.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.208+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:108 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94862239718592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5646d619a030 con 0x7f477808c630 2024-04-23T19:10:32.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(108 notify.7 [watch unwatch cookie 94862239718592] v30'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f477c02a040 con 0x7f477808c630 2024-04-23T19:10:32.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 20 remove_watcher() i=7 2024-04-23T19:10:32.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f477808c630 msgr2=0x7f477808cab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f477808c630 0x7f477808cab0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d618dee0 msgr2=0x5646d61903a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d618dee0 0x5646d61903a0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4780017f80 msgr2=0x7f478004e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4780017f80 0x7f478004e9b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5646d6194130 msgr2=0x5646d6196520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5646d6194130 0x5646d6196520 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4650036e60 msgr2=0x7f4650039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4650036e60 0x7f4650039320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f477805d040 tx=0x7f4778085000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.209+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 msgr2=0x5646d6182040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f47867c2640 1 -- 172.21.15.63:0/1414007921 reap_dead start 2024-04-23T19:10:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d6182040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4780001da0 tx=0x7f478004cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 shutdown_connections 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/1414007921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d617ab40 0x5646d6182040 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 >> 172.21.15.63:0/1414007921 conn(0x5646d6176e10 msgr2=0x5646d6174470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 shutdown_connections 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/1414007921 wait complete. 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4660036e60 msgr2=0x7f4660039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4660036e60 0x7f4660039320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f477805cb80 tx=0x7f477807c040 comp rx=0 tx=0).stop 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d616d7b0 msgr2=0x5646d616dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d616d7b0 0x5646d616dbd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5646d6159c00 tx=0x7f47800178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 shutdown_connections 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4660036e60 0x7f4660039320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/2804535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d616d7b0 0x5646d616dbd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 >> 172.21.15.63:0/2804535751 conn(0x5646d6161850 msgr2=0x5646d6161c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 shutdown_connections 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/2804535751 wait complete. 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5646d615ca40 msgr2=0x5646d615ee30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5646d615ca40 0x5646d615ee30 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d60563c0 msgr2=0x5646d6159270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d60563c0 0x5646d6159270 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f466c036e60 msgr2=0x7f466c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f466c036e60 0x7f466c039320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f477c000f60 tx=0x7f477c0012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 msgr2=0x5646d6152080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.210+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6152080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f477801ced0 tx=0x7f4778002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 shutdown_connections 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5646d615ca40 0x5646d615ee30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5646d60563c0 0x5646d6159270 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f466c036e60 0x7f466c039320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 --2- 172.21.15.63:0/3421827628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646d6055a60 0x5646d6152080 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 >> 172.21.15.63:0/3421827628 conn(0x5646d6051600 msgr2=0x5646d6144720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 shutdown_connections 2024-04-23T19:10:32.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.211+0000 7f478b31f800 1 -- 172.21.15.63:0/3421827628 wait complete. 2024-04-23T19:10:32.220 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-23T19:10:32.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-23T19:10:32.220 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-23T19:10:32.264 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:32.265 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:32.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.265+0000 7fe31752f800 1 Processor -- start 2024-04-23T19:10:32.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.265+0000 7fe31752f800 1 -- start start 2024-04-23T19:10:32.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.265+0000 7fe31752f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x562061ff2730 0x562061ff4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.265+0000 7fe31752f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562061ea07f0 con 0x562061feec20 2024-04-23T19:10:32.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.265+0000 7fe31752f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x562061e8fe20 con 0x562061ff2730 2024-04-23T19:10:32.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.266+0000 7fe312ac4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x562061ff2730 0x562061ff4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.266+0000 7fe3132c5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x562061feec20 0x562061ff1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38878/0 (socket says 172.21.15.63:38878) 2024-04-23T19:10:32.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.266+0000 7fe3132c5640 1 -- 172.21.15.63:0/195374099 learned_addr learned my addr 172.21.15.63:0/195374099 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:32.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.266+0000 7fe3122c3640 1 -- 172.21.15.63:0/195374099 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3464253789 0 0) 0x562061ea07f0 con 0x562061feec20 2024-04-23T19:10:32.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.266+0000 7fe3122c3640 1 -- 172.21.15.63:0/195374099 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe300001e10 con 0x562061feec20 2024-04-23T19:10:32.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.266+0000 7fe312ac4640 1 -- 172.21.15.63:0/195374099 >> v1:172.21.15.16:6789/0 conn(0x562061feec20 legacy=0x562061ff1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.266+0000 7fe312ac4640 1 -- 172.21.15.63:0/195374099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562061eaeb50 con 0x562061ff2730 2024-04-23T19:10:32.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.268+0000 7fe312ac4640 1 --2- 172.21.15.63:0/195374099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061ff2730 0x562061ff4b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00a470 tx=0x7fe2fc01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86749ed68b0002f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.268+0000 7fe3122c3640 1 -- 172.21.15.63:0/195374099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2fc057070 con 0x562061ff2730 2024-04-23T19:10:32.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.268+0000 7fe3122c3640 1 -- 172.21.15.63:0/195374099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2fc052570 con 0x562061ff2730 2024-04-23T19:10:32.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.268+0000 7fe3122c3640 1 -- 172.21.15.63:0/195374099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2fc052850 con 0x562061ff2730 2024-04-23T19:10:32.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.269+0000 7fe31752f800 1 -- 172.21.15.63:0/195374099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061ff2730 msgr2=0x562061ff4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.269+0000 7fe31752f800 1 --2- 172.21.15.63:0/195374099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061ff2730 0x562061ff4b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00a470 tx=0x7fe2fc01cdb0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.269+0000 7fe31752f800 1 -- 172.21.15.63:0/195374099 shutdown_connections 2024-04-23T19:10:32.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.269+0000 7fe31752f800 1 --2- 172.21.15.63:0/195374099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061ff2730 0x562061ff4b20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.269+0000 7fe31752f800 1 -- 172.21.15.63:0/195374099 >> 172.21.15.63:0/195374099 conn(0x562061fe8630 msgr2=0x562061feaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.269+0000 7fe31752f800 1 -- 172.21.15.63:0/195374099 shutdown_connections 2024-04-23T19:10:32.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.269+0000 7fe31752f800 1 -- 172.21.15.63:0/195374099 wait complete. 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.281+0000 7fe31752f800 1 Processor -- start 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.281+0000 7fe31752f800 1 -- start start 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.281+0000 7fe31752f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x5620620df120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.281+0000 7fe31752f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562061f396f0 con 0x562061feec20 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe3132c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x5620620df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe3132c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x5620620df120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55166/0 (socket says 172.21.15.63:55166) 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe3132c5640 1 -- 172.21.15.63:0/2722695732 learned_addr learned my addr 172.21.15.63:0/2722695732 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe3132c5640 1 -- 172.21.15.63:0/2722695732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620620df6f0 con 0x562061feec20 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe3132c5640 1 --2- 172.21.15.63:0/2722695732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x5620620df120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe304006f60 tx=0x7fe304006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b64e2d32b3d4065 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe20160a640 1 -- 172.21.15.63:0/2722695732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe304004520 con 0x562061feec20 2024-04-23T19:10:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe20160a640 1 -- 172.21.15.63:0/2722695732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3040046e0 con 0x562061feec20 2024-04-23T19:10:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.282+0000 7fe20160a640 1 -- 172.21.15.63:0/2722695732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe304004a80 con 0x562061feec20 2024-04-23T19:10:32.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 -- 172.21.15.63:0/2722695732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 msgr2=0x5620620df120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 --2- 172.21.15.63:0/2722695732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x5620620df120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe304006f60 tx=0x7fe304006f90 comp rx=0 tx=0).stop 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 -- 172.21.15.63:0/2722695732 shutdown_connections 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 --2- 172.21.15.63:0/2722695732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x5620620df120 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 -- 172.21.15.63:0/2722695732 >> 172.21.15.63:0/2722695732 conn(0x562061fe8630 msgr2=0x5620620daa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 -- 172.21.15.63:0/2722695732 shutdown_connections 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 -- 172.21.15.63:0/2722695732 wait complete. 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.283+0000 7fe31752f800 1 Processor -- start 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.284+0000 7fe31752f800 1 -- start start 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.284+0000 7fe31752f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x562061fee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.284+0000 7fe31752f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562061ff7370 con 0x562061feec20 2024-04-23T19:10:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.284+0000 7fe3132c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x562061fee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.284+0000 7fe3132c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x562061fee520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55168/0 (socket says 172.21.15.63:55168) 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.284+0000 7fe3132c5640 1 -- 172.21.15.63:0/1133151067 learned_addr learned my addr 172.21.15.63:0/1133151067 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.284+0000 7fe3132c5640 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562061fecc10 con 0x562061feec20 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.285+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x562061fee520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5620620dfeb0 tx=0x7fe304000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.285+0000 7fe1ffe07640 1 -- 172.21.15.63:0/1133151067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe304022030 con 0x562061feec20 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.285+0000 7fe1ffe07640 1 -- 172.21.15.63:0/1133151067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe304019040 con 0x562061feec20 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.285+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562061fecea0 con 0x562061feec20 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.285+0000 7fe1ffe07640 1 -- 172.21.15.63:0/1133151067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe304014b80 con 0x562061feec20 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.285+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562061fed3c0 con 0x562061feec20 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe1ffe07640 1 -- 172.21.15.63:0/1133151067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe304025070 con 0x562061feec20 2024-04-23T19:10:32.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe1ffe07640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1f8036e60 0x7fe1f8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe1ffe07640 1 -- 172.21.15.63:0/1133151067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe304023070 con 0x562061feec20 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe312ac4640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1f8036e60 0x7fe1f8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562061ff2730 0x5620620f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620620f0650 con 0x562061ff2730 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe313ac6640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562061ff2730 0x5620620f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe312ac4640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1f8036e60 0x7fe1f8039320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc054000 tx=0x7fe2fc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.286+0000 7fe313ac6640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562061ff2730 0x5620620f00c0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.287+0000 7fe313ac6640 1 -- 172.21.15.63:0/1133151067 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe30c0083e0 con 0x562061ff2730 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.287+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562061fed730 con 0x562061ff2730 2024-04-23T19:10:32.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.287+0000 7fe313ac6640 1 -- 172.21.15.63:0/1133151067 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe30c0083e0 con 0x562061ff2730 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.287+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620620f2470 con 0x562061ff2730 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.287+0000 7fe313ac6640 1 -- 172.21.15.63:0/1133151067 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe30c0083e0 con 0x562061ff2730 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.288+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620620f17b0 con 0x562061ff2730 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.288+0000 7fe313ac6640 1 -- 172.21.15.63:0/1133151067 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe30c0083e0 con 0x562061ff2730 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.288+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620620f3820 0x5620620f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.288+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620620f6190 con 0x5620620f3820 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.288+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620620f3820 0x5620620f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.288+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620620f3820 0x5620620f5c10 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe3132c5640 1 -- 172.21.15.63:0/1133151067 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe30400f030 con 0x5620620f3820 2024-04-23T19:10:32.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe31752f800 1 Processor -- start 2024-04-23T19:10:32.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe31752f800 1 -- start start 2024-04-23T19:10:32.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe31752f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621045d0 0x5620621049f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe31752f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562061edfe30 con 0x5620621045d0 2024-04-23T19:10:32.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe313ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621045d0 0x5620621049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe313ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621045d0 0x5620621049f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55170/0 (socket says 172.21.15.63:55170) 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.289+0000 7fe313ac6640 1 -- 172.21.15.63:0/3360747573 learned_addr learned my addr 172.21.15.63:0/3360747573 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.290+0000 7fe313ac6640 1 -- 172.21.15.63:0/3360747573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562062104f30 con 0x5620621045d0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.290+0000 7fe313ac6640 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621045d0 0x5620621049f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x562061fedd30 tx=0x7fe30c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.290+0000 7fe1fd602640 1 -- 172.21.15.63:0/3360747573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe30c01c070 con 0x5620621045d0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.290+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5620621059a0 con 0x5620621045d0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.290+0000 7fe1fd602640 1 -- 172.21.15.63:0/3360747573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c018410 con 0x5620621045d0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.290+0000 7fe1fd602640 1 -- 172.21.15.63:0/3360747573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe30c018730 con 0x5620621045d0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.290+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562062105ea0 con 0x5620621045d0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe1fd602640 1 -- 172.21.15.63:0/3360747573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe30c01f070 con 0x5620621045d0 2024-04-23T19:10:32.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe1fd602640 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1ec036e60 0x7fe1ec039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe1fd602640 1 -- 172.21.15.63:0/3360747573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe30c04e940 con 0x5620621045d0 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe3132c5640 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1ec036e60 0x7fe1ec039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe31752f800 1 Processor -- start 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe31752f800 1 -- start start 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe3132c5640 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1ec036e60 0x7fe1ec039320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe304027fd0 tx=0x7fe30401d770 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.291+0000 7fe31752f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062111e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe31752f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620620dfa70 con 0x5620621119e0 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe313ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062111e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe313ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062111e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55172/0 (socket says 172.21.15.63:55172) 2024-04-23T19:10:32.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe313ac6640 1 -- 172.21.15.63:0/384848440 learned_addr learned my addr 172.21.15.63:0/384848440 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe313ac6640 1 -- 172.21.15.63:0/384848440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620621123d0 con 0x5620621119e0 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe313ac6640 1 --2- 172.21.15.63:0/384848440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062111e00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe30c04ccc0 tx=0x7fe30c029350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3890d187f515e6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe1f6ffd640 1 -- 172.21.15.63:0/384848440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe30c01fc30 con 0x5620621119e0 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe1f6ffd640 1 -- 172.21.15.63:0/384848440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c01fdd0 con 0x5620621119e0 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.292+0000 7fe1f6ffd640 1 -- 172.21.15.63:0/384848440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe30c046bb0 con 0x5620621119e0 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 -- 172.21.15.63:0/384848440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 msgr2=0x562062111e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 --2- 172.21.15.63:0/384848440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062111e00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe30c04ccc0 tx=0x7fe30c029350 comp rx=0 tx=0).stop 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 -- 172.21.15.63:0/384848440 shutdown_connections 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 --2- 172.21.15.63:0/384848440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062111e00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 -- 172.21.15.63:0/384848440 >> 172.21.15.63:0/384848440 conn(0x56206210dcf0 msgr2=0x56206210bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 -- 172.21.15.63:0/384848440 shutdown_connections 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 -- 172.21.15.63:0/384848440 wait complete. 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 Processor -- start 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 -- start start 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062118ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe31752f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562062112b20 con 0x5620621119e0 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe313ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062118ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe313ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062118ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55178/0 (socket says 172.21.15.63:55178) 2024-04-23T19:10:32.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.293+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 learned_addr learned my addr 172.21.15.63:0/1555700091 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.294+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562062119430 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.294+0000 7fe313ac6640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062118ef0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe30c029390 tx=0x7fe30c04cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.294+0000 7fe1f57fa640 1 -- 172.21.15.63:0/1555700091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe30c039070 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.294+0000 7fe1f57fa640 1 -- 172.21.15.63:0/1555700091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c018bb0 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.294+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5620621196c0 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.294+0000 7fe1f57fa640 1 -- 172.21.15.63:0/1555700091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe30c04e110 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.294+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206211c370 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe1f57fa640 1 -- 172.21.15.63:0/1555700091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe30c04e2b0 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe1f57fa640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1dc036e60 0x7fe1dc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe1f57fa640 1 -- 172.21.15.63:0/1555700091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fe30c031070 con 0x5620621119e0 2024-04-23T19:10:32.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1dc036e60 0x7fe1dc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562062124a00 0x562062126ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062127610 con 0x562062124a00 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe312ac4640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562062124a00 0x562062126ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.295+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1dc036e60 0x7fe1dc039320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe304012fd0 tx=0x7fe304030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe312ac4640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562062124a00 0x562062126ec0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 20 realm 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620621279e0 con 0x562062124a00 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620621282b0 con 0x562062124a00 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.296+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062128650 con 0x562062124a00 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062128a70 con 0x562062124a00 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe31752f800 20 searching for the correct realm 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56206212aa70 0x56206212cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x56206212d590 con 0x56206212aa70 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe313ac6640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56206212aa70 0x56206212cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.297+0000 7fe313ac6640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56206212aa70 0x56206212cec0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe313ac6640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe30c021070 0x7fe30c021450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe30c021070 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe30c021070 0x7fe30c021450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe30c021070 0x7fe30c021450 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe3132c5640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe304027030 0x7fe3040408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.298+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.299+0000 7fe312ac4640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe304027030 0x7fe3040408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.299+0000 7fe312ac4640 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe304027030 0x7fe3040408e0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.300+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe30c021070 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.301+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe30c021070 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.302+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.303+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.304+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.304+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.304+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.304+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.304+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.304+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe30c021070 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.305+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.306+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.306+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe30c021070 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.306+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.306+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x56206212aa70 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.306+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.306+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe304027030 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x562062124a00 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fe30c04ec60 con 0x7fe30c021070 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:32.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620621293e0 con 0x562062124a00 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.307+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620621297d0 con 0x562062124a00 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620621297d0 con 0x7fe304027030 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 zone default found 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 4 Realm: () 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 zonegroup default 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062129bf0 con 0x7fe304027030 2024-04-23T19:10:32.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.308+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206212a010 con 0x56206212aa70 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe31752f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206212f440 con 0x56206212aa70 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe31752f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe31752f800 20 started sync module instance, tier type = 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe31752f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.309+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56206212e8e0 con 0x7fe30c021070 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.310+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.310+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94697084349696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56206212f830 con 0x7fe30c021070 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.310+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062130080 con 0x562062124a00 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.311+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94697084349696] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.311+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.311+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94697084353488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5620621304a0 con 0x562062124a00 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.311+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062130e60 con 0x56206212aa70 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.312+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.312+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94697084357040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062131280 con 0x56206212aa70 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.312+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062131d80 con 0x7fe30c021070 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.312+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94697084353488] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.313+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94697084357040] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.313+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.313+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94697084358096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x56206212fff0 con 0x7fe30c021070 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.313+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062132740 con 0x562062124a00 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.314+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.314+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94697084363408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062132b60 con 0x562062124a00 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.314+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5620621335e0 con 0x7fe304027030 2024-04-23T19:10:32.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.315+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94697084358096] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.315+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.315+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94697084367088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5620621335e0 con 0x7fe304027030 2024-04-23T19:10:32.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.315+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062134020 con 0x7fe304027030 2024-04-23T19:10:32.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.315+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94697084363408] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.316+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94697084367088] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.317+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.317+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94697084360816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062133b70 con 0x7fe304027030 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.317+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062134530 con 0x7fe304027030 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94697084360816] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94697084364464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062134530 con 0x7fe304027030 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 20 add_watcher() i=0 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 20 add_watcher() i=1 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 20 add_watcher() i=2 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 20 add_watcher() i=3 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 20 add_watcher() i=4 2024-04-23T19:10:32.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 20 add_watcher() i=5 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe31752f800 20 add_watcher() i=6 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.318+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94697084364464] v30'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.319+0000 7fe31752f800 20 add_watcher() i=7 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.319+0000 7fe31752f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.319+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062137830 con 0x562062124a00 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.319+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.319+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94697084371440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062134ac0 con 0x562062124a00 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.320+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94697084371440] v30'144 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.320+0000 7fe1c9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.320+0000 7fe31752f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.320+0000 7fe31752f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.320+0000 7fe31752f800 5 note: GC not initialized 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe1c8fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe31752f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe31752f800 20 Filter name: none 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe1427fc640 20 reqs_thread_entry: start 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206216acc0 con 0x7fe30c021070 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.321+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206216b550 con 0x562062124a00 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206216a060 con 0x7fe30c021070 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206216cbd0 con 0x562062124a00 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 4.20 4:061ef416:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206216c630 con 0x562062124a00 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 4.3a 4:5ce4e6b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062171520 con 0x56206212aa70 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.322+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 4.11 4:88bce1a7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062173d90 con 0x7fe304027030 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 4.3d 4:bd56d65b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062173d90 con 0x56206212aa70 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 4.7 4:e0c3751e:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062171520 con 0x7fe304027030 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 4.25 4:a745f109:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206217bb30 con 0x56206212aa70 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 4.22 4:4491d676:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206217e5e0 con 0x562062124a00 2024-04-23T19:10:32.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 4.3e 4:7f79def2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062181430 con 0x7fe30c021070 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:74 4.1d 4:b89cda5a:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5620621842d0 con 0x56206212aa70 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 4.16 4:681bb7d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x562062187190 con 0x56206212aa70 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:76 4.d 4:b3b67914:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x56206218a050 con 0x56206212aa70 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(69 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.5 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(73 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(68 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(75 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(70 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(74 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.8 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(76 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.323+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(72 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1.6 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.324+0000 7fe1c9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.324+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94697084371440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x562062124a00 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.325+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94697084371440] v30'145 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.325+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94697084349696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x7fe30c021070 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.326+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94697084349696] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.326+0000 7fe31752f800 20 remove_watcher() i=0 2024-04-23T19:10:32.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.326+0000 7fe31752f800 2 removed watcher, disabling cache 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.326+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94697084353488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x562062124a00 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.327+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94697084353488] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.327+0000 7fe31752f800 20 remove_watcher() i=1 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.327+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94697084357040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x56206212aa70 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.328+0000 7fe313ac6640 1 -- 172.21.15.63:0/1555700091 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94697084357040] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30c03b030 con 0x56206212aa70 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.328+0000 7fe31752f800 20 remove_watcher() i=2 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.328+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94697084358096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x7fe30c021070 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.329+0000 7fe3132c5640 1 -- 172.21.15.63:0/1555700091 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94697084358096] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe30400f030 con 0x7fe30c021070 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.329+0000 7fe31752f800 20 remove_watcher() i=3 2024-04-23T19:10:32.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.329+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94697084363408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x562062124a00 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.330+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94697084363408] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x562062124a00 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.330+0000 7fe31752f800 20 remove_watcher() i=4 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.330+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94697084367088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x7fe304027030 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.331+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94697084367088] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.331+0000 7fe31752f800 20 remove_watcher() i=5 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.331+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94697084360816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x7fe304027030 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.332+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94697084360816] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.332+0000 7fe31752f800 20 remove_watcher() i=6 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.332+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94697084364464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x562062153200 con 0x7fe304027030 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94697084364464] v30'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe2fc05a040 con 0x7fe304027030 2024-04-23T19:10:32.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 20 remove_watcher() i=7 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe304027030 msgr2=0x7fe3040408e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe304027030 0x7fe3040408e0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562062124a00 msgr2=0x562062126ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562062124a00 0x562062126ec0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe30c021070 msgr2=0x7fe30c021450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe30c021070 0x7fe30c021450 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56206212aa70 msgr2=0x56206212cec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56206212aa70 0x56206212cec0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1dc036e60 msgr2=0x7fe1dc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1dc036e60 0x7fe1dc039320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe304012fd0 tx=0x7fe304030000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 msgr2=0x562062118ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 --2- 172.21.15.63:0/1555700091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621119e0 0x562062118ef0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe30c029390 tx=0x7fe30c04cc20 comp rx=0 tx=0).stop 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 reap_dead start 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe312ac4640 1 -- 172.21.15.63:0/1555700091 reap_dead start 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 shutdown_connections 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 >> 172.21.15.63:0/1555700091 conn(0x56206210dcf0 msgr2=0x56206210b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 shutdown_connections 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.334+0000 7fe31752f800 1 -- 172.21.15.63:0/1555700091 wait complete. 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1ec036e60 msgr2=0x7fe1ec039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1ec036e60 0x7fe1ec039320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe304027fd0 tx=0x7fe30401d770 comp rx=0 tx=0).stop 2024-04-23T19:10:32.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621045d0 msgr2=0x5620621049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621045d0 0x5620621049f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x562061fedd30 tx=0x7fe30c00ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 shutdown_connections 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1ec036e60 0x7fe1ec039320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/3360747573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620621045d0 0x5620621049f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 >> 172.21.15.63:0/3360747573 conn(0x5620620f8670 msgr2=0x5620620f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 shutdown_connections 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/3360747573 wait complete. 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620620f3820 msgr2=0x5620620f5c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620620f3820 0x5620620f5c10 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562061ff2730 msgr2=0x5620620f00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562061ff2730 0x5620620f00c0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1f8036e60 msgr2=0x7fe1f8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1f8036e60 0x7fe1f8039320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc054000 tx=0x7fe2fc04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 msgr2=0x562061fee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x562061fee520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5620620dfeb0 tx=0x7fe304000f40 comp rx=0 tx=0).stop 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 shutdown_connections 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620620f3820 0x5620620f5c10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x562061ff2730 0x5620620f00c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe1f8036e60 0x7fe1f8039320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 --2- 172.21.15.63:0/1133151067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562061feec20 0x562061fee520 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 >> 172.21.15.63:0/1133151067 conn(0x562061fe8630 msgr2=0x5620620d9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 shutdown_connections 2024-04-23T19:10:32.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.335+0000 7fe31752f800 1 -- 172.21.15.63:0/1133151067 wait complete. 2024-04-23T19:10:32.364 INFO:teuthology.orchestra.run.smithi063.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"8815d3af-8342-460f-8220-31ec38570cc3","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","marker":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-23T19:10:31.835113Z","creation_time":"2024-04-23T19:10:29.008577Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":0,"size_actual":0,"size_utilized":0,"size_kb":0,"size_kb_actual":0,"size_kb_utilized":0,"num_objects":0}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-04-23T19:10:32.364 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:10:31.835113Z', 'creation_time': '2024-04-23T19:10:29.008577Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-23T19:10:32.364 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-04-23T19:10:32.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-04-23T19:10:32.365 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-04-23T19:10:32.446 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:32.446 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:32.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.446+0000 7fd59a41f800 1 Processor -- start 2024-04-23T19:10:32.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.446+0000 7fd59a41f800 1 -- start start 2024-04-23T19:10:32.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd59a41f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5590ce23e490 0x5590ce240880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd59a41f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5590ce0ed7f0 con 0x5590ce23a000 2024-04-23T19:10:32.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd59a41f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5590ce0dce20 con 0x5590ce23e490 2024-04-23T19:10:32.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd59771d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5590ce23e490 0x5590ce240880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd597f1e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5590ce23a000 0x5590ce23a420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38888/0 (socket says 172.21.15.63:38888) 2024-04-23T19:10:32.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd59771d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5590ce23e490 0x5590ce240880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55186/0 (socket says 172.21.15.63:55186) 2024-04-23T19:10:32.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd597f1e640 1 -- 172.21.15.63:0/3800425775 learned_addr learned my addr 172.21.15.63:0/3800425775 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:32.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd59771d640 1 -- 172.21.15.63:0/3800425775 >> v1:172.21.15.16:6789/0 conn(0x5590ce23a000 legacy=0x5590ce23a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.447+0000 7fd59771d640 1 -- 172.21.15.63:0/3800425775 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ce0fbb50 con 0x5590ce23e490 2024-04-23T19:10:32.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.449+0000 7fd59771d640 1 --2- 172.21.15.63:0/3800425775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23e490 0x5590ce240880 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00a470 tx=0x7fd58c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a8e6b91ae71b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.449+0000 7fd596f1c640 1 -- 172.21.15.63:0/3800425775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd58c057070 con 0x5590ce23e490 2024-04-23T19:10:32.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd596f1c640 1 -- 172.21.15.63:0/3800425775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58c052570 con 0x5590ce23e490 2024-04-23T19:10:32.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd596f1c640 1 -- 172.21.15.63:0/3800425775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd58c052870 con 0x5590ce23e490 2024-04-23T19:10:32.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd59a41f800 1 -- 172.21.15.63:0/3800425775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23e490 msgr2=0x5590ce240880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd59a41f800 1 --2- 172.21.15.63:0/3800425775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23e490 0x5590ce240880 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00a470 tx=0x7fd58c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:32.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd59a41f800 1 -- 172.21.15.63:0/3800425775 shutdown_connections 2024-04-23T19:10:32.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd59a41f800 1 --2- 172.21.15.63:0/3800425775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23e490 0x5590ce240880 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd59a41f800 1 -- 172.21.15.63:0/3800425775 >> 172.21.15.63:0/3800425775 conn(0x5590ce235620 msgr2=0x5590ce237a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd59a41f800 1 -- 172.21.15.63:0/3800425775 shutdown_connections 2024-04-23T19:10:32.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.450+0000 7fd59a41f800 1 -- 172.21.15.63:0/3800425775 wait complete. 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.463+0000 7fd59a41f800 1 Processor -- start 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.463+0000 7fd59a41f800 1 -- start start 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.463+0000 7fd59a41f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce1ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.463+0000 7fd59a41f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5590ce1866e0 con 0x5590ce23a000 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.463+0000 7fd597f1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce1ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.463+0000 7fd597f1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce1ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55188/0 (socket says 172.21.15.63:55188) 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.463+0000 7fd597f1e640 1 -- 172.21.15.63:0/4000985421 learned_addr learned my addr 172.21.15.63:0/4000985421 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.464+0000 7fd597f1e640 1 -- 172.21.15.63:0/4000985421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ce32f040 con 0x5590ce23a000 2024-04-23T19:10:32.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.464+0000 7fd597f1e640 1 --2- 172.21.15.63:0/4000985421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce1ae0d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd588001d80 tx=0x7fd588001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ce27fd9611b49a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.464+0000 7fd485e0b640 1 -- 172.21.15.63:0/4000985421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd588013030 con 0x5590ce23a000 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.464+0000 7fd485e0b640 1 -- 172.21.15.63:0/4000985421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58800e470 con 0x5590ce23a000 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.464+0000 7fd485e0b640 1 -- 172.21.15.63:0/4000985421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd58800e810 con 0x5590ce23a000 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 -- 172.21.15.63:0/4000985421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 msgr2=0x5590ce1ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 --2- 172.21.15.63:0/4000985421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce1ae0d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd588001d80 tx=0x7fd588001e70 comp rx=0 tx=0).stop 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 -- 172.21.15.63:0/4000985421 shutdown_connections 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 --2- 172.21.15.63:0/4000985421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce1ae0d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 -- 172.21.15.63:0/4000985421 >> 172.21.15.63:0/4000985421 conn(0x5590ce235620 msgr2=0x5590ce326b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 -- 172.21.15.63:0/4000985421 shutdown_connections 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 -- 172.21.15.63:0/4000985421 wait complete. 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 Processor -- start 2024-04-23T19:10:32.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 -- start start 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce333f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.465+0000 7fd59a41f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5590ce241150 con 0x5590ce23a000 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd597f1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce333f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd597f1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce333f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55194/0 (socket says 172.21.15.63:55194) 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd597f1e640 1 -- 172.21.15.63:0/236039330 learned_addr learned my addr 172.21.15.63:0/236039330 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd597f1e640 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ce334450 con 0x5590ce23a000 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd597f1e640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce333f10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd588004000 tx=0x7fd588000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd47ffff640 1 -- 172.21.15.63:0/236039330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd588027070 con 0x5590ce23a000 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd47ffff640 1 -- 172.21.15.63:0/236039330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd588001400 con 0x5590ce23a000 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd47ffff640 1 -- 172.21.15.63:0/236039330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd588019a70 con 0x5590ce23a000 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5590ce3346e0 con 0x5590ce23a000 2024-04-23T19:10:32.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.466+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ce337390 con 0x5590ce23a000 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.467+0000 7fd47ffff640 1 -- 172.21.15.63:0/236039330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd58802a070 con 0x5590ce23a000 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.467+0000 7fd47ffff640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd480036e60 0x7fd480039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.467+0000 7fd47ffff640 1 -- 172.21.15.63:0/236039330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd58801f070 con 0x5590ce23a000 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.467+0000 7fd59771d640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd480036e60 0x7fd480039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.467+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce23e490 0x5590ce33b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.467+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce33b6e0 con 0x5590ce23e490 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.468+0000 7fd59871f640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce23e490 0x5590ce33b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.468+0000 7fd59771d640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd480036e60 0x7fd480039320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd58c054000 tx=0x7fd58c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.468+0000 7fd59871f640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce23e490 0x5590ce33b150 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.468+0000 7fd59871f640 1 -- 172.21.15.63:0/236039330 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd59000a4a0 con 0x5590ce23e490 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.468+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce338720 con 0x5590ce23e490 2024-04-23T19:10:32.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.468+0000 7fd59871f640 1 -- 172.21.15.63:0/236039330 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd59000a4a0 con 0x5590ce23e490 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.468+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce33d520 con 0x5590ce23e490 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.469+0000 7fd59871f640 1 -- 172.21.15.63:0/236039330 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd59000a4a0 con 0x5590ce23e490 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.469+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce33c860 con 0x5590ce23e490 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.469+0000 7fd59871f640 1 -- 172.21.15.63:0/236039330 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd59000a4a0 con 0x5590ce23e490 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.469+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5590ce33e8d0 0x5590ce340cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.469+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce341240 con 0x5590ce33e8d0 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.469+0000 7fd597f1e640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5590ce33e8d0 0x5590ce340cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.470+0000 7fd597f1e640 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5590ce33e8d0 0x5590ce340cc0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.470+0000 7fd597f1e640 1 -- 172.21.15.63:0/236039330 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd588029030 con 0x5590ce33e8d0 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.470+0000 7fd59a41f800 1 Processor -- start 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.470+0000 7fd59a41f800 1 -- start start 2024-04-23T19:10:32.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.470+0000 7fd59a41f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce34f5f0 0x5590ce34fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.470+0000 7fd59a41f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5590ce12ce30 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd59871f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce34f5f0 0x5590ce34fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd59871f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce34f5f0 0x5590ce34fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55208/0 (socket says 172.21.15.63:55208) 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd59871f640 1 -- 172.21.15.63:0/70329693 learned_addr learned my addr 172.21.15.63:0/70329693 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd59871f640 1 -- 172.21.15.63:0/70329693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ce34ff50 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd59871f640 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce34f5f0 0x5590ce34fa10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5590ce33bae0 tx=0x7fd5900178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd47d7fa640 1 -- 172.21.15.63:0/70329693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd590013440 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd47d7fa640 1 -- 172.21.15.63:0/70329693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5900135e0 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5590ce3509c0 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd47d7fa640 1 -- 172.21.15.63:0/70329693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5900138c0 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.471+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ce350ec0 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.472+0000 7fd47d7fa640 1 -- 172.21.15.63:0/70329693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd590011040 con 0x5590ce34f5f0 2024-04-23T19:10:32.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.472+0000 7fd47d7fa640 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd470036f00 0x7fd4700393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.472+0000 7fd47d7fa640 1 -- 172.21.15.63:0/70329693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd59001d070 con 0x5590ce34f5f0 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.472+0000 7fd597f1e640 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd470036f00 0x7fd4700393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59a41f800 1 Processor -- start 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59a41f800 1 -- start start 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd597f1e640 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd470036f00 0x7fd4700393c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5590ce341620 tx=0x7fd588017f30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59a41f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce35cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59a41f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5590ce32f390 con 0x5590ce35c900 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59871f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce35cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59871f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce35cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55216/0 (socket says 172.21.15.63:55216) 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59871f640 1 -- 172.21.15.63:0/3925454361 learned_addr learned my addr 172.21.15.63:0/3925454361 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59871f640 1 -- 172.21.15.63:0/3925454361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ce35d2f0 con 0x5590ce35c900 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd59871f640 1 --2- 172.21.15.63:0/3925454361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce35cd20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5900155a0 tx=0x7fd590012860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca5354b52086ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.473+0000 7fd476ffd640 1 -- 172.21.15.63:0/3925454361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd590042020 con 0x5590ce35c900 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd476ffd640 1 -- 172.21.15.63:0/3925454361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd590021a50 con 0x5590ce35c900 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd476ffd640 1 -- 172.21.15.63:0/3925454361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd590001310 con 0x5590ce35c900 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 -- 172.21.15.63:0/3925454361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 msgr2=0x5590ce35cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 --2- 172.21.15.63:0/3925454361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce35cd20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5900155a0 tx=0x7fd590012860 comp rx=0 tx=0).stop 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 -- 172.21.15.63:0/3925454361 shutdown_connections 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 --2- 172.21.15.63:0/3925454361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce35cd20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 -- 172.21.15.63:0/3925454361 >> 172.21.15.63:0/3925454361 conn(0x5590ce358c10 msgr2=0x5590ce356c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 -- 172.21.15.63:0/3925454361 shutdown_connections 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 -- 172.21.15.63:0/3925454361 wait complete. 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 Processor -- start 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.474+0000 7fd59a41f800 1 -- start start 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59a41f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce363df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59a41f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5590ce35da40 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59871f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce363df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59871f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce363df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55220/0 (socket says 172.21.15.63:55220) 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 learned_addr learned my addr 172.21.15.63:0/1649592486 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ce364330 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59871f640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce363df0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd59004bb90 tx=0x7fd59004cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd4757fa640 1 -- 172.21.15.63:0/1649592486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59002a350 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd4757fa640 1 -- 172.21.15.63:0/1649592486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd590030030 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd4757fa640 1 -- 172.21.15.63:0/1649592486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59002a620 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5590ce3645c0 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.475+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ce367270 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.476+0000 7fd4757fa640 1 -- 172.21.15.63:0/1649592486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd590045070 con 0x5590ce35c900 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.476+0000 7fd4757fa640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd460036eb0 0x7fd460039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd4757fa640 1 -- 172.21.15.63:0/1649592486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7fd59000e040 con 0x5590ce35c900 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd597f1e640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd460036eb0 0x7fd460039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce36fb00 0x5590ce371fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce3726f0 con 0x5590ce36fb00 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59771d640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce36fb00 0x5590ce371fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd597f1e640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd460036eb0 0x7fd460039370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd58804b1d0 tx=0x7fd588037000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59771d640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce36fb00 0x5590ce371fa0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59a41f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59a41f800 20 realm 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.477+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce372ac0 con 0x5590ce36fb00 2024-04-23T19:10:32.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce373390 con 0x5590ce36fb00 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce373730 con 0x5590ce36fb00 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.478+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce373b50 con 0x5590ce36fb00 2024-04-23T19:10:32.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59a41f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59a41f800 20 searching for the correct realm 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5590ce375b50 0x5590ce377fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5590ce378670 con 0x5590ce375b50 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59871f640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5590ce375b50 0x5590ce377fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59871f640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5590ce375b50 0x5590ce377fa0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59871f640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5900123a0 0x7fd59002c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.479+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5590ce378670 con 0x7fd5900123a0 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.480+0000 7fd597f1e640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5900123a0 0x7fd59002c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.480+0000 7fd597f1e640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5900123a0 0x7fd59002c930 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.480+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.480+0000 7fd597f1e640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd58802c060 0x7fd58802a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.480+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.480+0000 7fd59771d640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd58802c060 0x7fd58802a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.481+0000 7fd59771d640 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd58802c060 0x7fd58802a580 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.481+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.481+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.481+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.481+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.481+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.482+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.482+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.482+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.482+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.482+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd5900123a0 2024-04-23T19:10:32.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.482+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.482+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd5900123a0 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.483+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.484+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.485+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.486+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.486+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd5900123a0 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.486+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.486+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.486+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.486+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.487+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.487+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.487+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.487+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.487+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.487+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd5900123a0 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce375b50 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd58802c060 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x5590ce36fb00 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.488+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd588059040 con 0x7fd5900123a0 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce374620 con 0x5590ce36fb00 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce374ec0 con 0x5590ce36fb00 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.489+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce374260 con 0x7fd58802c060 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 zone default found 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 4 Realm: () 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 zonegroup default 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce374ec0 con 0x7fd58802c060 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37a520 con 0x5590ce375b50 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37a8c0 con 0x5590ce375b50 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 started sync module instance, tier type = 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.490+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce3799c0 con 0x7fd5900123a0 2024-04-23T19:10:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.492+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.492+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94080423402928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37b1d0 con 0x7fd5900123a0 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.492+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37ba20 con 0x5590ce36fb00 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.493+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94080423402928] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.493+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.493+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94080423406880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37bae0 con 0x5590ce36fb00 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.493+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37c600 con 0x5590ce375b50 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.494+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94080423406880] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.494+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.494+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94080423411920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37c9f0 con 0x5590ce375b50 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.494+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x7fd5900123a0 2024-04-23T19:10:32.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.495+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94080423411920] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.495+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.495+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94080423409008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d840 con 0x7fd5900123a0 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.495+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37e280 con 0x5590ce36fb00 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.496+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.496+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94080423419216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37e240 con 0x5590ce36fb00 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.496+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37f230 con 0x7fd58802c060 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.496+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94080423409008] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.497+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.497+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94080423423296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37f230 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.497+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37fc70 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.497+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94080423419216] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.498+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94080423423296] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.498+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.498+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94080423415600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37f7c0 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.498+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce380050 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94080423415600] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94080423421392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce380230 con 0x7fd58802c060 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 20 add_watcher() i=0 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 20 add_watcher() i=1 2024-04-23T19:10:32.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 20 add_watcher() i=2 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 20 add_watcher() i=3 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 20 add_watcher() i=4 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 20 add_watcher() i=5 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.499+0000 7fd59a41f800 20 add_watcher() i=6 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.500+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94080423421392] v30'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.500+0000 7fd59a41f800 20 add_watcher() i=7 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.500+0000 7fd59a41f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.500+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x5590ce383600 con 0x5590ce36fb00 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.501+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.501+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94080423428384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce3809f0 con 0x5590ce36fb00 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94080423428384] v30'146 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd44dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd59a41f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd59a41f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd59a41f800 5 note: GC not initialized 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd44cfe1640 20 reqs_thread_entry: start 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd59a41f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd59a41f800 20 Filter name: none 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd3c67fc640 20 reqs_thread_entry: start 2024-04-23T19:10:32.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.502+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 6.0 6:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5590ce3b5e30 con 0x7fd58802c060 2024-04-23T19:10:32.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.503+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(62 [pgnls start_epoch 30 out=254b] v30'34 uv18 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.503+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.10 6:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5590ce374620 con 0x7fd58802c060 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.504+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(63 [pgnls start_epoch 30 out=131b] v30'4 uv4 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.504+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.8 6:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce375b50 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.504+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 [pgnls start_epoch 30 out=295b] v30'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.504+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:65 6.18 6:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37c600 con 0x7fd58802c060 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.505+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(65 [pgnls start_epoch 30 out=172b] v30'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.505+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce36fb00 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.505+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(66 [pgnls start_epoch 30 out=335b] v30'146 uv23 ondisk = 1) ==== 144+0+335 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.505+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 6.14 6:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd5900123a0 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.506+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(67 [pgnls start_epoch 30 out=213b] v30'10 uv10 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.506+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 6.c 6:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x5590ce375b50 2024-04-23T19:10:32.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.506+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 [pgnls start_epoch 30 out=254b] v30'10 uv10 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.506+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 6.1c 6:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.506+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(69 [pgnls start_epoch 30 out=295b] v30'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.506+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 6.2 6:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd590064040 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(70 [pgnls start_epoch 30 out=213b] v30'11 uv10 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 6.12 6:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd590064040 con 0x7fd5900123a0 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(71 [pgnls start_epoch 30 out=172b] v30'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.a 6:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd590064040 con 0x7fd5900123a0 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(72 [pgnls start_epoch 30 out=336b] v30'27 uv19 ondisk = 1) ==== 144+0+336 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 6.1a 6:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd590064040 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(73 [pgnls start_epoch 30 out=131b] v30'17 uv9 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.507+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:74 6.6 6:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd590064040 con 0x5590ce375b50 2024-04-23T19:10:32.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.508+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(74 [pgnls start_epoch 30 out=254b] v30'12 uv12 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.508+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 6.16 6:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd590064040 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.508+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(75 [pgnls start_epoch 30 out=90b] v30'17 uv9 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.508+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.e 6:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd590064040 con 0x5590ce36fb00 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.508+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(76 [pgnls start_epoch 30 out=213b] v30'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.508+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.1e 6:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(77 [pgnls start_epoch 30 out=254b] v30'10 uv10 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:78 6.1 6:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce36fb00 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(78 [pgnls start_epoch 30 out=254b] v30'14 uv14 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.11 6:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(79 [pgnls start_epoch 30 out=213b] v30'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:80 6.9 6:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(80 [pgnls start_epoch 30 out=172b] v30'28 uv12 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.509+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.19 6:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce36fb00 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.510+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(81 [pgnls start_epoch 30 out=295b] v30'24 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.510+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:82 6.5 6:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd5900123a0 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.510+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(82 [pgnls start_epoch 30 out=276b] v30'15 uv15 ondisk = 1) ==== 144+0+276 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.510+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.15 6:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd5900123a0 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.510+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(83 [pgnls start_epoch 30 out=172b] v30'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.510+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:84 6.d 6:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce36fb00 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.511+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(84 [pgnls start_epoch 30 out=295b] v30'25 uv17 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.511+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:85 6.1d 6:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd58802c060 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.511+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(85 [pgnls start_epoch 30 out=172b] v30'28 uv12 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.511+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:86 6.3 6:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd5900123a0 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.512+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(86 [pgnls start_epoch 30 out=213b] v30'43 uv19 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.512+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:87 6.13 6:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd58802c060 2024-04-23T19:10:32.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.512+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(87 [pgnls start_epoch 30 out=172b] v30'19 uv11 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.512+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:88 6.b 6:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce375b50 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(88 [pgnls start_epoch 30 out=172b] v30'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1b 6:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce375b50 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(89 [pgnls start_epoch 30 out=90b] v30'13 uv5 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:90 6.7 6:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce375b50 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(90 [pgnls start_epoch 30 out=90b] v30'38 uv13 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:91 6.17 6:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x5590ce36fb00 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(91 [pgnls start_epoch 30 out=131b] v30'6 uv6 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.513+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:92 6.f 6:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd5900123a0 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.514+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(92 [pgnls start_epoch 30 out=295b] v30'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.514+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:93 6.1f 6:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x7fd58c091040 con 0x7fd58802c060 2024-04-23T19:10:32.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.514+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(93 [pgnls start_epoch 30 out=213b] v30'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.527 INFO:teuthology.orchestra.run.smithi063.stdout:["obj_delete_at_hint.0000000078","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","data_loggenerations_metadata","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","obj_delete_at_hint.0000000027","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","data_log.0","obj_delete_at_hint.0000000012","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2024-04-23T19:10:32.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.516+0000 7fd44dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.516+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94080423428384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce374620 con 0x5590ce36fb00 2024-04-23T19:10:32.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.517+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94080423428384] v30'147 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.517+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94080423402928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce3b4360 con 0x7fd5900123a0 2024-04-23T19:10:32.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.518+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94080423402928] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.518+0000 7fd59a41f800 20 remove_watcher() i=0 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.518+0000 7fd59a41f800 2 removed watcher, disabling cache 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.518+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94080423406880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x5590ce36fb00 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.519+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94080423406880] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.519+0000 7fd59a41f800 20 remove_watcher() i=1 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.519+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94080423411920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce3b54d0 con 0x5590ce375b50 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.520+0000 7fd59871f640 1 -- 172.21.15.63:0/1649592486 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94080423411920] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd590044030 con 0x5590ce375b50 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.520+0000 7fd59a41f800 20 remove_watcher() i=2 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.520+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94080423409008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x7fd5900123a0 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.521+0000 7fd597f1e640 1 -- 172.21.15.63:0/1649592486 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94080423409008] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd588029030 con 0x7fd5900123a0 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.521+0000 7fd59a41f800 20 remove_watcher() i=3 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.521+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94080423419216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x5590ce36fb00 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.522+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94080423419216] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x5590ce36fb00 2024-04-23T19:10:32.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.522+0000 7fd59a41f800 20 remove_watcher() i=4 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.522+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94080423423296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x7fd58802c060 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.523+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94080423423296] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.523+0000 7fd59a41f800 20 remove_watcher() i=5 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.523+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94080423415600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x7fd58802c060 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.524+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94080423415600] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.524+0000 7fd59a41f800 20 remove_watcher() i=6 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.524+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94080423421392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x5590ce37d420 con 0x7fd58802c060 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94080423421392] v30'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd58c05a040 con 0x7fd58802c060 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 20 remove_watcher() i=7 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd58802c060 msgr2=0x7fd58802a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd58802c060 0x7fd58802a580 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce36fb00 msgr2=0x5590ce371fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce36fb00 0x5590ce371fa0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5900123a0 msgr2=0x7fd59002c930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd5900123a0 0x7fd59002c930 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5590ce375b50 msgr2=0x5590ce377fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5590ce375b50 0x5590ce377fa0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd460036eb0 msgr2=0x7fd460039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd460036eb0 0x7fd460039370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd58804b1d0 tx=0x7fd588037000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 msgr2=0x5590ce363df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59a41f800 1 --2- 172.21.15.63:0/1649592486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce35c900 0x5590ce363df0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd59004bb90 tx=0x7fd59004cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 reap_dead start 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.525+0000 7fd59771d640 1 -- 172.21.15.63:0/1649592486 reap_dead start 2024-04-23T19:10:32.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 shutdown_connections 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 >> 172.21.15.63:0/1649592486 conn(0x5590ce358c10 msgr2=0x5590ce356270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 shutdown_connections 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/1649592486 wait complete. 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd470036f00 msgr2=0x7fd4700393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd470036f00 0x7fd4700393c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5590ce341620 tx=0x7fd588017f30 comp rx=0 tx=0).stop 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce34f5f0 msgr2=0x5590ce34fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce34f5f0 0x5590ce34fa10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5590ce33bae0 tx=0x7fd5900178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 shutdown_connections 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd470036f00 0x7fd4700393c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/70329693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce34f5f0 0x5590ce34fa10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 >> 172.21.15.63:0/70329693 conn(0x5590ce343690 msgr2=0x5590ce343a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 shutdown_connections 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/70329693 wait complete. 2024-04-23T19:10:32.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5590ce33e8d0 msgr2=0x5590ce340cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5590ce33e8d0 0x5590ce340cc0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce23e490 msgr2=0x5590ce33b150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce23e490 0x5590ce33b150 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd480036e60 msgr2=0x7fd480039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd480036e60 0x7fd480039320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd58c054000 tx=0x7fd58c04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 msgr2=0x5590ce333f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce333f10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd588004000 tx=0x7fd588000bc0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 shutdown_connections 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.526+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5590ce33e8d0 0x5590ce340cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.527+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5590ce23e490 0x5590ce33b150 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.527+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd480036e60 0x7fd480039320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.527+0000 7fd59a41f800 1 --2- 172.21.15.63:0/236039330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5590ce23a000 0x5590ce333f10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.527+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 >> 172.21.15.63:0/236039330 conn(0x5590ce235620 msgr2=0x5590ce325350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.527+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 shutdown_connections 2024-04-23T19:10:32.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.527+0000 7fd59a41f800 1 -- 172.21.15.63:0/236039330 wait complete. 2024-04-23T19:10:32.539 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-04-23T19:10:32.539 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-23T19:10:32.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-23T19:10:32.539 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-23T19:10:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92e3936800 1 Processor -- start 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92e3936800 1 -- start start 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92e3936800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558f3fa32c20 0x558f3fa31180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92e3936800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558f3f96f7f0 con 0x558f3fa316c0 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92e3936800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x558f3f95ee20 con 0x558f3fa32c20 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92df6cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558f3fa32c20 0x558f3fa31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92df6cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558f3fa32c20 0x558f3fa31180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55224/0 (socket says 172.21.15.63:55224) 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92df6cc640 1 -- 172.21.15.63:0/2242808734 learned_addr learned my addr 172.21.15.63:0/2242808734 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.624+0000 7f92deecb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x558f3fa316c0 0x558f3fac4ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38890/0 (socket says 172.21.15.63:38890) 2024-04-23T19:10:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.625+0000 7f92df6cc640 1 -- 172.21.15.63:0/2242808734 >> v1:172.21.15.16:6789/0 conn(0x558f3fa316c0 legacy=0x558f3fac4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.625+0000 7f92df6cc640 1 -- 172.21.15.63:0/2242808734 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3f97db50 con 0x558f3fa32c20 2024-04-23T19:10:32.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.626+0000 7f92df6cc640 1 --2- 172.21.15.63:0/2242808734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa32c20 0x558f3fa31180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f92d000a520 tx=0x7f92d001cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29dedf912708d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92de6ca640 1 -- 172.21.15.63:0/2242808734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d0057070 con 0x558f3fa32c20 2024-04-23T19:10:32.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92de6ca640 1 -- 172.21.15.63:0/2242808734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d004ad80 con 0x558f3fa32c20 2024-04-23T19:10:32.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92de6ca640 1 -- 172.21.15.63:0/2242808734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d0052590 con 0x558f3fa32c20 2024-04-23T19:10:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92e3936800 1 -- 172.21.15.63:0/2242808734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa32c20 msgr2=0x558f3fa31180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92e3936800 1 --2- 172.21.15.63:0/2242808734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa32c20 0x558f3fa31180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f92d000a520 tx=0x7f92d001cd50 comp rx=0 tx=0).stop 2024-04-23T19:10:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92e3936800 1 -- 172.21.15.63:0/2242808734 shutdown_connections 2024-04-23T19:10:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92e3936800 1 --2- 172.21.15.63:0/2242808734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa32c20 0x558f3fa31180 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92e3936800 1 -- 172.21.15.63:0/2242808734 >> 172.21.15.63:0/2242808734 conn(0x558f3fab87e0 msgr2=0x558f3fabac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92e3936800 1 -- 172.21.15.63:0/2242808734 shutdown_connections 2024-04-23T19:10:32.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.627+0000 7f92e3936800 1 -- 172.21.15.63:0/2242808734 wait complete. 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92e3936800 1 Processor -- start 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92e3936800 1 -- start start 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92e3936800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92e3936800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f3fa086f0 con 0x558f3fa316c0 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92df6cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92df6cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55240/0 (socket says 172.21.15.63:55240) 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92df6cc640 1 -- 172.21.15.63:0/3215706587 learned_addr learned my addr 172.21.15.63:0/3215706587 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.640+0000 7f92df6cc640 1 -- 172.21.15.63:0/3215706587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3fbb0fd0 con 0x558f3fa316c0 2024-04-23T19:10:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.641+0000 7f92df6cc640 1 --2- 172.21.15.63:0/3215706587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb0a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f92d0055040 tx=0x7f92d004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9d3285aa4e4f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.641+0000 7f91cd60a640 1 -- 172.21.15.63:0/3215706587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d0057040 con 0x558f3fa316c0 2024-04-23T19:10:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.641+0000 7f91cd60a640 1 -- 172.21.15.63:0/3215706587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d0063070 con 0x558f3fa316c0 2024-04-23T19:10:32.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.641+0000 7f91cd60a640 1 -- 172.21.15.63:0/3215706587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d005d750 con 0x558f3fa316c0 2024-04-23T19:10:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.641+0000 7f92e3936800 1 -- 172.21.15.63:0/3215706587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 msgr2=0x558f3fbb0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.641+0000 7f92e3936800 1 --2- 172.21.15.63:0/3215706587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb0a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f92d0055040 tx=0x7f92d004c000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.641+0000 7f92e3936800 1 -- 172.21.15.63:0/3215706587 shutdown_connections 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 --2- 172.21.15.63:0/3215706587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb0a00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 -- 172.21.15.63:0/3215706587 >> 172.21.15.63:0/3215706587 conn(0x558f3fab87e0 msgr2=0x558f3fbac320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 -- 172.21.15.63:0/3215706587 shutdown_connections 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 -- 172.21.15.63:0/3215706587 wait complete. 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 Processor -- start 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 -- start start 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92e3936800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f3fac89c0 con 0x558f3fa316c0 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92df6cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92df6cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55250/0 (socket says 172.21.15.63:55250) 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92df6cc640 1 -- 172.21.15.63:0/3634824031 learned_addr learned my addr 172.21.15.63:0/3634824031 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.642+0000 7f92df6cc640 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3fbb89f0 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.643+0000 7f92df6cc640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb84b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f92d004ad50 tx=0x7f92d0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.643+0000 7f91cbe07640 1 -- 172.21.15.63:0/3634824031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d0057050 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.643+0000 7f91cbe07640 1 -- 172.21.15.63:0/3634824031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d005da20 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.643+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f3fbb8c80 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.643+0000 7f91cbe07640 1 -- 172.21.15.63:0/3634824031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d005dd40 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.643+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3fbbb930 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f91cbe07640 1 -- 172.21.15.63:0/3634824031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f92d0052440 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f91cbe07640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91c4036e60 0x7f91c4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f91cbe07640 1 -- 172.21.15.63:0/3634824031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f92d008dfe0 con 0x558f3fa316c0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f92deecb640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91c4036e60 0x7f91c4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fa32c20 0x558f3fbbf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbbfc30 con 0x558f3fa32c20 2024-04-23T19:10:32.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f92dfecd640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fa32c20 0x558f3fbbf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.644+0000 7f92deecb640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91c4036e60 0x7f91c4039320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000f60 tx=0x7f92d4001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.645+0000 7f92dfecd640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fa32c20 0x558f3fbbf6f0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.645+0000 7f92dfecd640 1 -- 172.21.15.63:0/3634824031 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f92d800a4a0 con 0x558f3fa32c20 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.645+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbbcca0 con 0x558f3fa32c20 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.645+0000 7f92dfecd640 1 -- 172.21.15.63:0/3634824031 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f92d800a4a0 con 0x558f3fa32c20 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.645+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbc19f0 con 0x558f3fa32c20 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.645+0000 7f92dfecd640 1 -- 172.21.15.63:0/3634824031 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f92d800a4a0 con 0x558f3fa32c20 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.645+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbc0d30 con 0x558f3fa32c20 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.646+0000 7f92dfecd640 1 -- 172.21.15.63:0/3634824031 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f92d800a4a0 con 0x558f3fa32c20 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.646+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f3fbc2da0 0x558f3fbc5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.646+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbc5710 con 0x558f3fbc2da0 2024-04-23T19:10:32.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.646+0000 7f92df6cc640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f3fbc2da0 0x558f3fbc5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.646+0000 7f92df6cc640 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f3fbc2da0 0x558f3fbc5190 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92df6cc640 1 -- 172.21.15.63:0/3634824031 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f92d0078070 con 0x558f3fbc2da0 2024-04-23T19:10:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92e3936800 1 Processor -- start 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92e3936800 1 -- start start 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92e3936800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbd3b30 0x558f3fbd3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92e3936800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f3f9aee30 con 0x558f3fbd3b30 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92dfecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbd3b30 0x558f3fbd3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92dfecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbd3b30 0x558f3fbd3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55262/0 (socket says 172.21.15.63:55262) 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.647+0000 7f92dfecd640 1 -- 172.21.15.63:0/2745840270 learned_addr learned my addr 172.21.15.63:0/2745840270 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.648+0000 7f92dfecd640 1 -- 172.21.15.63:0/2745840270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3fbd4490 con 0x558f3fbd3b30 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.648+0000 7f92dfecd640 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbd3b30 0x558f3fbd3f50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x558f3fbb98b0 tx=0x7f92d80178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.648+0000 7f91c9602640 1 -- 172.21.15.63:0/2745840270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8013440 con 0x558f3fbd3b30 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.648+0000 7f91c9602640 1 -- 172.21.15.63:0/2745840270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d80135e0 con 0x558f3fbd3b30 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.648+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f3fbd4f00 con 0x558f3fbd3b30 2024-04-23T19:10:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.648+0000 7f91c9602640 1 -- 172.21.15.63:0/2745840270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d80138c0 con 0x558f3fbd3b30 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.648+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3fbd5400 con 0x558f3fbd3b30 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.649+0000 7f91c9602640 1 -- 172.21.15.63:0/2745840270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f92d8011040 con 0x558f3fbd3b30 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.649+0000 7f91c9602640 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91b8036eb0 0x7f91b8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.649+0000 7f91c9602640 1 -- 172.21.15.63:0/2745840270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f92d801d070 con 0x558f3fbd3b30 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.649+0000 7f92df6cc640 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91b8036eb0 0x7f91b8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92e3936800 1 Processor -- start 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92e3936800 1 -- start start 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92df6cc640 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91b8036eb0 0x7f91b8039370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f92d005c910 tx=0x7f92d005c980 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92e3936800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92e3936800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f3fbb1290 con 0x558f3fbe0e50 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92dfecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92dfecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55274/0 (socket says 172.21.15.63:55274) 2024-04-23T19:10:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92dfecd640 1 -- 172.21.15.63:0/1283741010 learned_addr learned my addr 172.21.15.63:0/1283741010 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92dfecd640 1 -- 172.21.15.63:0/1283741010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3fbe1840 con 0x558f3fbe0e50 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.650+0000 7f92dfecd640 1 --2- 172.21.15.63:0/1283741010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe1270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f92d804cca0 tx=0x7f92d8012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d61ebda19ff5c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f91c2ffd640 1 -- 172.21.15.63:0/1283741010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8042030 con 0x558f3fbe0e50 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f91c2ffd640 1 -- 172.21.15.63:0/1283741010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d8001310 con 0x558f3fbe0e50 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f91c2ffd640 1 -- 172.21.15.63:0/1283741010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8027040 con 0x558f3fbe0e50 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 -- 172.21.15.63:0/1283741010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 msgr2=0x558f3fbe1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 --2- 172.21.15.63:0/1283741010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe1270 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f92d804cca0 tx=0x7f92d8012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 -- 172.21.15.63:0/1283741010 shutdown_connections 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 --2- 172.21.15.63:0/1283741010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe1270 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 -- 172.21.15.63:0/1283741010 >> 172.21.15.63:0/1283741010 conn(0x558f3fbdd170 msgr2=0x558f3fbdb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 -- 172.21.15.63:0/1283741010 shutdown_connections 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 -- 172.21.15.63:0/1283741010 wait complete. 2024-04-23T19:10:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.651+0000 7f92e3936800 1 Processor -- start 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92e3936800 1 -- start start 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92e3936800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92e3936800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f3fbe1f60 con 0x558f3fbe0e50 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92dfecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92dfecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55288/0 (socket says 172.21.15.63:55288) 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 learned_addr learned my addr 172.21.15.63:0/4074963749 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3fbe8930 con 0x558f3fbe0e50 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92dfecd640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe83f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92d8001df0 tx=0x7f92d804cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f91c17fa640 1 -- 172.21.15.63:0/4074963749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d8027030 con 0x558f3fbe0e50 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f91c17fa640 1 -- 172.21.15.63:0/4074963749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d802a350 con 0x558f3fbe0e50 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f3fbe8bc0 con 0x558f3fbe0e50 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f91c17fa640 1 -- 172.21.15.63:0/4074963749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92d802a650 con 0x558f3fbe0e50 2024-04-23T19:10:32.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.652+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3fbeb870 con 0x558f3fbe0e50 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.653+0000 7f91c17fa640 1 -- 172.21.15.63:0/4074963749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f92d8045070 con 0x558f3fbe0e50 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.653+0000 7f91c17fa640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91a8036eb0 0x7f91a8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.653+0000 7f91c17fa640 1 -- 172.21.15.63:0/4074963749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f92d8043070 con 0x558f3fbe0e50 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.653+0000 7f92df6cc640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91a8036eb0 0x7f91a8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.654+0000 7f92df6cc640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91a8036eb0 0x7f91a8039370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f92d005b6b0 tx=0x7f92d0086000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.654+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.654+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fbf4040 0x558f3fbf6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.654+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf6c50 con 0x558f3fbf4040 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.654+0000 7f92deecb640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fbf4040 0x558f3fbf6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.654+0000 7f92deecb640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fbf4040 0x558f3fbf6500 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 realm 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf3490 con 0x558f3fbf4040 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf7740 con 0x558f3fbf4040 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf8010 con 0x558f3fbf4040 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.655+0000 7f92e3936800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf83b0 con 0x558f3fbf4040 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92e3936800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92e3936800 20 searching for the correct realm 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f3fbfa290 0x558f3fbfc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92dfecd640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f3fbfa290 0x558f3fbfc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.656+0000 7f92dfecd640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f3fbfa290 0x558f3fbfc6c0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92dfecd640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f92d80123a0 0x7f92d804e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d80123a0 2024-04-23T19:10:32.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92df6cc640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f92d80123a0 0x7f92d804e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92df6cc640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f92d80123a0 0x7f92d804e980 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92df6cc640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f92d008c7a0 0x7f92d005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.657+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.658+0000 7f92deecb640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f92d008c7a0 0x7f92d005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.658+0000 7f92deecb640 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f92d008c7a0 0x7f92d005e760 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.658+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.658+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.658+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.658+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d80123a0 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.659+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.660+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d80123a0 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.661+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.662+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.663+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.663+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.663+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.663+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d80123a0 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.663+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.663+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.663+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.664+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.664+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.664+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.664+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d8029070 con 0x558f3fbfa290 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.664+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.665+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.665+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d80123a0 2024-04-23T19:10:32.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.665+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.665+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbfa290 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.665+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d806a040 con 0x558f3fbfa290 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.665+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d008c7a0 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.666+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.666+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x558f3fbf4040 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.666+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.666+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfcd90 con 0x7f92d80123a0 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:32.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf8d30 con 0x558f3fbf4040 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf95d0 con 0x558f3fbf4040 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.667+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfd210 con 0x7f92d008c7a0 2024-04-23T19:10:32.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 zone default found 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 4 Realm: () 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 zonegroup default 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbf9230 con 0x7f92d008c7a0 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbff120 con 0x558f3fbfa290 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f92d806a040 con 0x558f3fbfa290 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbff4c0 con 0x558f3fbfa290 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f92d806a040 con 0x558f3fbfa290 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 started sync module instance, tier type = 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.668+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfe5c0 con 0x7f92d80123a0 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.669+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.669+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94073738226144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfe220 con 0x7f92d80123a0 2024-04-23T19:10:32.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.669+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc003f0 con 0x558f3fbf4040 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.670+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94073738226144] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.671+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.671+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94073738231616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc00360 con 0x558f3fbf4040 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.671+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc012b0 con 0x558f3fbfa290 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.672+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94073738231616] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.672+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d806a040 con 0x558f3fbfa290 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.672+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94073738235392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc016a0 con 0x558f3fbfa290 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.672+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc020f0 con 0x7f92d80123a0 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.673+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.673+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94073738238944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc02990 con 0x7f92d80123a0 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.673+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc03470 con 0x558f3fbf4040 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.673+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94073738235392] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d806a040 con 0x558f3fbfa290 2024-04-23T19:10:32.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.674+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.674+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94073738243968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc03890 con 0x558f3fbf4040 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.674+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc04290 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.674+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94073738238944] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.675+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.675+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94073738243968] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.675+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94073738241248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc04250 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.675+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc04760 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.676+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94073738241248] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.676+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(56 notify.6 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.676+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94073738233744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc04940 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.676+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc05190 con 0x7f92d008c7a0 2024-04-23T19:10:32.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94073738233744] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94073738250560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc05210 con 0x7f92d008c7a0 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 20 add_watcher() i=0 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 20 add_watcher() i=1 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 20 add_watcher() i=2 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 20 add_watcher() i=3 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 20 add_watcher() i=4 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 20 add_watcher() i=5 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.677+0000 7f92e3936800 20 add_watcher() i=6 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.678+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94073738250560] v30'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.678+0000 7f92e3936800 20 add_watcher() i=7 2024-04-23T19:10:32.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.678+0000 7f92e3936800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.678+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc084a0 con 0x558f3fbf4040 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.679+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.679+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94073738251984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc057a0 con 0x558f3fbf4040 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94073738251984] v30'148 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f9195fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 5 note: GC not initialized 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f9194fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 20 Filter name: none 2024-04-23T19:10:32.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f910e7fc640 20 reqs_thread_entry: start 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.680+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc3ba20 con 0x558f3fbf4040 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc3e350 con 0x7f92d80123a0 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 20 disabling bucket name=myfoo 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc3c330 con 0x7f92d80123a0 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.681+0000 7f92e3936800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc3eb10 con 0x558f3fbf4040 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:32.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.682+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc466c0 con 0x558f3fbf4040 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.683+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v30'8 uv8 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.683+0000 7f92e3936800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.683+0000 7f92e3936800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.683+0000 7f92e3936800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.683+0000 7f92e3936800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.683+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94073738490288 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc41d20 con 0x7f92d008c7a0 2024-04-23T19:10:32.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== watch-notify(notify (1) cookie 94073738241248 notify 128849018895 ret 0) ==== 794+0+0 (crc 0 0 0) 0x558f3fc3ef00 con 0x7f92d008c7a0 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f91c2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018895 cookie 94073738241248 notifier 4590 bl.length()=752 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(67 notify.5 [notify cookie 94073738490288 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f91c2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f91c2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f91c2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f91c2ffd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f91a4003ea0 con 0x7f92d008c7a0 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== watch-notify(notify_complete (2) cookie 94073738490288 notify 128849018895 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558f3fc3e050 con 0x7f92d008c7a0 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92e3936800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_sources= new_sources= 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92e3936800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential sources added=[] removed=[] 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92e3936800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_dests= new_dests= 2024-04-23T19:10:32.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92e3936800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential dests added=[] removed=[] 2024-04-23T19:10:32.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.684+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc41830 con 0x558f3fbf4040 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v30'12 uv12 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f92e3936800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f92e3936800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f92e3936800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94073738469760 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc41830 con 0x558f3fbf4040 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== watch-notify(notify (1) cookie 94073738243968 notify 128849018896 ret 0) ==== 609+0+0 (crc 0 0 0) 0x558f3fc3e050 con 0x558f3fbf4040 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f91c2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018896 cookie 94073738243968 notifier 4590 bl.length()=567 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.685+0000 7f91c2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.686+0000 7f91c2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:32.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.686+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(70 notify.4 [notify cookie 94073738469760 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.686+0000 7f91c2ffd640 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f91a4003ea0 con 0x558f3fbf4040 2024-04-23T19:10:32.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.686+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.686+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== watch-notify(notify_complete (2) cookie 94073738469760 notify 128849018896 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f92d4018040 con 0x558f3fbf4040 2024-04-23T19:10:32.701 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:32.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.687+0000 7f9195fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.687+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94073738251984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc44360 con 0x558f3fbf4040 2024-04-23T19:10:32.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.688+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94073738251984] v30'149 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.688+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94073738226144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc04ed0 con 0x7f92d80123a0 2024-04-23T19:10:32.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.689+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94073738226144] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.689+0000 7f92e3936800 20 remove_watcher() i=0 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.689+0000 7f92e3936800 2 removed watcher, disabling cache 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.689+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94073738231616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc04ed0 con 0x558f3fbf4040 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.690+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94073738231616] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.690+0000 7f92e3936800 20 remove_watcher() i=1 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.690+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94073738235392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfff40 con 0x558f3fbfa290 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.691+0000 7f92dfecd640 1 -- 172.21.15.63:0/4074963749 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94073738235392] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d806a040 con 0x558f3fbfa290 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.691+0000 7f92e3936800 20 remove_watcher() i=2 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.691+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94073738238944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc04ed0 con 0x7f92d80123a0 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.692+0000 7f92df6cc640 1 -- 172.21.15.63:0/4074963749 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94073738238944] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d0078070 con 0x7f92d80123a0 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.692+0000 7f92e3936800 20 remove_watcher() i=3 2024-04-23T19:10:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.692+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94073738243968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc00e00 con 0x558f3fbf4040 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.693+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94073738243968] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x558f3fbf4040 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.693+0000 7f92e3936800 20 remove_watcher() i=4 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.693+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94073738241248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fbfff40 con 0x7f92d008c7a0 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.694+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94073738241248] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.694+0000 7f92e3936800 20 remove_watcher() i=5 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.694+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94073738233744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc02f80 con 0x7f92d008c7a0 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.695+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94073738233744] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.695+0000 7f92e3936800 20 remove_watcher() i=6 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.695+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94073738250560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x558f3fc00e00 con 0x7f92d008c7a0 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94073738250560] v30'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f92d400f610 con 0x7f92d008c7a0 2024-04-23T19:10:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 20 remove_watcher() i=7 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f92d008c7a0 msgr2=0x7f92d005e760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f92d008c7a0 0x7f92d005e760 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fbf4040 msgr2=0x558f3fbf6500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fbf4040 0x558f3fbf6500 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f92d80123a0 msgr2=0x7f92d804e980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f92d80123a0 0x7f92d804e980 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f3fbfa290 msgr2=0x558f3fbfc6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558f3fbfa290 0x558f3fbfc6c0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91a8036eb0 msgr2=0x7f91a8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91a8036eb0 0x7f91a8039370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f92d005b6b0 tx=0x7f92d0086000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 msgr2=0x558f3fbe83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92deecb640 1 -- 172.21.15.63:0/4074963749 reap_dead start 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe83f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92d8001df0 tx=0x7f92d804cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 shutdown_connections 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 --2- 172.21.15.63:0/4074963749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbe0e50 0x558f3fbe83f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 >> 172.21.15.63:0/4074963749 conn(0x558f3fbdd170 msgr2=0x558f3fbda7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 shutdown_connections 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.696+0000 7f92e3936800 1 -- 172.21.15.63:0/4074963749 wait complete. 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91b8036eb0 msgr2=0x7f91b8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91b8036eb0 0x7f91b8039370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f92d005c910 tx=0x7f92d005c980 comp rx=0 tx=0).stop 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbd3b30 msgr2=0x558f3fbd3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbd3b30 0x558f3fbd3f50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x558f3fbb98b0 tx=0x7f92d80178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 shutdown_connections 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91b8036eb0 0x7f91b8039370 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/2745840270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fbd3b30 0x558f3fbd3f50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 >> 172.21.15.63:0/2745840270 conn(0x558f3fbc7bd0 msgr2=0x558f3fbc7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 shutdown_connections 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/2745840270 wait complete. 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f3fbc2da0 msgr2=0x558f3fbc5190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f3fbc2da0 0x558f3fbc5190 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fa32c20 msgr2=0x558f3fbbf6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fa32c20 0x558f3fbbf6f0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91c4036e60 msgr2=0x7f91c4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91c4036e60 0x7f91c4039320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000f60 tx=0x7f92d4001370 comp rx=0 tx=0).stop 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 msgr2=0x558f3fbb84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb84b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f92d004ad50 tx=0x7f92d0002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 shutdown_connections 2024-04-23T19:10:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558f3fbc2da0 0x558f3fbc5190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558f3fa32c20 0x558f3fbbf6f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f91c4036e60 0x7f91c4039320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 --2- 172.21.15.63:0/3634824031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f3fa316c0 0x558f3fbb84b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 >> 172.21.15.63:0/3634824031 conn(0x558f3fab87e0 msgr2=0x558f3fbaab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 shutdown_connections 2024-04-23T19:10:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.697+0000 7f92e3936800 1 -- 172.21.15.63:0/3634824031 wait complete. 2024-04-23T19:10:32.713 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:32.716 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-23T19:10:32.716 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-23T19:10:32.716 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-23T19:10:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:32.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14bc040800 1 Processor -- start 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14bc040800 1 -- start start 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14bc040800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55fe3f25a000 0x55fe3f25a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14bc040800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55fe3f10d7f0 con 0x55fe3f25b260 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14bc040800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55fe3f0fce20 con 0x55fe3f25a000 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14b6cdc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55fe3f25a000 0x55fe3f25a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14b64db640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55fe3f25b260 0x55fe3f25b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:38898/0 (socket says 172.21.15.63:38898) 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14b64db640 1 -- 172.21.15.63:0/928170452 learned_addr learned my addr 172.21.15.63:0/928170452 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14b6cdc640 1 -- 172.21.15.63:0/928170452 >> v1:172.21.15.16:6789/0 conn(0x55fe3f25b260 legacy=0x55fe3f25b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.761+0000 7f14b6cdc640 1 -- 172.21.15.63:0/928170452 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3f11bb50 con 0x55fe3f25a000 2024-04-23T19:10:32.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.763+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/928170452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f25a420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f14a800a470 tx=0x7f14a801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2bb7972f986473 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.763+0000 7f14b5cda640 1 -- 172.21.15.63:0/928170452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a8057070 con 0x55fe3f25a000 2024-04-23T19:10:32.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14b5cda640 1 -- 172.21.15.63:0/928170452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a8052570 con 0x55fe3f25a000 2024-04-23T19:10:32.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14b5cda640 1 -- 172.21.15.63:0/928170452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a8052870 con 0x55fe3f25a000 2024-04-23T19:10:32.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14bc040800 1 -- 172.21.15.63:0/928170452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 msgr2=0x55fe3f25a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14bc040800 1 --2- 172.21.15.63:0/928170452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f25a420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f14a800a470 tx=0x7f14a801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:32.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14bc040800 1 -- 172.21.15.63:0/928170452 shutdown_connections 2024-04-23T19:10:32.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14bc040800 1 --2- 172.21.15.63:0/928170452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f25a420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14bc040800 1 -- 172.21.15.63:0/928170452 >> 172.21.15.63:0/928170452 conn(0x55fe3f255600 msgr2=0x55fe3f257a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14bc040800 1 -- 172.21.15.63:0/928170452 shutdown_connections 2024-04-23T19:10:32.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.764+0000 7f14bc040800 1 -- 172.21.15.63:0/928170452 wait complete. 2024-04-23T19:10:32.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.777+0000 7f14bc040800 1 Processor -- start 2024-04-23T19:10:32.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.777+0000 7f14bc040800 1 -- start start 2024-04-23T19:10:32.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.777+0000 7f14bc040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f1ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.777+0000 7f14bc040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe3f1a66f0 con 0x55fe3f25a000 2024-04-23T19:10:32.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.777+0000 7f14b6cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f1ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.777+0000 7f14b6cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f1ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55296/0 (socket says 172.21.15.63:55296) 2024-04-23T19:10:32.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.777+0000 7f14b6cdc640 1 -- 172.21.15.63:0/2645639829 learned_addr learned my addr 172.21.15.63:0/2645639829 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.778+0000 7f14b6cdc640 1 -- 172.21.15.63:0/2645639829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3f34f4a0 con 0x55fe3f25a000 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.778+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/2645639829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f1ce970 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f14a801ced0 tx=0x7f14a804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425dabd91e0792f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.778+0000 7f13a4e09640 1 -- 172.21.15.63:0/2645639829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a8057040 con 0x55fe3f25a000 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.778+0000 7f13a4e09640 1 -- 172.21.15.63:0/2645639829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a80529a0 con 0x55fe3f25a000 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.778+0000 7f13a4e09640 1 -- 172.21.15.63:0/2645639829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a8052cc0 con 0x55fe3f25a000 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 -- 172.21.15.63:0/2645639829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 msgr2=0x55fe3f1ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 --2- 172.21.15.63:0/2645639829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f1ce970 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f14a801ced0 tx=0x7f14a804c000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 -- 172.21.15.63:0/2645639829 shutdown_connections 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 --2- 172.21.15.63:0/2645639829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f1ce970 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 -- 172.21.15.63:0/2645639829 >> 172.21.15.63:0/2645639829 conn(0x55fe3f255600 msgr2=0x55fe3f346f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 -- 172.21.15.63:0/2645639829 shutdown_connections 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 -- 172.21.15.63:0/2645639829 wait complete. 2024-04-23T19:10:32.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 Processor -- start 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 -- start start 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f354120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.779+0000 7f14bc040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe3f2636d0 con 0x55fe3f25a000 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f14b6cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f354120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f14b6cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f354120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55304/0 (socket says 172.21.15.63:55304) 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f14b6cdc640 1 -- 172.21.15.63:0/2919843286 learned_addr learned my addr 172.21.15.63:0/2919843286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f14b6cdc640 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3f354660 con 0x55fe3f25a000 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f354120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f14a8052740 tx=0x7f14a8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f13a3606640 1 -- 172.21.15.63:0/2919843286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a8073070 con 0x55fe3f25a000 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f13a3606640 1 -- 172.21.15.63:0/2919843286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a805d650 con 0x55fe3f25a000 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe3f3548f0 con 0x55fe3f25a000 2024-04-23T19:10:32.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f13a3606640 1 -- 172.21.15.63:0/2919843286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a8057070 con 0x55fe3f25a000 2024-04-23T19:10:32.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.780+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3f3575a0 con 0x55fe3f25a000 2024-04-23T19:10:32.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.781+0000 7f13a3606640 1 -- 172.21.15.63:0/2919843286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f14a8076070 con 0x55fe3f25a000 2024-04-23T19:10:32.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.781+0000 7f13a3606640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f139c036eb0 0x7f139c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.781+0000 7f13a3606640 1 -- 172.21.15.63:0/2919843286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f14a808e660 con 0x55fe3f25a000 2024-04-23T19:10:32.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.781+0000 7f14b64db640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f139c036eb0 0x7f139c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.782+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f25b260 0x55fe3f35b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.782+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f35b940 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.782+0000 7f14b74dd640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f25b260 0x55fe3f35b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.782+0000 7f14b64db640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f139c036eb0 0x7f139c039370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f14ac000f60 tx=0x7f14ac0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.782+0000 7f14b74dd640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f25b260 0x55fe3f35b360 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.782+0000 7f14b74dd640 1 -- 172.21.15.63:0/2919843286 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14b000a4a0 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.782+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f358930 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.783+0000 7f14b74dd640 1 -- 172.21.15.63:0/2919843286 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f14b000a4a0 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.783+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f35d780 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.783+0000 7f14b74dd640 1 -- 172.21.15.63:0/2919843286 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f14b000a4a0 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.783+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f35cac0 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.783+0000 7f14b74dd640 1 -- 172.21.15.63:0/2919843286 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f14b000a4a0 con 0x55fe3f25b260 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.783+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe3f35eb30 0x55fe3f360f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.783+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3614a0 con 0x55fe3f35eb30 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.784+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe3f35eb30 0x55fe3f360f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.784+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe3f35eb30 0x55fe3f360f20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.784+0000 7f14b6cdc640 1 -- 172.21.15.63:0/2919843286 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f14a8075030 con 0x55fe3f35eb30 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14bc040800 1 Processor -- start 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14bc040800 1 -- start start 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14bc040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f36f8c0 0x55fe3f36fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14bc040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe3f14ce30 con 0x55fe3f36f8c0 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14b74dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f36f8c0 0x55fe3f36fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14b74dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f36f8c0 0x55fe3f36fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55308/0 (socket says 172.21.15.63:55308) 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14b74dd640 1 -- 172.21.15.63:0/3409768733 learned_addr learned my addr 172.21.15.63:0/3409768733 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.785+0000 7f14b74dd640 1 -- 172.21.15.63:0/3409768733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3f370220 con 0x55fe3f36f8c0 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.786+0000 7f14b74dd640 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f36f8c0 0x55fe3f36fce0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55fe3f355520 tx=0x7f14b00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.786+0000 7f13a0e01640 1 -- 172.21.15.63:0/3409768733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0013440 con 0x55fe3f36f8c0 2024-04-23T19:10:32.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.786+0000 7f13a0e01640 1 -- 172.21.15.63:0/3409768733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b00135e0 con 0x55fe3f36f8c0 2024-04-23T19:10:32.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.786+0000 7f13a0e01640 1 -- 172.21.15.63:0/3409768733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b00138c0 con 0x55fe3f36f8c0 2024-04-23T19:10:32.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.786+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe3f370c90 con 0x55fe3f36f8c0 2024-04-23T19:10:32.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.786+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3f371190 con 0x55fe3f36f8c0 2024-04-23T19:10:32.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f13a0e01640 1 -- 172.21.15.63:0/3409768733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f14b0011040 con 0x55fe3f36f8c0 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f13a0e01640 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1390036e60 0x7f1390039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f13a0e01640 1 -- 172.21.15.63:0/3409768733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f14b001d070 con 0x55fe3f36f8c0 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1390036e60 0x7f1390039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f14bc040800 1 Processor -- start 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1390036e60 0x7f1390039320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55fe3f361880 tx=0x7f14a8052500 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f14bc040800 1 -- start start 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f14bc040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f37d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.787+0000 7f14bc040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe3f34f730 con 0x55fe3f37cbe0 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f14b74dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f37d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f14b74dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f37d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55314/0 (socket says 172.21.15.63:55314) 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f14b74dd640 1 -- 172.21.15.63:0/3383109875 learned_addr learned my addr 172.21.15.63:0/3383109875 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f14b74dd640 1 -- 172.21.15.63:0/3383109875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3f37d5d0 con 0x55fe3f37cbe0 2024-04-23T19:10:32.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f14b74dd640 1 --2- 172.21.15.63:0/3383109875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f37d000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f14b004ccb0 tx=0x7f14b0012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebd93aa655f5d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f139a7fc640 1 -- 172.21.15.63:0/3383109875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0012df0 con 0x55fe3f37cbe0 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f139a7fc640 1 -- 172.21.15.63:0/3383109875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b0001310 con 0x55fe3f37cbe0 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.788+0000 7f139a7fc640 1 -- 172.21.15.63:0/3383109875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0001630 con 0x55fe3f37cbe0 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 -- 172.21.15.63:0/3383109875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 msgr2=0x55fe3f37d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 --2- 172.21.15.63:0/3383109875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f37d000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f14b004ccb0 tx=0x7f14b0012910 comp rx=0 tx=0).stop 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 -- 172.21.15.63:0/3383109875 shutdown_connections 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 --2- 172.21.15.63:0/3383109875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f37d000 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 -- 172.21.15.63:0/3383109875 >> 172.21.15.63:0/3383109875 conn(0x55fe3f378f40 msgr2=0x55fe3f376fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 -- 172.21.15.63:0/3383109875 shutdown_connections 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 -- 172.21.15.63:0/3383109875 wait complete. 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 Processor -- start 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 -- start start 2024-04-23T19:10:32.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f3841d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14bc040800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe3f37dd40 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14b74dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f3841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14b74dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f3841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:55318/0 (socket says 172.21.15.63:55318) 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.789+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 learned_addr learned my addr 172.21.15.63:0/66332407 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.790+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3f384710 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.790+0000 7f14b74dd640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f3841d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f14b0015ac0 tx=0x7f14b004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.790+0000 7f1398ff9640 1 -- 172.21.15.63:0/66332407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b0021c60 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.790+0000 7f1398ff9640 1 -- 172.21.15.63:0/66332407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b0021e40 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.790+0000 7f1398ff9640 1 -- 172.21.15.63:0/66332407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b002a350 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.790+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe3f3849a0 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.790+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3f387650 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f1398ff9640 1 -- 172.21.15.63:0/66332407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f14b0040040 con 0x55fe3f37cbe0 2024-04-23T19:10:32.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f1398ff9640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1380036e60 0x7f1380039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f1398ff9640 1 -- 172.21.15.63:0/66332407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6110+0+0 (secure 0 0 0) 0x7f14b0033030 con 0x55fe3f37cbe0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1380036e60 0x7f1380039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1380036e60 0x7f1380039320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14a8006d80 tx=0x7f14a8087000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f38fcd0 0x55fe3f392190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.791+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3928e0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14b64db640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f38fcd0 0x55fe3f392190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14b64db640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f38fcd0 0x55fe3f392190 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14ac009050 con 0x55fe3f38fcd0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 20 realm 2024-04-23T19:10:32.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f38f120 con 0x55fe3f38fcd0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14ac009050 con 0x55fe3f38fcd0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.792+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3933d0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14ac009050 con 0x55fe3f38fcd0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f393ca0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f14ac009050 con 0x55fe3f38fcd0 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f394040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f14ac009050 con 0x55fe3f38fcd0 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 20 searching for the correct realm 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe3f395f20 0x55fe3f398310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.793+0000 7f14b74dd640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe3f395f20 0x55fe3f398310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.794+0000 7f14b74dd640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe3f395f20 0x55fe3f398310 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.794+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.794+0000 7f14b74dd640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f14b00123a0 0x7f14b0022830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.794+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14b00123a0 2024-04-23T19:10:32.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.794+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f14b00123a0 0x7f14b0022830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.795+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f14b00123a0 0x7f14b0022830 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.795+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.795+0000 7f14b6cdc640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f14a809d040 0x7f14a808f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.795+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.795+0000 7f14b64db640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f14a809d040 0x7f14a808f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.796+0000 7f14b64db640 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f14a809d040 0x7f14a808f980 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.796+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac009050 con 0x7f14a809d040 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.796+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.796+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac009050 con 0x55fe3f38fcd0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.796+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.796+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.796+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 30 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14b00123a0 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.797+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14b00123a0 2024-04-23T19:10:32.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.798+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.799+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.799+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.799+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.799+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.799+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.799+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.800+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14b00123a0 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.801+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14b00123a0 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.802+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f395f20 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.803+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.803+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14a809d040 2024-04-23T19:10:32.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.803+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.803+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.803+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.803+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3989c0 con 0x7f14b00123a0 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.803+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(37 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3949c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f395260 con 0x55fe3f38fcd0 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f398e40 con 0x7f14a809d040 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 zone default found 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 4 Realm: () 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:32.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 zonegroup default 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.804+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3992f0 con 0x7f14a809d040 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39acd0 con 0x55fe3f395f20 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39b0f0 con 0x55fe3f395f20 2024-04-23T19:10:32.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 20 started sync module instance, tier type = 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.805+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39a160 con 0x7f14b00123a0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.806+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(44 notify.0 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.806+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94550470796624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f399dc0 con 0x7f14b00123a0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.806+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39c020 con 0x55fe3f38fcd0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.807+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94550470796624] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.808+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.808+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94550470802288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39c440 con 0x55fe3f38fcd0 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.808+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39ce00 con 0x55fe3f395f20 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.809+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.809+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94550470805840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39d690 con 0x55fe3f395f20 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.809+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39e160 con 0x7f14b00123a0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.809+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94550470802288] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.810+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94550470805840] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.810+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(50 notify.3 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.810+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94550470810704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39d1a0 con 0x7f14b00123a0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.810+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39eb20 con 0x55fe3f38fcd0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.811+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.811+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94550470813776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39f2c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.811+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39fde0 con 0x7f14a809d040 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.811+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94550470810704] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.812+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(54 notify.5 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.812+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94550470817968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39fda0 con 0x7f14a809d040 2024-04-23T19:10:32.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.812+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3a0840 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.812+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94550470813776] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.813+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(56 notify.6 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.813+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94550470820720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3a0840 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.813+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3a1200 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.813+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94550470817968] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94550470820720] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(58 notify.7 [create] v30'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94550470814992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3a0d50 con 0x7f14a809d040 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 20 add_watcher() i=0 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 20 add_watcher() i=1 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 20 add_watcher() i=2 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 20 add_watcher() i=3 2024-04-23T19:10:32.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 20 add_watcher() i=4 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 20 add_watcher() i=5 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.814+0000 7f14bc040800 20 add_watcher() i=6 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.815+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94550470814992] v30'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.815+0000 7f14bc040800 20 add_watcher() i=7 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.815+0000 7f14bc040800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.815+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3a4230 con 0x55fe3f38fcd0 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.816+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.816+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94550470823248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3a1650 con 0x55fe3f38fcd0 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94550470823248] v30'150 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f136d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f14bc040800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f14bc040800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:32.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f14bc040800 5 note: GC not initialized 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f12e7fff640 20 reqs_thread_entry: start 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f14bc040800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.817+0000 7f14bc040800 20 Filter name: none 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f12e5ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3d7620 con 0x55fe3f38fcd0 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv12 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:32.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3d7100 con 0x7f14b00123a0 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 20 enabling bucket name=myfoo 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.818+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3d7fa0 con 0x7f14b00123a0 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 : miss 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 20 rados->read ofs=0 len=0 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3d99c0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv8 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:32.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:32.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:32.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:32.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:32.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 2024-04-23T19:10:32.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.819+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.8 8:1253f2a4:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3e25e0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.820+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v30'9 uv9 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.820+0000 7f14bc040800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.820+0000 7f14bc040800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.820+0000 7f14bc040800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.820+0000 7f14bc040800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1, ofs0, ns] 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94550471089792 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3dc720 con 0x7f14a809d040 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== watch-notify(notify (1) cookie 94550470817968 notify 128849018896 ret 0) ==== 794+0+0 (crc 0 0 0) 0x55fe3f3dafb0 con 0x7f14a809d040 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f139a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018896 cookie 94550470817968 notifier 4597 bl.length()=752 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f139a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 info.flags=0x17 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f139a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1 to cache LRU end 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f139a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:32.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(67 notify.5 [notify cookie 94550471089792 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f139a7fc640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f137c003ea0 con 0x7f14a809d040 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.821+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.822+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== watch-notify(notify_complete (2) cookie 94550471089792 notify 128849018896 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55fe3f3d7e60 con 0x7f14a809d040 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.822+0000 7f14bc040800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_sources= new_sources= 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.822+0000 7f14bc040800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential sources added=[] removed=[] 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.822+0000 7f14bc040800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): orig_dests= new_dests= 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.822+0000 7f14bc040800 20 handle_bi_update(): bucket=:myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1]): potential dests added=[] removed=[] 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.822+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3dd7a0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v30'13 uv13 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14bc040800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14bc040800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:32.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14bc040800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94550471040384 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3dd7a0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== watch-notify(notify (1) cookie 94550470813776 notify 128849018897 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55fe3f3d7e60 con 0x55fe3f38fcd0 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f1399ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018897 cookie 94550470813776 notifier 4597 bl.length()=567 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f1399ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f1399ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f1399ffb640 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) -- 0x7f12ec003430 con 0x55fe3f38fcd0 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(70 notify.4 [notify cookie 94550471040384 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== watch-notify(notify_complete (2) cookie 94550471040384 notify 128849018897 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55fe3f10d7f0 con 0x55fe3f38fcd0 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.823+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.837 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.824+0000 7f136d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.824+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94550470823248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3e0120 con 0x55fe3f38fcd0 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.825+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94550470823248] v30'151 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.825+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94550470796624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f3e0120 con 0x7f14b00123a0 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.826+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94550470796624] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.826+0000 7f14bc040800 20 remove_watcher() i=0 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.826+0000 7f14bc040800 2 removed watcher, disabling cache 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.827+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94550470802288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39a550 con 0x55fe3f38fcd0 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.828+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94550470802288] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.828+0000 7f14bc040800 20 remove_watcher() i=1 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.828+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94550470805840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39a550 con 0x55fe3f395f20 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.829+0000 7f14b74dd640 1 -- 172.21.15.63:0/66332407 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94550470805840] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14b0029070 con 0x55fe3f395f20 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.829+0000 7f14bc040800 20 remove_watcher() i=2 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.829+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94550470810704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39a550 con 0x7f14b00123a0 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.830+0000 7f14b6cdc640 1 -- 172.21.15.63:0/66332407 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94550470810704] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14a8075030 con 0x7f14b00123a0 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.830+0000 7f14bc040800 20 remove_watcher() i=3 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.830+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94550470813776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39dc50 con 0x55fe3f38fcd0 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.831+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94550470813776] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x55fe3f38fcd0 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.831+0000 7f14bc040800 20 remove_watcher() i=4 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.831+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94550470817968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39a550 con 0x7f14a809d040 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.832+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94550470817968] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.832+0000 7f14bc040800 20 remove_watcher() i=5 2024-04-23T19:10:32.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.832+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94550470820720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39e850 con 0x7f14a809d040 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.833+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94550470820720] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.833+0000 7f14bc040800 20 remove_watcher() i=6 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.833+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94550470814992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) -- 0x55fe3f39f8b0 con 0x7f14a809d040 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94550470814992] v30'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14ac02a040 con 0x7f14a809d040 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 20 remove_watcher() i=7 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f14a809d040 msgr2=0x7f14a808f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f14a809d040 0x7f14a808f980 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f38fcd0 msgr2=0x55fe3f392190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f38fcd0 0x55fe3f392190 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f14b00123a0 msgr2=0x7f14b0022830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f14b00123a0 0x7f14b0022830 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe3f395f20 msgr2=0x55fe3f398310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55fe3f395f20 0x55fe3f398310 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1380036e60 msgr2=0x7f1380039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.834+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1380036e60 0x7f1380039320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14a8006d80 tx=0x7f14a8087000 comp rx=0 tx=0).stop 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 msgr2=0x55fe3f3841d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14b64db640 1 -- 172.21.15.63:0/66332407 reap_dead start 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f3841d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f14b0015ac0 tx=0x7f14b004cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 shutdown_connections 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/66332407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f37cbe0 0x55fe3f3841d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 >> 172.21.15.63:0/66332407 conn(0x55fe3f378f40 msgr2=0x55fe3f3765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 shutdown_connections 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/66332407 wait complete. 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1390036e60 msgr2=0x7f1390039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1390036e60 0x7f1390039320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55fe3f361880 tx=0x7f14a8052500 comp rx=0 tx=0).stop 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f36f8c0 msgr2=0x55fe3f36fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f36f8c0 0x55fe3f36fce0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55fe3f355520 tx=0x7f14b00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 shutdown_connections 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1390036e60 0x7f1390039320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/3409768733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f36f8c0 0x55fe3f36fce0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 >> 172.21.15.63:0/3409768733 conn(0x55fe3f363960 msgr2=0x55fe3f363d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 shutdown_connections 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/3409768733 wait complete. 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe3f35eb30 msgr2=0x55fe3f360f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe3f35eb30 0x55fe3f360f20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f25b260 msgr2=0x55fe3f35b360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.835+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f25b260 0x55fe3f35b360 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f139c036eb0 msgr2=0x7f139c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f139c036eb0 0x7f139c039370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f14ac000f60 tx=0x7f14ac0012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 msgr2=0x55fe3f354120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f354120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f14a8052740 tx=0x7f14a8003020 comp rx=0 tx=0).stop 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 shutdown_connections 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55fe3f35eb30 0x55fe3f360f20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55fe3f25b260 0x55fe3f35b360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f139c036eb0 0x7f139c039370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 --2- 172.21.15.63:0/2919843286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3f25a000 0x55fe3f354120 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 >> 172.21.15.63:0/2919843286 conn(0x55fe3f255600 msgr2=0x55fe3f345790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 shutdown_connections 2024-04-23T19:10:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:32.836+0000 7f14bc040800 1 -- 172.21.15.63:0/2919843286 wait complete. 2024-04-23T19:10:32.849 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:32.856 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-23T19:10:33.118 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-04-23T19:10:33.124 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:48.125 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-23T19:10:48.126 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-23T19:10:48.126 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-23T19:10:48.177 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:48.177 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.177+0000 7f3c53a19800 1 Processor -- start 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.177+0000 7f3c53a19800 1 -- start start 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.178+0000 7f3c53a19800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5563604ad490 0x5563604af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.178+0000 7f3c53a19800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55636035c7f0 con 0x5563604a9000 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.178+0000 7f3c53a19800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55636034be20 con 0x5563604ad490 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.178+0000 7f3c50d17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5563604ad490 0x5563604af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.178+0000 7f3c51518640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5563604a9000 0x5563604a9420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49642/0 (socket says 172.21.15.63:49642) 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.178+0000 7f3c50d17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5563604ad490 0x5563604af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35464/0 (socket says 172.21.15.63:35464) 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.178+0000 7f3c51518640 1 -- 172.21.15.63:0/1347462874 learned_addr learned my addr 172.21.15.63:0/1347462874 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.179+0000 7f3c50d17640 1 -- 172.21.15.63:0/1347462874 >> v1:172.21.15.16:6789/0 conn(0x5563604a9000 legacy=0x5563604a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.179+0000 7f3c50d17640 1 -- 172.21.15.63:0/1347462874 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636036ab50 con 0x5563604ad490 2024-04-23T19:10:48.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c50d17640 1 --2- 172.21.15.63:0/1347462874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604ad490 0x5563604af880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800a470 tx=0x7f3c4801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c083b08f0d1d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c43fff640 1 -- 172.21.15.63:0/1347462874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48057070 con 0x5563604ad490 2024-04-23T19:10:48.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c43fff640 1 -- 172.21.15.63:0/1347462874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c48052570 con 0x5563604ad490 2024-04-23T19:10:48.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c43fff640 1 -- 172.21.15.63:0/1347462874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48052870 con 0x5563604ad490 2024-04-23T19:10:48.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c53a19800 1 -- 172.21.15.63:0/1347462874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604ad490 msgr2=0x5563604af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c53a19800 1 --2- 172.21.15.63:0/1347462874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604ad490 0x5563604af880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800a470 tx=0x7f3c4801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:48.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c53a19800 1 -- 172.21.15.63:0/1347462874 shutdown_connections 2024-04-23T19:10:48.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c53a19800 1 --2- 172.21.15.63:0/1347462874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604ad490 0x5563604af880 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c53a19800 1 -- 172.21.15.63:0/1347462874 >> 172.21.15.63:0/1347462874 conn(0x5563604a4600 msgr2=0x5563604a6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c53a19800 1 -- 172.21.15.63:0/1347462874 shutdown_connections 2024-04-23T19:10:48.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.181+0000 7f3c53a19800 1 -- 172.21.15.63:0/1347462874 wait complete. 2024-04-23T19:10:48.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.196+0000 7f3c53a19800 1 Processor -- start 2024-04-23T19:10:48.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.197+0000 7f3c53a19800 1 -- start start 2024-04-23T19:10:48.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.197+0000 7f3c53a19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x55636041d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.197+0000 7f3c53a19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563603f56f0 con 0x5563604a9000 2024-04-23T19:10:48.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.197+0000 7f3c51518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x55636041d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.197+0000 7f3c51518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x55636041d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35468/0 (socket says 172.21.15.63:35468) 2024-04-23T19:10:48.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.197+0000 7f3c51518640 1 -- 172.21.15.63:0/3169988566 learned_addr learned my addr 172.21.15.63:0/3169988566 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.197+0000 7f3c51518640 1 -- 172.21.15.63:0/3169988566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636059e0a0 con 0x5563604a9000 2024-04-23T19:10:48.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.198+0000 7f3c51518640 1 --2- 172.21.15.63:0/3169988566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x55636041d970 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3c44001e10 tx=0x7f3c4400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621b7806ea5db241 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.198+0000 7f3b3f606640 1 -- 172.21.15.63:0/3169988566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c44012020 con 0x5563604a9000 2024-04-23T19:10:48.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.198+0000 7f3b3f606640 1 -- 172.21.15.63:0/3169988566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4400e5a0 con 0x5563604a9000 2024-04-23T19:10:48.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.198+0000 7f3b3f606640 1 -- 172.21.15.63:0/3169988566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4400e8e0 con 0x5563604a9000 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 -- 172.21.15.63:0/3169988566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 msgr2=0x55636041d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3169988566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x55636041d970 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3c44001e10 tx=0x7f3c4400b780 comp rx=0 tx=0).stop 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 -- 172.21.15.63:0/3169988566 shutdown_connections 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3169988566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x55636041d970 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 -- 172.21.15.63:0/3169988566 >> 172.21.15.63:0/3169988566 conn(0x5563604a4600 msgr2=0x556360595b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 -- 172.21.15.63:0/3169988566 shutdown_connections 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 -- 172.21.15.63:0/3169988566 wait complete. 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 Processor -- start 2024-04-23T19:10:48.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 -- start start 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x5563605a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c53a19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563604b0150 con 0x5563604a9000 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c51518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x5563605a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c51518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x5563605a2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35476/0 (socket says 172.21.15.63:35476) 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.199+0000 7f3c51518640 1 -- 172.21.15.63:0/929691893 learned_addr learned my addr 172.21.15.63:0/929691893 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.200+0000 7f3c51518640 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563605a34b0 con 0x5563604a9000 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.200+0000 7f3c51518640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x5563605a2f70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400b7c0 tx=0x7f3c44000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.200+0000 7f3b3de03640 1 -- 172.21.15.63:0/929691893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c44012030 con 0x5563604a9000 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.200+0000 7f3b3de03640 1 -- 172.21.15.63:0/929691893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c44001470 con 0x5563604a9000 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.200+0000 7f3b3de03640 1 -- 172.21.15.63:0/929691893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4401a8d0 con 0x5563604a9000 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.200+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5563605a3740 con 0x5563604a9000 2024-04-23T19:10:48.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.200+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563605a63f0 con 0x5563604a9000 2024-04-23T19:10:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.201+0000 7f3b3de03640 1 -- 172.21.15.63:0/929691893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3c44015020 con 0x5563604a9000 2024-04-23T19:10:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.201+0000 7f3b3de03640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b38036e60 0x7f3b38039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.201+0000 7f3b3de03640 1 -- 172.21.15.63:0/929691893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f3c44020030 con 0x5563604a9000 2024-04-23T19:10:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.202+0000 7f3c50d17640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b38036e60 0x7f3b38039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.202+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563604ad490 0x5563605aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.202+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605aa780 con 0x5563604ad490 2024-04-23T19:10:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.202+0000 7f3c51d19640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563604ad490 0x5563605aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.202+0000 7f3c50d17640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b38036e60 0x7f3b38039320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c48054000 tx=0x7f3c4804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.202+0000 7f3c51d19640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563604ad490 0x5563605aa1b0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.203+0000 7f3c51d19640 1 -- 172.21.15.63:0/929691893 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c4c00a4a0 con 0x5563604ad490 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.203+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605a7760 con 0x5563604ad490 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.203+0000 7f3c51d19640 1 -- 172.21.15.63:0/929691893 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3c4c00a4a0 con 0x5563604ad490 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.203+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605ac580 con 0x5563604ad490 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.203+0000 7f3c51d19640 1 -- 172.21.15.63:0/929691893 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3c4c00a4a0 con 0x5563604ad490 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.203+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605ab8c0 con 0x5563604ad490 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.204+0000 7f3c51d19640 1 -- 172.21.15.63:0/929691893 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3c4c00a4a0 con 0x5563604ad490 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.204+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5563605ad930 0x5563605afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.204+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605b0260 con 0x5563605ad930 2024-04-23T19:10:48.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.204+0000 7f3c51518640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5563605ad930 0x5563605afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.205+0000 7f3c51518640 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5563605ad930 0x5563605afd20 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.205+0000 7f3c51518640 1 -- 172.21.15.63:0/929691893 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3c4402dcf0 con 0x5563605ad930 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.205+0000 7f3c53a19800 1 Processor -- start 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.205+0000 7f3c53a19800 1 -- start start 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3c53a19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605be660 0x5563605bea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3c53a19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55636039be30 con 0x5563605be660 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3c51d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605be660 0x5563605bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3c51d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605be660 0x5563605bea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35490/0 (socket says 172.21.15.63:35490) 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3c51d19640 1 -- 172.21.15.63:0/1710088768 learned_addr learned my addr 172.21.15.63:0/1710088768 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3c51d19640 1 -- 172.21.15.63:0/1710088768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563605befc0 con 0x5563605be660 2024-04-23T19:10:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3c51d19640 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605be660 0x5563605bea80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5563605aab60 tx=0x7f3c4c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.206+0000 7f3b36ffd640 1 -- 172.21.15.63:0/1710088768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c013440 con 0x5563605be660 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.207+0000 7f3b36ffd640 1 -- 172.21.15.63:0/1710088768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c0135e0 con 0x5563605be660 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.207+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5563605bfa30 con 0x5563605be660 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.207+0000 7f3b36ffd640 1 -- 172.21.15.63:0/1710088768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c0138c0 con 0x5563605be660 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.207+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563605bff30 con 0x5563605be660 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.207+0000 7f3b36ffd640 1 -- 172.21.15.63:0/1710088768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3c4c011040 con 0x5563605be660 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3b36ffd640 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b2c036eb0 0x7f3b2c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3b36ffd640 1 -- 172.21.15.63:0/1710088768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f3c4c01d070 con 0x5563605be660 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3c51518640 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b2c036eb0 0x7f3b2c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3c53a19800 1 Processor -- start 2024-04-23T19:10:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3c53a19800 1 -- start start 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3c51518640 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b2c036eb0 0x7f3b2c039370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5563605abc60 tx=0x7f3c4404b1c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3c53a19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x5563605cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3c53a19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55636059e3f0 con 0x5563605cb900 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.208+0000 7f3c51d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x5563605cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.209+0000 7f3c51d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x5563605cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35506/0 (socket says 172.21.15.63:35506) 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.209+0000 7f3c51d19640 1 -- 172.21.15.63:0/510049212 learned_addr learned my addr 172.21.15.63:0/510049212 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.209+0000 7f3c51d19640 1 -- 172.21.15.63:0/510049212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563605cc2f0 con 0x5563605cb900 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.209+0000 7f3c51d19640 1 --2- 172.21.15.63:0/510049212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x5563605cbd20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c04cc30 tx=0x7f3c4c0129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb194e6b08743909 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.209+0000 7f3b34ff9640 1 -- 172.21.15.63:0/510049212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c0123a0 con 0x5563605cb900 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.209+0000 7f3b34ff9640 1 -- 172.21.15.63:0/510049212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c04a020 con 0x5563605cb900 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.209+0000 7f3b34ff9640 1 -- 172.21.15.63:0/510049212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c001310 con 0x5563605cb900 2024-04-23T19:10:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 -- 172.21.15.63:0/510049212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 msgr2=0x5563605cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 --2- 172.21.15.63:0/510049212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x5563605cbd20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c04cc30 tx=0x7f3c4c0129d0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 -- 172.21.15.63:0/510049212 shutdown_connections 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 --2- 172.21.15.63:0/510049212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x5563605cbd20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 -- 172.21.15.63:0/510049212 >> 172.21.15.63:0/510049212 conn(0x5563605c7c60 msgr2=0x5563605c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 -- 172.21.15.63:0/510049212 shutdown_connections 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 -- 172.21.15.63:0/510049212 wait complete. 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 Processor -- start 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 -- start start 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x55636059b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.210+0000 7f3c53a19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563605cca60 con 0x5563605cb900 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3c51d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x55636059b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3c51d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x55636059b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35508/0 (socket says 172.21.15.63:35508) 2024-04-23T19:10:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 learned_addr learned my addr 172.21.15.63:0/3559179622 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636059a0c0 con 0x5563605cb900 2024-04-23T19:10:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3c51d19640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x55636059b9d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c001df0 tx=0x7f3c4c04cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3b22ffd640 1 -- 172.21.15.63:0/3559179622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c02aa60 con 0x5563605cb900 2024-04-23T19:10:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3b22ffd640 1 -- 172.21.15.63:0/3559179622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c02ac00 con 0x5563605cb900 2024-04-23T19:10:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55636059a350 con 0x5563605cb900 2024-04-23T19:10:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.211+0000 7f3b22ffd640 1 -- 172.21.15.63:0/3559179622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c02a350 con 0x5563605cb900 2024-04-23T19:10:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.212+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636059a850 con 0x5563605cb900 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.212+0000 7f3b22ffd640 1 -- 172.21.15.63:0/3559179622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3c4c045040 con 0x5563605cb900 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.212+0000 7f3b22ffd640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b1c036e60 0x7f3b1c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3b22ffd640 1 -- 172.21.15.63:0/3559179622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f3c4c043020 con 0x5563605cb900 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3c51518640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b1c036e60 0x7f3b1c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563605e0da0 0x5563605e3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e39b0 con 0x5563605e0da0 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3c50d17640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563605e0da0 0x5563605e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3c51518640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b1c036e60 0x7f3b1c039320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3c4404b2f0 tx=0x7f3c44033000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.213+0000 7f3c50d17640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563605e0da0 0x5563605e3260 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 realm 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e3d80 con 0x5563605e0da0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e4650 con 0x5563605e0da0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e49f0 con 0x5563605e0da0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.214+0000 7f3c53a19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e4e10 con 0x5563605e0da0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c53a19800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c53a19800 20 searching for the correct realm 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5563605e6e10 0x5563605e9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c51d19640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5563605e6e10 0x5563605e9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.215+0000 7f3c51d19640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5563605e6e10 0x5563605e9260 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51d19640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3c4c05b040 0x7f3c4c04ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c4c05b040 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51518640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3c4c05b040 0x7f3c4c04ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51518640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3c4c05b040 0x7f3c4c04ea10 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51518640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3c44017050 0x7f3c4403f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.216+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.217+0000 7f3c50d17640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3c44017050 0x7f3c4403f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.217+0000 7f3c50d17640 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3c44017050 0x7f3c4403f730 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.218+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.218+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.218+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.218+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.218+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.218+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c4c05b040 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.219+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.220+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.220+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.220+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.220+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.220+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.220+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c4c05b040 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.221+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.221+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.221+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.221+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.221+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.221+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.221+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.222+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.223+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.223+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.223+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.223+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c4c05b040 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.224+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.224+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.224+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.224+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c4c05b040 2024-04-23T19:10:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.224+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.224+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.224+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.225+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.225+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.225+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c4c05b040 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.225+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.225+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e6e10 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.225+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.225+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c44017050 2024-04-23T19:10:48.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x5563605e0da0 2024-04-23T19:10:48.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9930 con 0x7f3c4c05b040 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c53a19800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c53a19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c53a19800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c53a19800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.226+0000 7f3c53a19800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e58b0 con 0x5563605e0da0 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e6150 con 0x5563605e0da0 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e54f0 con 0x7f3c44017050 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 20 zone default found 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 4 Realm: () 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.227+0000 7f3c53a19800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:48.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 zonegroup default 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605e9e00 con 0x7f3c4c05b040 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605ec030 con 0x5563605e6e10 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605ec3d0 con 0x5563605e6e10 2024-04-23T19:10:48.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 started sync module instance, tier type = 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.228+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605eb4c0 con 0x7f3c4c05b040 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.230+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.230+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93885306943664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605eb120 con 0x7f3c4c05b040 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.230+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ed290 con 0x5563605e0da0 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.231+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93885306943664] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.231+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.231+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93885306949088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ed6b0 con 0x5563605e0da0 2024-04-23T19:10:48.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.231+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ee070 con 0x5563605e6e10 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.232+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93885306949088] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.233+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.233+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93885306952640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ee410 con 0x5563605e6e10 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.233+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605eeee0 con 0x7f3c4c05b040 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.234+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93885306952640] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.234+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.234+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93885306956240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ef300 con 0x7f3c4c05b040 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.234+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605efd20 con 0x5563605e0da0 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.235+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.235+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93885306959888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f05c0 con 0x5563605e0da0 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.235+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f10a0 con 0x7f3c44017050 2024-04-23T19:10:48.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.235+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93885306956240] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.236+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93885306959888] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.236+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.236+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93885306964912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f0040 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.237+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f1940 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.238+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93885306964912] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.238+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.238+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93885306950144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f0040 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.238+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f1f00 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93885306950144] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93885306968560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f1ec0 con 0x7f3c44017050 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 20 add_watcher() i=0 2024-04-23T19:10:48.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 20 add_watcher() i=1 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 20 add_watcher() i=2 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 20 add_watcher() i=3 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 20 add_watcher() i=4 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 20 add_watcher() i=5 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.239+0000 7f3c53a19800 20 add_watcher() i=6 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.240+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93885306968560] v32'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.240+0000 7f3c53a19800 20 add_watcher() i=7 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.240+0000 7f3c53a19800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.240+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5563605f52f0 con 0x5563605e0da0 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.241+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.241+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93885306966912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f27e0 con 0x5563605e0da0 2024-04-23T19:10:48.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.242+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93885306966912] v32'152 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.242+0000 7f3b077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:48.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.242+0000 7f3c53a19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:48.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.242+0000 7f3c53a19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:48.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.242+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fd9d0 con 0x5563605e0da0 2024-04-23T19:10:48.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.243+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.243+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fdf40 con 0x7f3c44017050 2024-04-23T19:10:48.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.245+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.245+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fe3f0 con 0x5563605e6e10 2024-04-23T19:10:48.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.246+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.246+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fe3f0 con 0x5563605e0da0 2024-04-23T19:10:48.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.247+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'153 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.247+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fe850 con 0x7f3c4c05b040 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.248+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'44 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.248+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fed00 con 0x5563605e6e10 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.249+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'29 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.249+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fed00 con 0x5563605e0da0 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.250+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.250+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fed00 con 0x7f3c44017050 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.251+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'29 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.251+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fed00 con 0x7f3c4c05b040 2024-04-23T19:10:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.253+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'19 uv18 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.253+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605feed0 con 0x5563605e6e10 2024-04-23T19:10:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.254+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'24 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.254+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605feed0 con 0x7f3c4c05b040 2024-04-23T19:10:48.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.255+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'45 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.255+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605feed0 con 0x7f3c44017050 2024-04-23T19:10:48.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.256+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'20 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.256+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605feed0 con 0x7f3c4c05b040 2024-04-23T19:10:48.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.257+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.257+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x7f3c44017050 2024-04-23T19:10:48.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.258+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'35 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.258+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e0da0 2024-04-23T19:10:48.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.259+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'154 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.259+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.261+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'39 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.261+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x7f3c4c05b040 2024-04-23T19:10:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.262+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'46 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.262+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.262+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.263+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e0da0 2024-04-23T19:10:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.264+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.264+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e0da0 2024-04-23T19:10:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.265+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'155 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.265+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.266+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'40 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.266+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x7f3c4c05b040 2024-04-23T19:10:48.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.267+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'28 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.267+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.268+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.268+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.269+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.269+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.270+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.270+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.271+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'30 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.271+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x7f3c44017050 2024-04-23T19:10:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.273+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.273+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.274+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.274+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x7f3c44017050 2024-04-23T19:10:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.275+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'30 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.275+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x5563605e6e10 2024-04-23T19:10:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.276+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'41 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.276+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x7f3c44017050 2024-04-23T19:10:48.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.277+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'36 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.277+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ff290 con 0x7f3c4c05b040 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.278+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3b067e4640 20 reqs_thread_entry: start 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3c53a19800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3c53a19800 20 Filter name: none 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3a77fff640 20 reqs_thread_entry: start 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e0da0 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x5563605e0da0 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.279+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e0da0 2024-04-23T19:10:48.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.280+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.280+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c44017050 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.280+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.280+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x7f3c44017050 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.280+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.280+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c44017050 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.281+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.281+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e6e10 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.281+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.281+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x5563605e6e10 2024-04-23T19:10:48.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.281+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.281+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e6e10 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e0da0 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv13 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x5563605e0da0 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv13 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e0da0 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv13 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.282+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c4c05b040 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.283+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.283+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x7f3c4c05b040 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.283+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv9 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.283+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c4c05b040 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.283+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.283+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e6e10 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x5563605e6e10 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv8 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e6e10 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e0da0 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.284+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x5563605e0da0 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.285+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.285+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x5563605e0da0 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.285+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.285+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c44017050 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.285+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.285+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x7f3c44017050 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.286+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv8 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.286+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c44017050 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.286+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.286+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c4c05b040 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.286+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv18 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.286+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556360629b60 con 0x7f3c4c05b040 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.286+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv18 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.287+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062ab70 con 0x7f3c4c05b040 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.287+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(120 gc.8 [call out=1701b] v0'0 uv18 ondisk = 0) ==== 148+0+1701 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.287+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.287+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.287+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.287+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv9 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.287+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.288+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.288+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.288+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.288+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c4c05b040 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.288+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.288+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.289+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.289+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.289+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.289+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c44017050 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.289+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.289+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.290+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.290+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.290+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.290+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c4c05b040 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.290+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.290+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.291+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.291+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.291+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.291+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c44017050 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.291+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv12 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.291+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.292+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.292+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e0da0 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.292+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.292+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e0da0 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.292+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv15 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.292+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e0da0 2024-04-23T19:10:48.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 36 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 37 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.293+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.294+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.294+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c4c05b040 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.294+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv13 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.294+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.294+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 34 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.294+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.295+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 38 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.295+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.295+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 39 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.295+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.295+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 40 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.295+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e0da0 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e0da0 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e0da0 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e0da0 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.296+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e0da0 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e0da0 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 41 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 42 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.297+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 43 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 35 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c4c05b040 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 36 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 37 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.298+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 44 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 45 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 46 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 47 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.299+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 48 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 49 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 50 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 51 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.300+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 52 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 53 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 54 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv4 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 55 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.301+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.302+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.302+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c44017050 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.302+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.302+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 56 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 57 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 58 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.303+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.304+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.304+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c44017050 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.304+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv4 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.304+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.305+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.305+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.305+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 59 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.305+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x5563605e6e10 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.306+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 60 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv9 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.306+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x5563605e6e10 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.306+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 61 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.306+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.306+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv6 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.306+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c44017050 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 42 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv6 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c44017050 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 43 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv6 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 38 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062a7c0 con 0x7f3c4c05b040 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 39 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.307+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55636062b7d0 con 0x7f3c4c05b040 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.308+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 40 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.308+0000 7f3b077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.309+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93885306966912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605fbcf0 con 0x5563605e0da0 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.310+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 93885306966912] v32'156 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.310+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93885306943664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f1380 con 0x7f3c4c05b040 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.311+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 41 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 93885306943664] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.311+0000 7f3c53a19800 20 remove_watcher() i=0 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.311+0000 7f3c53a19800 2 removed watcher, disabling cache 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.311+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93885306949088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f1380 con 0x5563605e0da0 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.312+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 93885306949088] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.312+0000 7f3c53a19800 20 remove_watcher() i=1 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.312+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93885306952640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ecde0 con 0x5563605e6e10 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.313+0000 7f3c51d19640 1 -- 172.21.15.63:0/3559179622 <== osd.3 v2:172.21.15.63:6816/1289558434 62 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 93885306952640] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4c04cd70 con 0x5563605e6e10 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.313+0000 7f3c53a19800 20 remove_watcher() i=2 2024-04-23T19:10:48.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.313+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93885306956240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f1380 con 0x7f3c4c05b040 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.314+0000 7f3c51518640 1 -- 172.21.15.63:0/3559179622 <== osd.2 v2:172.21.15.63:6808/753864785 42 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 93885306956240] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4402dcf0 con 0x7f3c4c05b040 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.314+0000 7f3c53a19800 20 remove_watcher() i=3 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.314+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93885306959888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605edbc0 con 0x5563605e0da0 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.315+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 93885306959888] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x5563605e0da0 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.315+0000 7f3c53a19800 20 remove_watcher() i=4 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.315+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93885306964912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605ecde0 con 0x7f3c44017050 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.316+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 44 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 93885306964912] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.316+0000 7f3c53a19800 20 remove_watcher() i=5 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.316+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93885306950144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605f1380 con 0x7f3c44017050 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.317+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 45 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 93885306950144] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.317+0000 7f3c53a19800 20 remove_watcher() i=6 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.317+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93885306968560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5563605edbc0 con 0x7f3c44017050 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 <== osd.0 v2:172.21.15.16:6800/2262294208 46 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 93885306968560] v32'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4805a040 con 0x7f3c44017050 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 20 remove_watcher() i=7 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3c44017050 msgr2=0x7f3c4403f730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3c44017050 0x7f3c4403f730 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563605e0da0 msgr2=0x5563605e3260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563605e0da0 0x5563605e3260 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3c4c05b040 msgr2=0x7f3c4c04ea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f3c4c05b040 0x7f3c4c04ea10 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5563605e6e10 msgr2=0x5563605e9260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.318+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5563605e6e10 0x5563605e9260 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b1c036e60 msgr2=0x7f3b1c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b1c036e60 0x7f3b1c039320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3c4404b2f0 tx=0x7f3c44033000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 msgr2=0x55636059b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c50d17640 1 -- 172.21.15.63:0/3559179622 reap_dead start 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x55636059b9d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c001df0 tx=0x7f3c4c04cbc0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 shutdown_connections 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/3559179622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605cb900 0x55636059b9d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 >> 172.21.15.63:0/3559179622 conn(0x5563605c7c60 msgr2=0x5563605c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 shutdown_connections 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/3559179622 wait complete. 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b2c036eb0 msgr2=0x7f3b2c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b2c036eb0 0x7f3b2c039370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5563605abc60 tx=0x7f3c4404b1c0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605be660 msgr2=0x5563605bea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605be660 0x5563605bea80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5563605aab60 tx=0x7f3c4c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 shutdown_connections 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b2c036eb0 0x7f3b2c039370 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/1710088768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563605be660 0x5563605bea80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 >> 172.21.15.63:0/1710088768 conn(0x5563605b2700 msgr2=0x5563605b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 shutdown_connections 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/1710088768 wait complete. 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5563605ad930 msgr2=0x5563605afd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5563605ad930 0x5563605afd20 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563604ad490 msgr2=0x5563605aa1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.319+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563604ad490 0x5563605aa1b0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b38036e60 msgr2=0x7f3b38039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b38036e60 0x7f3b38039320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c48054000 tx=0x7f3c4804e000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 msgr2=0x5563605a2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x5563605a2f70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400b7c0 tx=0x7f3c44000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 shutdown_connections 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5563605ad930 0x5563605afd20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5563604ad490 0x5563605aa1b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3b38036e60 0x7f3b38039320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 --2- 172.21.15.63:0/929691893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563604a9000 0x5563605a2f70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 >> 172.21.15.63:0/929691893 conn(0x5563604a4600 msgr2=0x556360594390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 shutdown_connections 2024-04-23T19:10:48.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.320+0000 7f3c53a19800 1 -- 172.21.15.63:0/929691893 wait complete. 2024-04-23T19:10:48.331 INFO:teuthology.orchestra.run.smithi063.stdout:[{"tag":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.6599408857962759447\u0000","time":"2024-04-23T19:10:48.123981+0000","objs":[{"pool":"default.rgw.buckets.data","oid":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_1","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_2","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_3","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_4","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_5","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_6","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_7","key":"","instance":""}]}] 2024-04-23T19:10:48.331 DEBUG:tasks.util.rgw: json result: [{'tag': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.6599408857962759447\x00', 'time': '2024-04-23T19:10:48.123981+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_7', 'key': '', 'instance': ''}]}] 2024-04-23T19:10:48.331 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-04-23T19:10:48.332 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-04-23T19:10:48.332 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-04-23T19:10:48.373 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:48.374 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:48.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.374+0000 7f9843830800 1 Processor -- start 2024-04-23T19:10:48.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.374+0000 7f9843830800 1 -- start start 2024-04-23T19:10:48.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.374+0000 7f9843830800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557c4a6dea20 0x557c4a6dee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.374+0000 7f9843830800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557c4a5927f0 con 0x557c4a6df380 2024-04-23T19:10:48.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.374+0000 7f9843830800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x557c4a581e20 con 0x557c4a6dea20 2024-04-23T19:10:48.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.375+0000 7f983ea92640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557c4a6dea20 0x557c4a6dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.375+0000 7f983ea92640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557c4a6dea20 0x557c4a6dee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35520/0 (socket says 172.21.15.63:35520) 2024-04-23T19:10:48.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.375+0000 7f983ea92640 1 -- 172.21.15.63:0/3752013672 learned_addr learned my addr 172.21.15.63:0/3752013672 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.375+0000 7f983e291640 1 --1- 172.21.15.63:0/3752013672 >> v1:172.21.15.16:6789/0 conn(0x557c4a6df380 0x557c4a6df830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49650/0 (socket says 172.21.15.63:49650) 2024-04-23T19:10:48.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.375+0000 7f983ea92640 1 -- 172.21.15.63:0/3752013672 >> v1:172.21.15.16:6789/0 conn(0x557c4a6df380 legacy=0x557c4a6df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.375+0000 7f983ea92640 1 -- 172.21.15.63:0/3752013672 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4a5a0b50 con 0x557c4a6dea20 2024-04-23T19:10:48.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.377+0000 7f983ea92640 1 --2- 172.21.15.63:0/3752013672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a6dee40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9830005fd0 tx=0x7f983001cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef447f29b6e2c60f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.377+0000 7f983da90640 1 -- 172.21.15.63:0/3752013672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830052540 con 0x557c4a6dea20 2024-04-23T19:10:48.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.377+0000 7f983da90640 1 -- 172.21.15.63:0/3752013672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983004e070 con 0x557c4a6dea20 2024-04-23T19:10:48.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.377+0000 7f983da90640 1 -- 172.21.15.63:0/3752013672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830052890 con 0x557c4a6dea20 2024-04-23T19:10:48.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.378+0000 7f9843830800 1 -- 172.21.15.63:0/3752013672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 msgr2=0x557c4a6dee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.378+0000 7f9843830800 1 --2- 172.21.15.63:0/3752013672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a6dee40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9830005fd0 tx=0x7f983001cd80 comp rx=0 tx=0).stop 2024-04-23T19:10:48.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.378+0000 7f9843830800 1 -- 172.21.15.63:0/3752013672 shutdown_connections 2024-04-23T19:10:48.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.378+0000 7f9843830800 1 --2- 172.21.15.63:0/3752013672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a6dee40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.378+0000 7f9843830800 1 -- 172.21.15.63:0/3752013672 >> 172.21.15.63:0/3752013672 conn(0x557c4a6da620 msgr2=0x557c4a6dca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.378+0000 7f9843830800 1 -- 172.21.15.63:0/3752013672 shutdown_connections 2024-04-23T19:10:48.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.378+0000 7f9843830800 1 -- 172.21.15.63:0/3752013672 wait complete. 2024-04-23T19:10:48.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.390+0000 7f9843830800 1 Processor -- start 2024-04-23T19:10:48.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.390+0000 7f9843830800 1 -- start start 2024-04-23T19:10:48.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.390+0000 7f9843830800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.390+0000 7f9843830800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c4a62b6e0 con 0x557c4a6dea20 2024-04-23T19:10:48.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.391+0000 7f983ea92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.391+0000 7f983ea92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7d3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35526/0 (socket says 172.21.15.63:35526) 2024-04-23T19:10:48.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.391+0000 7f983ea92640 1 -- 172.21.15.63:0/3198265506 learned_addr learned my addr 172.21.15.63:0/3198265506 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.391+0000 7f983ea92640 1 -- 172.21.15.63:0/3198265506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4a7d3bf0 con 0x557c4a6dea20 2024-04-23T19:10:48.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.392+0000 7f983ea92640 1 --2- 172.21.15.63:0/3198265506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7d3620 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f983000a190 tx=0x7f983004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154f2eb62dcb3bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.392+0000 7f972ce09640 1 -- 172.21.15.63:0/3198265506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830052410 con 0x557c4a6dea20 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.392+0000 7f972ce09640 1 -- 172.21.15.63:0/3198265506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9830062070 con 0x557c4a6dea20 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.392+0000 7f972ce09640 1 -- 172.21.15.63:0/3198265506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830052700 con 0x557c4a6dea20 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.392+0000 7f9843830800 1 -- 172.21.15.63:0/3198265506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 msgr2=0x557c4a7d3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.392+0000 7f9843830800 1 --2- 172.21.15.63:0/3198265506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7d3620 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f983000a190 tx=0x7f983004c000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 -- 172.21.15.63:0/3198265506 shutdown_connections 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 --2- 172.21.15.63:0/3198265506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7d3620 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 -- 172.21.15.63:0/3198265506 >> 172.21.15.63:0/3198265506 conn(0x557c4a6da620 msgr2=0x557c4a7cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 -- 172.21.15.63:0/3198265506 shutdown_connections 2024-04-23T19:10:48.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 -- 172.21.15.63:0/3198265506 wait complete. 2024-04-23T19:10:48.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 Processor -- start 2024-04-23T19:10:48.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 -- start start 2024-04-23T19:10:48.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f9843830800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c4a6e01e0 con 0x557c4a6dea20 2024-04-23T19:10:48.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f983ea92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.393+0000 7f983ea92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7db0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35528/0 (socket says 172.21.15.63:35528) 2024-04-23T19:10:48.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f983ea92640 1 -- 172.21.15.63:0/3945257574 learned_addr learned my addr 172.21.15.63:0/3945257574 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f983ea92640 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4a7db600 con 0x557c4a6dea20 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f983ea92640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7db0c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9830000f80 tx=0x7f9830003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f972b606640 1 -- 172.21.15.63:0/3945257574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983005d630 con 0x557c4a6dea20 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f972b606640 1 -- 172.21.15.63:0/3945257574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9830058040 con 0x557c4a6dea20 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c4a7db890 con 0x557c4a6dea20 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f972b606640 1 -- 172.21.15.63:0/3945257574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983005d920 con 0x557c4a6dea20 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.394+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c4a7de540 con 0x557c4a6dea20 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.395+0000 7f972b606640 1 -- 172.21.15.63:0/3945257574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9830050070 con 0x557c4a6dea20 2024-04-23T19:10:48.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.395+0000 7f972b606640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9724036eb0 0x7f9724039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.395+0000 7f972b606640 1 -- 172.21.15.63:0/3945257574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f983008edd0 con 0x557c4a6dea20 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.395+0000 7f983e291640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9724036eb0 0x7f9724039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.396+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a6df380 0x557c4a7e2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.396+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a7e2840 con 0x557c4a6df380 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.396+0000 7f983f293640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a6df380 0x557c4a7e2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.396+0000 7f983e291640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9724036eb0 0x7f9724039370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9828000f60 tx=0x7f9828001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.396+0000 7f983f293640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a6df380 0x557c4a7e2300 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.396+0000 7f983f293640 1 -- 172.21.15.63:0/3945257574 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f98380083e0 con 0x557c4a6df380 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.396+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a7df8b0 con 0x557c4a6df380 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.397+0000 7f983f293640 1 -- 172.21.15.63:0/3945257574 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f98380083e0 con 0x557c4a6df380 2024-04-23T19:10:48.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.397+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a7e4620 con 0x557c4a6df380 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.397+0000 7f983f293640 1 -- 172.21.15.63:0/3945257574 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f98380083e0 con 0x557c4a6df380 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.397+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a7e3960 con 0x557c4a6df380 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.397+0000 7f983f293640 1 -- 172.21.15.63:0/3945257574 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f98380083e0 con 0x557c4a6df380 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.397+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557c4a7e59d0 0x557c4a7e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.397+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a7e8340 con 0x557c4a7e59d0 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.398+0000 7f983ea92640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557c4a7e59d0 0x557c4a7e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.398+0000 7f983ea92640 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557c4a7e59d0 0x557c4a7e7dc0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.398+0000 7f983ea92640 1 -- 172.21.15.63:0/3945257574 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9830077020 con 0x557c4a7e59d0 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f9843830800 1 Processor -- start 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f9843830800 1 -- start start 2024-04-23T19:10:48.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f9843830800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a7f6780 0x557c4a7f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f9843830800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c4a5d1e30 con 0x557c4a7f6780 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f983f293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a7f6780 0x557c4a7f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f983f293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a7f6780 0x557c4a7f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35534/0 (socket says 172.21.15.63:35534) 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f983f293640 1 -- 172.21.15.63:0/2679943927 learned_addr learned my addr 172.21.15.63:0/2679943927 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f983f293640 1 -- 172.21.15.63:0/2679943927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4a7f70e0 con 0x557c4a7f6780 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.399+0000 7f983f293640 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a7f6780 0x557c4a7f6ba0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x557c4a7dc4c0 tx=0x7f983800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.400+0000 7f9728e01640 1 -- 172.21.15.63:0/2679943927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983801c070 con 0x557c4a7f6780 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.400+0000 7f9728e01640 1 -- 172.21.15.63:0/2679943927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983800bdc0 con 0x557c4a7f6780 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.400+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c4a7f7b50 con 0x557c4a7f6780 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.400+0000 7f9728e01640 1 -- 172.21.15.63:0/2679943927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838018540 con 0x557c4a7f6780 2024-04-23T19:10:48.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.400+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c4a7f8050 con 0x557c4a7f6780 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.400+0000 7f9728e01640 1 -- 172.21.15.63:0/2679943927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f983801f070 con 0x557c4a7f6780 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f9728e01640 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9718036e60 0x7f9718039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f9728e01640 1 -- 172.21.15.63:0/2679943927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f983804e9e0 con 0x557c4a7f6780 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f983ea92640 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9718036e60 0x7f9718039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f9843830800 1 Processor -- start 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f9843830800 1 -- start start 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f983ea92640 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9718036e60 0x7f9718039320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f983007a040 tx=0x7f983008d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f9843830800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a803fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f9843830800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c4a7d3eb0 con 0x557c4a803b80 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f983f293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a803fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f983f293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a803fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35540/0 (socket says 172.21.15.63:35540) 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.401+0000 7f983f293640 1 -- 172.21.15.63:0/2868549502 learned_addr learned my addr 172.21.15.63:0/2868549502 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f983f293640 1 -- 172.21.15.63:0/2868549502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4a804570 con 0x557c4a803b80 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f983f293640 1 --2- 172.21.15.63:0/2868549502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a803fa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f98380174e0 tx=0x7f98380175d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6755ad5ab67b1de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f97227fc640 1 -- 172.21.15.63:0/2868549502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983804a070 con 0x557c4a803b80 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f97227fc640 1 -- 172.21.15.63:0/2868549502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9838029c70 con 0x557c4a803b80 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f97227fc640 1 -- 172.21.15.63:0/2868549502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838026040 con 0x557c4a803b80 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f9843830800 1 -- 172.21.15.63:0/2868549502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 msgr2=0x557c4a803fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f9843830800 1 --2- 172.21.15.63:0/2868549502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a803fa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f98380174e0 tx=0x7f98380175d0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f9843830800 1 -- 172.21.15.63:0/2868549502 shutdown_connections 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f9843830800 1 --2- 172.21.15.63:0/2868549502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a803fa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f9843830800 1 -- 172.21.15.63:0/2868549502 >> 172.21.15.63:0/2868549502 conn(0x557c4a7ffe00 msgr2=0x557c4a7fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f9843830800 1 -- 172.21.15.63:0/2868549502 shutdown_connections 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.402+0000 7f9843830800 1 -- 172.21.15.63:0/2868549502 wait complete. 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f9843830800 1 Processor -- start 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f9843830800 1 -- start start 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f9843830800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a80b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f9843830800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c4a804ca0 con 0x557c4a803b80 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f983f293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a80b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f983f293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a80b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35546/0 (socket says 172.21.15.63:35546) 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 learned_addr learned my addr 172.21.15.63:0/2485669759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4a80b5e0 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.403+0000 7f983f293640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a80b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f98380174e0 tx=0x7f983804ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.404+0000 7f9720ff9640 1 -- 172.21.15.63:0/2485669759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838026030 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.404+0000 7f9720ff9640 1 -- 172.21.15.63:0/2485669759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9838029e20 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.404+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c4a80b870 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.404+0000 7f9720ff9640 1 -- 172.21.15.63:0/2485669759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983804ce60 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.404+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c4a80e520 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.404+0000 7f9720ff9640 1 -- 172.21.15.63:0/2485669759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9838018730 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f9720ff9640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9708036e60 0x7f9708039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f9720ff9640 1 -- 172.21.15.63:0/2485669759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9838036d50 con 0x557c4a803b80 2024-04-23T19:10:48.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f983ea92640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9708036e60 0x7f9708039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f983ea92640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9708036e60 0x7f9708039320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9830051fe0 tx=0x7f9830092000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a816bf0 0x557c4a8190b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a819800 con 0x557c4a816bf0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.405+0000 7f983e291640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a816bf0 0x557c4a8190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f983e291640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a816bf0 0x557c4a8190b0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 20 realm 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a816040 con 0x557c4a816bf0 2024-04-23T19:10:48.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81a2f0 con 0x557c4a816bf0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.406+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81abc0 con 0x557c4a816bf0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f9843830800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81af60 con 0x557c4a816bf0 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f9843830800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f9843830800 20 searching for the correct realm 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557c4a81ce40 0x557c4a81f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f983f293640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557c4a81ce40 0x557c4a81f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.407+0000 7f983f293640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557c4a81ce40 0x557c4a81f270 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983f293640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9838021070 0x7f9838021450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f9838021070 2024-04-23T19:10:48.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983ea92640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9838021070 0x7f9838021450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983ea92640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9838021070 0x7f9838021450 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9830077020 con 0x7f9838021070 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983ea92640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f98300a6040 0x7f98300540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.408+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.409+0000 7f983e291640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f98300a6040 0x7f98300540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.409+0000 7f983e291640 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f98300a6040 0x7f98300540c0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.409+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.409+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.410+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f9838021070 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9830077020 con 0x7f9838021070 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.411+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f9838021070 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9830077020 con 0x7f9838021070 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.412+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.413+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.413+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.413+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.413+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.413+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.413+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.414+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f9838021070 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9830077020 con 0x7f9838021070 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f9838021070 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9830077020 con 0x7f9838021070 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.415+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.416+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.416+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f9838021070 2024-04-23T19:10:48.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.416+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.416+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a81ce40 2024-04-23T19:10:48.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f98300a6040 2024-04-23T19:10:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x557c4a816bf0 2024-04-23T19:10:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81f940 con 0x7f9838021070 2024-04-23T19:10:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.417+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81b8b0 con 0x557c4a816bf0 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81c150 con 0x557c4a816bf0 2024-04-23T19:10:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81fdc0 con 0x7f98300a6040 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 zone default found 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 4 Realm: () 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 zonegroup default 2024-04-23T19:10:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.418+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a81bda0 con 0x7f9838021070 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a8221a0 con 0x557c4a81ce40 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a822540 con 0x557c4a81ce40 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 20 started sync module instance, tier type = 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.419+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a8215e0 con 0x7f9838021070 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.420+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.420+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93992314343728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a822930 con 0x7f9838021070 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.420+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a823180 con 0x557c4a816bf0 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.421+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.421+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93992314351072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a823810 con 0x557c4a816bf0 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.421+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93992314343728] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.421+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a821a40 con 0x557c4a81ce40 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.422+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93992314351072] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.423+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.423+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93992314352224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a8237d0 con 0x557c4a81ce40 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.423+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a8248d0 con 0x7f9838021070 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.424+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.424+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93992314356768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a825120 con 0x7f9838021070 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.424+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a825c10 con 0x557c4a816bf0 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.424+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93992314352224] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.425+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.425+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93992314361568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a826000 con 0x557c4a816bf0 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.425+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a826a40 con 0x7f98300a6040 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.425+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93992314356768] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.426+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.426+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93992314365200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a826a00 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.426+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a827450 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.426+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93992314361568] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.427+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93992314365200] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.427+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.427+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93992314358896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a826fa0 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.427+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a827cf0 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93992314358896] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93992314355200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a827af0 con 0x7f98300a6040 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 20 add_watcher() i=0 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 20 add_watcher() i=1 2024-04-23T19:10:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 20 add_watcher() i=2 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 20 add_watcher() i=3 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 20 add_watcher() i=4 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 20 add_watcher() i=5 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.428+0000 7f9843830800 20 add_watcher() i=6 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.429+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93992314355200] v32'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.429+0000 7f9843830800 20 add_watcher() i=7 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.429+0000 7f9843830800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.429+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a82ac90 con 0x557c4a816bf0 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.430+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.430+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93992314381888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a827a10 con 0x557c4a816bf0 2024-04-23T19:10:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.431+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93992314381888] v32'157 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.431+0000 7f96f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.431+0000 7f9843830800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.431+0000 7f9843830800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.431+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a833620 con 0x557c4a816bf0 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.432+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'26 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.432+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a833cf0 con 0x7f98300a6040 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.433+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.433+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834110 con 0x557c4a81ce40 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.434+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.434+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a8340e0 con 0x557c4a816bf0 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.435+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'158 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.435+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834100 con 0x7f9838021070 2024-04-23T19:10:48.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.436+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'47 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.436+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834c00 con 0x557c4a81ce40 2024-04-23T19:10:48.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.437+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'31 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.437+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834c00 con 0x557c4a816bf0 2024-04-23T19:10:48.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.438+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.438+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834c00 con 0x7f98300a6040 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.439+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'31 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.439+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834c00 con 0x7f9838021070 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.440+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'20 uv18 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.440+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834dd0 con 0x557c4a81ce40 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.441+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'25 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.441+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a834dd0 con 0x7f9838021070 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.442+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'48 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.442+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835190 con 0x7f98300a6040 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.444+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'21 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.444+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835190 con 0x7f9838021070 2024-04-23T19:10:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.444+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.445+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x7f98300a6040 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.446+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'37 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.446+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a816bf0 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.447+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'159 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.447+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a81ce40 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.448+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'42 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.448+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x7f9838021070 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.449+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'49 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.449+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a81ce40 2024-04-23T19:10:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.450+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.450+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a816bf0 2024-04-23T19:10:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.451+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.451+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a816bf0 2024-04-23T19:10:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.453+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'160 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.453+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a81ce40 2024-04-23T19:10:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.454+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'43 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.454+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x7f9838021070 2024-04-23T19:10:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.455+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'29 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.455+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a81ce40 2024-04-23T19:10:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.456+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.456+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a81ce40 2024-04-23T19:10:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.457+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.457+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a81ce40 2024-04-23T19:10:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.458+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.458+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x557c4a81ce40 2024-04-23T19:10:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.459+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'32 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.459+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835220 con 0x7f98300a6040 2024-04-23T19:10:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.460+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.460+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835190 con 0x557c4a81ce40 2024-04-23T19:10:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.461+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.461+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835190 con 0x7f98300a6040 2024-04-23T19:10:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.462+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'32 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.462+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835190 con 0x557c4a81ce40 2024-04-23T19:10:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.463+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'44 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.463+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835190 con 0x7f98300a6040 2024-04-23T19:10:48.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.464+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'38 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.464+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a835190 con 0x7f9838021070 2024-04-23T19:10:48.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.465+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.465+0000 7f966ffff640 20 reqs_thread_entry: start 2024-04-23T19:10:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.466+0000 7f9843830800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.466+0000 7f9843830800 20 Filter name: none 2024-04-23T19:10:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.466+0000 7f966dffb640 20 reqs_thread_entry: start 2024-04-23T19:10:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.466+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-04-23T19:10:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.466+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:94 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.467+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(94 gc.17 [call] v32'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.467+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:95 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.467+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(95 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.467+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.467+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:96 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.467+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(96 gc.17 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.467+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.468+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(97 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.468+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.468+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.468+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(98 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.468+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.468+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:99 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.469+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(99 gc.17 [call] v32'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.469+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.469+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.470+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(100 gc.18 [call] v32'28 uv28 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.470+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.470+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(101 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.470+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.470+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:102 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a816bf0 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(102 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(103 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:104 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(104 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.471+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.472+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(105 gc.18 [call] v32'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.472+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-04-23T19:10:48.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.472+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:106 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.473+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(106 gc.19 [call] v32'161 uv161 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.473+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:107 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.473+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(107 gc.19 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.473+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.473+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:108 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(108 gc.19 [call out=24b] v0'0 uv161 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:109 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(109 gc.19 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:110 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(110 gc.19 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.474+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:111 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.475+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(111 gc.19 [call] v32'162 uv162 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x557c4a816bf0 2024-04-23T19:10:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.475+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-04-23T19:10:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.475+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:112 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.476+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(112 gc.20 [call] v32'45 uv45 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.476+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:113 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.476+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(113 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.476+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.476+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:114 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== osd_op_reply(114 gc.20 [call out=24b] v0'0 uv45 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:115 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== osd_op_reply(115 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:116 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 36 ==== osd_op_reply(116 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.477+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:117 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.478+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 37 ==== osd_op_reply(117 gc.20 [call] v32'46 uv46 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.478+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-04-23T19:10:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.478+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:118 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.479+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(118 gc.21 [call] v32'30 uv30 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.479+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:119 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.479+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(119 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.479+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.479+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:120 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f9838021070 2024-04-23T19:10:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(120 gc.21 [call out=24b] v0'0 uv30 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:121 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(121 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:122 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(122 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.480+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:123 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.481+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(123 gc.21 [call] v32'31 uv31 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.481+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-04-23T19:10:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.481+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:124 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.482+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 38 ==== osd_op_reply(124 gc.22 [call] v32'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.482+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:125 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 39 ==== osd_op_reply(125 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:126 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 40 ==== osd_op_reply(126 gc.22 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:127 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 41 ==== osd_op_reply(127 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:128 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 42 ==== osd_op_reply(128 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.483+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:129 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.484+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 43 ==== osd_op_reply(129 gc.22 [call] v32'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.484+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.484+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:130 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.485+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 44 ==== osd_op_reply(130 gc.23 [call] v32'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.485+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:131 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 45 ==== osd_op_reply(131 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:132 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 46 ==== osd_op_reply(132 gc.23 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:133 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 47 ==== osd_op_reply(133 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:134 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 48 ==== osd_op_reply(134 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.486+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:135 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.487+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 49 ==== osd_op_reply(135 gc.23 [call] v32'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.487+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.487+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:136 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.488+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 50 ==== osd_op_reply(136 gc.24 [call] v32'16 uv16 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.488+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:137 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.488+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 51 ==== osd_op_reply(137 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.488+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.488+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:138 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 52 ==== osd_op_reply(138 gc.24 [call out=24b] v0'0 uv16 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:139 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 53 ==== osd_op_reply(139 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:140 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 54 ==== osd_op_reply(140 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.489+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:141 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.490+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 55 ==== osd_op_reply(141 gc.24 [call] v32'17 uv17 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.490+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.490+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:142 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.491+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 56 ==== osd_op_reply(142 gc.25 [call] v32'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.491+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:143 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.491+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 57 ==== osd_op_reply(143 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.491+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:144 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 58 ==== osd_op_reply(144 gc.25 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:145 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 59 ==== osd_op_reply(145 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:146 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 60 ==== osd_op_reply(146 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.492+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:147 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.493+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 61 ==== osd_op_reply(147 gc.25 [call] v32'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.493+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.493+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:148 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(148 gc.26 [call] v32'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:149 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(149 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:150 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(150 gc.26 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:151 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(151 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.495+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:152 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.496+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(152 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.496+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.496+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:153 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.497+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(153 gc.26 [call] v32'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.497+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.497+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:154 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 62 ==== osd_op_reply(154 gc.27 [call] v32'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:155 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 63 ==== osd_op_reply(155 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:156 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 64 ==== osd_op_reply(156 gc.27 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:157 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 65 ==== osd_op_reply(157 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.498+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:158 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.499+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 66 ==== osd_op_reply(158 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.499+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.499+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:159 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.500+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 67 ==== osd_op_reply(159 gc.27 [call] v32'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.500+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-04-23T19:10:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.500+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:160 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.501+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(160 gc.28 [call] v32'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.501+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:161 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.501+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(161 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.501+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.501+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:162 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f98300a6040 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.501+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(162 gc.28 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.501+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:163 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.502+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== osd_op_reply(163 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.502+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.502+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:164 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.502+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(164 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.502+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.502+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:165 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.503+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== osd_op_reply(165 gc.28 [call] v32'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.503+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.503+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:166 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.504+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 68 ==== osd_op_reply(166 gc.29 [call] v32'47 uv47 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.504+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:167 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 69 ==== osd_op_reply(167 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:168 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 70 ==== osd_op_reply(168 gc.29 [call out=24b] v0'0 uv47 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:169 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 71 ==== osd_op_reply(169 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:170 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 72 ==== osd_op_reply(170 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.505+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:171 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.506+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 73 ==== osd_op_reply(171 gc.29 [call] v32'48 uv48 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.506+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.506+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:172 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(172 gc.30 [call] v32'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982800f610 con 0x7f98300a6040 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:173 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(173 gc.30 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:174 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f98300a6040 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(174 gc.30 [call out=24b] v0'0 uv39 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:175 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(175 gc.30 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.508+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:176 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.509+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(176 gc.30 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.509+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.509+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:177 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.510+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(177 gc.30 [call] v32'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.510+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.510+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:178 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(178 gc.31 [call] v32'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:179 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(179 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:180 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f9838021070 2024-04-23T19:10:48.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(180 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:181 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(181 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.511+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.512+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:182 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.512+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(182 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.512+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.512+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:183 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.513+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(183 gc.31 [call] v32'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.513+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.513+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:184 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== osd_op_reply(184 gc.0 [call] v32'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:185 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(185 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:186 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(186 gc.0 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:187 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== osd_op_reply(187 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.514+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:188 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.515+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(188 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.515+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.515+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:189 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.516+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(189 gc.0 [call] v32'28 uv28 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.516+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.516+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:190 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== osd_op_reply(190 gc.1 [call] v32'20 uv20 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:191 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 42 ==== osd_op_reply(191 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:192 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 43 ==== osd_op_reply(192 gc.1 [call out=24b] v0'0 uv20 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:193 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 44 ==== osd_op_reply(193 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.517+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:194 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.518+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 45 ==== osd_op_reply(194 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.518+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.518+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:195 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.519+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 46 ==== osd_op_reply(195 gc.1 [call] v32'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.519+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-04-23T19:10:48.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.519+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:196 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 74 ==== osd_op_reply(196 gc.2 [call] v32'20 uv20 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:197 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 75 ==== osd_op_reply(197 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:198 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 76 ==== osd_op_reply(198 gc.2 [call out=24b] v0'0 uv20 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:199 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 77 ==== osd_op_reply(199 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.520+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:200 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.521+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 78 ==== osd_op_reply(200 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.521+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.521+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:201 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.521+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 79 ==== osd_op_reply(201 gc.2 [call] v32'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.521+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.521+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:202 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.522+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(202 gc.3 [call] v32'163 uv163 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.522+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:203 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(203 gc.3 [call out=15b] v0'0 uv163 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:204 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(204 gc.3 [call out=24b] v0'0 uv163 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:205 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 49 ==== osd_op_reply(205 gc.3 [call out=15b] v0'0 uv163 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:206 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 50 ==== osd_op_reply(206 gc.3 [call out=15b] v0'0 uv163 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.523+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:207 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.524+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 51 ==== osd_op_reply(207 gc.3 [call] v32'164 uv164 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.524+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.524+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:208 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.525+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== osd_op_reply(208 gc.4 [call] v32'50 uv50 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.525+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:209 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== osd_op_reply(209 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:210 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 34 ==== osd_op_reply(210 gc.4 [call out=24b] v0'0 uv50 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:211 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 35 ==== osd_op_reply(211 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.526+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:212 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.527+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 36 ==== osd_op_reply(212 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.527+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.527+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:213 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.528+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 37 ==== osd_op_reply(213 gc.4 [call] v32'51 uv51 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.528+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-04-23T19:10:48.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.528+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:214 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 80 ==== osd_op_reply(214 gc.5 [call] v32'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:215 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 81 ==== osd_op_reply(215 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:216 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 82 ==== osd_op_reply(216 gc.5 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:217 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 83 ==== osd_op_reply(217 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.529+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:218 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.530+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 84 ==== osd_op_reply(218 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.530+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.530+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:219 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a81ce40 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.531+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 85 ==== osd_op_reply(219 gc.5 [call] v32'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.531+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.531+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:220 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.532+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 52 ==== osd_op_reply(220 gc.6 [call] v32'22 uv22 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.532+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:221 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.532+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 53 ==== osd_op_reply(221 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.532+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.532+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:222 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x557c4a816bf0 2024-04-23T19:10:48.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 54 ==== osd_op_reply(222 gc.6 [call out=24b] v0'0 uv22 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:223 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 55 ==== osd_op_reply(223 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:224 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 56 ==== osd_op_reply(224 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.533+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:225 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x557c4a816bf0 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.534+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 57 ==== osd_op_reply(225 gc.6 [call] v32'23 uv23 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.534+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.534+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:226 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.535+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 47 ==== osd_op_reply(226 gc.7 [call] v32'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.535+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:227 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.535+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 48 ==== osd_op_reply(227 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.535+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:228 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 49 ==== osd_op_reply(228 gc.7 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:229 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 50 ==== osd_op_reply(229 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:230 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 51 ==== osd_op_reply(230 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.536+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:231 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f98300a6040 2024-04-23T19:10:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.538+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 52 ==== osd_op_reply(231 gc.7 [call] v32'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.538+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-04-23T19:10:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.538+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:232 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 38 ==== osd_op_reply(232 gc.8 [call] v32'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:233 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 39 ==== osd_op_reply(233 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:234 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a860830 con 0x7f9838021070 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 40 ==== osd_op_reply(234 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:235 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.539+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 41 ==== osd_op_reply(235 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:236 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a861840 con 0x7f9838021070 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 42 ==== osd_op_reply(236 gc.8 [call out=1701b] v0'0 uv21 ondisk = 0) ==== 148+0+1701 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 20 garbage collection: RGWGC::process iterating over entry tag='bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.6599408857962759447', time=2024-04-23T19:10:48.123981+0000, chain.objs.size()=7 2024-04-23T19:10:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_1 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:237 3.4s0 3:2251347e:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_1:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85fae0 con 0x7f98300a6040 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_2 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:238 3.31s0 3:8ed0a418:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_2:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a864900 con 0x7f98300a6040 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_3 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:239 3.25s0 3:a407f911:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_3:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a864900 con 0x557c4a816bf0 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_4 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:240 3.25s0 3:a770902a:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_4:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a869700 con 0x557c4a816bf0 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_5 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:241 3.fs0 3:f2fc95c1:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_5:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86c060 con 0x7f9838021070 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_6 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:242 3.23s0 3:c404ed2c:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_6:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86e8d0 con 0x7f98300a6040 2024-04-23T19:10:48.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_7 2024-04-23T19:10:48.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.540+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:243 3.2s0 3:43c57b08:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_7:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a871140 con 0x557c4a81ce40 2024-04-23T19:10:48.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.542+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 53 ==== osd_op_reply(238 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_2 [call] v32'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.544+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 54 ==== osd_op_reply(237 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_1 [call] v32'4 uv4 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.544+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 55 ==== osd_op_reply(242 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_6 [call] v32'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.544+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 43 ==== osd_op_reply(241 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_5 [call] v32'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.547+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 86 ==== osd_op_reply(243 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_7 [call] v32'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.547+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 58 ==== osd_op_reply(239 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_3 [call] v32'3 uv3 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.547+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 59 ==== osd_op_reply(240 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1__shadow_.zhmWh07KRORvPcDqdbHYrFa243wvoGB_4 [call] v32'4 uv4 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.547+0000 7f9843830800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-23T19:10:48.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.547+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:244 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x7f9838021070 2024-04-23T19:10:48.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.552+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 44 ==== osd_op_reply(244 gc.8 [call] v32'22 uv22 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.552+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:245 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x7f9838021070 2024-04-23T19:10:48.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.555+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 45 ==== osd_op_reply(245 gc.8 [call] v32'23 uv23 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.555+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-04-23T19:10:48.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.555+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:246 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x557c4a81ce40 2024-04-23T19:10:48.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.556+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 87 ==== osd_op_reply(246 gc.9 [call] v32'26 uv26 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.556+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:247 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x557c4a81ce40 2024-04-23T19:10:48.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.556+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 88 ==== osd_op_reply(247 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.556+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.556+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:248 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86aba0 con 0x557c4a81ce40 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.556+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 89 ==== osd_op_reply(248 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.556+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:249 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x557c4a81ce40 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.557+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 90 ==== osd_op_reply(249 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.557+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.557+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:250 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x557c4a81ce40 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.557+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 91 ==== osd_op_reply(250 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.557+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.557+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:251 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x557c4a81ce40 2024-04-23T19:10:48.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.558+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 92 ==== osd_op_reply(251 gc.9 [call] v32'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.558+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-04-23T19:10:48.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.558+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:252 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a85f510 con 0x7f9838021070 2024-04-23T19:10:48.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 46 ==== osd_op_reply(252 gc.10 [call] v32'52 uv52 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:253 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bb20 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 47 ==== osd_op_reply(253 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:254 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86ab10 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 48 ==== osd_op_reply(254 gc.10 [call out=24b] v0'0 uv52 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:255 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bb20 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 49 ==== osd_op_reply(255 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.559+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:256 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bb20 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.560+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 50 ==== osd_op_reply(256 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.560+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.560+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:257 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bb20 con 0x7f9838021070 2024-04-23T19:10:48.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.560+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 51 ==== osd_op_reply(257 gc.10 [call] v32'53 uv53 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.561+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.561+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:258 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bb20 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 56 ==== osd_op_reply(258 gc.11 [call] v32'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:259 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bb20 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 57 ==== osd_op_reply(259 gc.11 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:260 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86ab10 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 58 ==== osd_op_reply(260 gc.11 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:261 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 59 ==== osd_op_reply(261 gc.11 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.562+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:262 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f98300a6040 2024-04-23T19:10:48.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.563+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 60 ==== osd_op_reply(262 gc.11 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.563+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.563+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:263 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f98300a6040 2024-04-23T19:10:48.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.564+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 61 ==== osd_op_reply(263 gc.11 [call] v32'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.564+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-04-23T19:10:48.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.564+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:264 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 52 ==== osd_op_reply(264 gc.12 [call] v32'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:265 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 53 ==== osd_op_reply(265 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:266 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bc50 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 54 ==== osd_op_reply(266 gc.12 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:267 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 55 ==== osd_op_reply(267 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:268 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.565+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 56 ==== osd_op_reply(268 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.566+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.566+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:269 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.566+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 57 ==== osd_op_reply(269 gc.12 [call] v32'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.566+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-04-23T19:10:48.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.566+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:270 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f98300a6040 2024-04-23T19:10:48.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.567+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 62 ==== osd_op_reply(270 gc.13 [call] v32'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.567+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:271 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 63 ==== osd_op_reply(271 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:272 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86a850 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 64 ==== osd_op_reply(272 gc.13 [call out=24b] v0'0 uv41 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:273 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86a850 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 65 ==== osd_op_reply(273 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.568+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:274 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86a850 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.569+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 66 ==== osd_op_reply(274 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.569+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.569+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:275 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f98300a6040 2024-04-23T19:10:48.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.570+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 67 ==== osd_op_reply(275 gc.13 [call] v32'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.570+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-04-23T19:10:48.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.570+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:276 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a816bf0 2024-04-23T19:10:48.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.571+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 60 ==== osd_op_reply(276 gc.14 [call] v32'165 uv165 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.571+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:277 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.571+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 61 ==== osd_op_reply(277 gc.14 [call out=15b] v0'0 uv165 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.571+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.571+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:278 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86a850 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.571+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 62 ==== osd_op_reply(278 gc.14 [call out=24b] v0'0 uv165 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.571+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:279 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86a850 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.572+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 63 ==== osd_op_reply(279 gc.14 [call out=15b] v0'0 uv165 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.572+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.572+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:280 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86a850 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.572+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 64 ==== osd_op_reply(280 gc.14 [call out=15b] v0'0 uv165 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.572+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.572+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:281 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a816bf0 2024-04-23T19:10:48.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.573+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 65 ==== osd_op_reply(281 gc.14 [call] v32'166 uv166 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.573+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-04-23T19:10:48.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.573+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:282 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a81ce40 2024-04-23T19:10:48.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.574+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 93 ==== osd_op_reply(282 gc.15 [call] v32'49 uv49 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.574+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:283 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.574+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 94 ==== osd_op_reply(283 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.574+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.574+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:284 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bc50 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 95 ==== osd_op_reply(284 gc.15 [call out=24b] v0'0 uv49 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:285 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 96 ==== osd_op_reply(285 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:286 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 97 ==== osd_op_reply(286 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.575+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:287 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x557c4a81ce40 2024-04-23T19:10:48.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.576+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 98 ==== osd_op_reply(287 gc.15 [call] v32'50 uv50 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.576+0000 7f9843830800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-04-23T19:10:48.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.576+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:288 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.577+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 58 ==== osd_op_reply(288 gc.16 [call] v32'54 uv54 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.577+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:289 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.577+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 59 ==== osd_op_reply(289 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.577+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.577+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:290 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bc50 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 60 ==== osd_op_reply(290 gc.16 [call out=24b] v0'0 uv54 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:291 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bc50 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 61 ==== osd_op_reply(291 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:292 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 62 ==== osd_op_reply(292 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f9843830800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:10:48.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.578+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:293 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86b860 con 0x7f9838021070 2024-04-23T19:10:48.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.579+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 63 ==== osd_op_reply(293 gc.16 [call] v32'55 uv55 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.580+0000 7f96f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:48.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.580+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93992314381888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bc50 con 0x557c4a816bf0 2024-04-23T19:10:48.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.581+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 66 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 93992314381888] v32'167 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.581+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93992314343728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a86bc50 con 0x7f9838021070 2024-04-23T19:10:48.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.582+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 64 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 93992314343728] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.582+0000 7f9843830800 20 remove_watcher() i=0 2024-04-23T19:10:48.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.582+0000 7f9843830800 2 removed watcher, disabling cache 2024-04-23T19:10:48.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.582+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93992314351072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a821130 con 0x557c4a816bf0 2024-04-23T19:10:48.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.583+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 67 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 93992314351072] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.583+0000 7f9843830800 20 remove_watcher() i=1 2024-04-23T19:10:48.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.583+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93992314352224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a821130 con 0x557c4a81ce40 2024-04-23T19:10:48.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.584+0000 7f983f293640 1 -- 172.21.15.63:0/2485669759 <== osd.3 v2:172.21.15.63:6816/1289558434 99 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 93992314352224] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9838056040 con 0x557c4a81ce40 2024-04-23T19:10:48.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.584+0000 7f9843830800 20 remove_watcher() i=2 2024-04-23T19:10:48.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.584+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93992314356768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a822de0 con 0x7f9838021070 2024-04-23T19:10:48.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.585+0000 7f983ea92640 1 -- 172.21.15.63:0/2485669759 <== osd.2 v2:172.21.15.63:6808/753864785 65 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 93992314356768] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f98300b3040 con 0x7f9838021070 2024-04-23T19:10:48.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.585+0000 7f9843830800 20 remove_watcher() i=3 2024-04-23T19:10:48.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.585+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93992314361568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a822de0 con 0x557c4a816bf0 2024-04-23T19:10:48.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.586+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.1 v2:172.21.15.63:6800/3216879695 68 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 93992314361568] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982804a040 con 0x557c4a816bf0 2024-04-23T19:10:48.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.586+0000 7f9843830800 20 remove_watcher() i=4 2024-04-23T19:10:48.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.586+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93992314365200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a8256e0 con 0x7f98300a6040 2024-04-23T19:10:48.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.587+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 68 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 93992314365200] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.587+0000 7f9843830800 20 remove_watcher() i=5 2024-04-23T19:10:48.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.587+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93992314358896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a822de0 con 0x7f98300a6040 2024-04-23T19:10:48.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.588+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 69 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 93992314358896] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.588+0000 7f9843830800 20 remove_watcher() i=6 2024-04-23T19:10:48.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.588+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93992314355200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557c4a8256e0 con 0x7f98300a6040 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 <== osd.0 v2:172.21.15.16:6800/2262294208 70 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 93992314355200] v32'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f982804a040 con 0x7f98300a6040 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 20 remove_watcher() i=7 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f98300a6040 msgr2=0x7f98300540c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f98300a6040 0x7f98300540c0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a816bf0 msgr2=0x557c4a8190b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a816bf0 0x557c4a8190b0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9838021070 msgr2=0x7f9838021450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9838021070 0x7f9838021450 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557c4a81ce40 msgr2=0x557c4a81f270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557c4a81ce40 0x557c4a81f270 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9708036e60 msgr2=0x7f9708039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9708036e60 0x7f9708039320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9830051fe0 tx=0x7f9830092000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 msgr2=0x557c4a80b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f983e291640 1 -- 172.21.15.63:0/2485669759 reap_dead start 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a80b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f98380174e0 tx=0x7f983804ccb0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.589+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 shutdown_connections 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/2485669759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a803b80 0x557c4a80b0a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 >> 172.21.15.63:0/2485669759 conn(0x557c4a7ffe00 msgr2=0x557c4a7fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 shutdown_connections 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2485669759 wait complete. 2024-04-23T19:10:48.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9718036e60 msgr2=0x7f9718039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9718036e60 0x7f9718039320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f983007a040 tx=0x7f983008d000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a7f6780 msgr2=0x557c4a7f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a7f6780 0x557c4a7f6ba0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x557c4a7dc4c0 tx=0x7f983800ef30 comp rx=0 tx=0).stop 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 shutdown_connections 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9718036e60 0x7f9718039320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/2679943927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a7f6780 0x557c4a7f6ba0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 >> 172.21.15.63:0/2679943927 conn(0x557c4a7ea820 msgr2=0x557c4a7eac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 shutdown_connections 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/2679943927 wait complete. 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557c4a7e59d0 msgr2=0x557c4a7e7dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557c4a7e59d0 0x557c4a7e7dc0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a6df380 msgr2=0x557c4a7e2300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a6df380 0x557c4a7e2300 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9724036eb0 msgr2=0x7f9724039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9724036eb0 0x7f9724039370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9828000f60 tx=0x7f9828001370 comp rx=0 tx=0).stop 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 msgr2=0x557c4a7db0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7db0c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9830000f80 tx=0x7f9830003330 comp rx=0 tx=0).stop 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 shutdown_connections 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557c4a7e59d0 0x557c4a7e7dc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557c4a6df380 0x557c4a7e2300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9724036eb0 0x7f9724039370 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.590+0000 7f9843830800 1 --2- 172.21.15.63:0/3945257574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c4a6dea20 0x557c4a7db0c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.591+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 >> 172.21.15.63:0/3945257574 conn(0x557c4a6da620 msgr2=0x557c4a7cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.591+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 shutdown_connections 2024-04-23T19:10:48.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.591+0000 7f9843830800 1 -- 172.21.15.63:0/3945257574 wait complete. 2024-04-23T19:10:48.599 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-23T19:10:48.599 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-23T19:10:48.600 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-23T19:10:48.682 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:48.682 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efecc536800 1 Processor -- start 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efecc536800 1 -- start start 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efecc536800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5570d2ece510 0x5570d2ece930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efecc536800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5570d2d817f0 con 0x5570d2ecf290 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efecc536800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5570d2d70e20 con 0x5570d2ece510 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efec82cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5570d2ece510 0x5570d2ece930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efec7acb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5570d2ecf290 0x5570d2ecf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49664/0 (socket says 172.21.15.63:49664) 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efec7acb640 1 -- 172.21.15.63:0/639376699 learned_addr learned my addr 172.21.15.63:0/639376699 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efec82cc640 1 -- 172.21.15.63:0/639376699 >> v1:172.21.15.16:6789/0 conn(0x5570d2ecf290 legacy=0x5570d2ecf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.683+0000 7efec82cc640 1 -- 172.21.15.63:0/639376699 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d2d8fb50 con 0x5570d2ece510 2024-04-23T19:10:48.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.685+0000 7efec82cc640 1 --2- 172.21.15.63:0/639376699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2ece930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efeb800a470 tx=0x7efeb801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43630f332ba3b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.685+0000 7efec72ca640 1 -- 172.21.15.63:0/639376699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb8057070 con 0x5570d2ece510 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.685+0000 7efec72ca640 1 -- 172.21.15.63:0/639376699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb804ad80 con 0x5570d2ece510 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.685+0000 7efec72ca640 1 -- 172.21.15.63:0/639376699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb8052590 con 0x5570d2ece510 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.686+0000 7efecc536800 1 -- 172.21.15.63:0/639376699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 msgr2=0x5570d2ece930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.686+0000 7efecc536800 1 --2- 172.21.15.63:0/639376699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2ece930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efeb800a470 tx=0x7efeb801ce50 comp rx=0 tx=0).stop 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.686+0000 7efecc536800 1 -- 172.21.15.63:0/639376699 shutdown_connections 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.686+0000 7efecc536800 1 --2- 172.21.15.63:0/639376699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2ece930 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.686+0000 7efecc536800 1 -- 172.21.15.63:0/639376699 >> 172.21.15.63:0/639376699 conn(0x5570d2ec8db0 msgr2=0x5570d2ecb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.686+0000 7efecc536800 1 -- 172.21.15.63:0/639376699 shutdown_connections 2024-04-23T19:10:48.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.686+0000 7efecc536800 1 -- 172.21.15.63:0/639376699 wait complete. 2024-04-23T19:10:48.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.699+0000 7efecc536800 1 Processor -- start 2024-04-23T19:10:48.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.699+0000 7efecc536800 1 -- start start 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.699+0000 7efecc536800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fbe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.699+0000 7efecc536800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570d2e1a6f0 con 0x5570d2ece510 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.699+0000 7efec82cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fbe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.699+0000 7efec82cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fbe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35556/0 (socket says 172.21.15.63:35556) 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.699+0000 7efec82cc640 1 -- 172.21.15.63:0/4024094820 learned_addr learned my addr 172.21.15.63:0/4024094820 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.700+0000 7efec82cc640 1 -- 172.21.15.63:0/4024094820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d2fbec70 con 0x5570d2ece510 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.700+0000 7efec82cc640 1 --2- 172.21.15.63:0/4024094820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fbe6a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efeb8055040 tx=0x7efeb804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b496b417557e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.700+0000 7efdb660c640 1 -- 172.21.15.63:0/4024094820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb8057040 con 0x5570d2ece510 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.700+0000 7efdb660c640 1 -- 172.21.15.63:0/4024094820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb8063070 con 0x5570d2ece510 2024-04-23T19:10:48.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.700+0000 7efdb660c640 1 -- 172.21.15.63:0/4024094820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb80527e0 con 0x5570d2ece510 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 -- 172.21.15.63:0/4024094820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 msgr2=0x5570d2fbe6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 --2- 172.21.15.63:0/4024094820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fbe6a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efeb8055040 tx=0x7efeb804c000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 -- 172.21.15.63:0/4024094820 shutdown_connections 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 --2- 172.21.15.63:0/4024094820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fbe6a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 -- 172.21.15.63:0/4024094820 >> 172.21.15.63:0/4024094820 conn(0x5570d2ec8db0 msgr2=0x5570d2fb9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 -- 172.21.15.63:0/4024094820 shutdown_connections 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 -- 172.21.15.63:0/4024094820 wait complete. 2024-04-23T19:10:48.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 Processor -- start 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 -- start start 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fc6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.701+0000 7efecc536800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570d2ecd340 con 0x5570d2ece510 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efec82cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fc6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efec82cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fc6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35560/0 (socket says 172.21.15.63:35560) 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efec82cc640 1 -- 172.21.15.63:0/1450376979 learned_addr learned my addr 172.21.15.63:0/1450376979 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efec82cc640 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d2fc6640 con 0x5570d2ece510 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efec82cc640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fc6100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efeb8060040 tx=0x7efeb8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efdb4e09640 1 -- 172.21.15.63:0/1450376979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb8057050 con 0x5570d2ece510 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efdb4e09640 1 -- 172.21.15.63:0/1450376979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb8052d10 con 0x5570d2ece510 2024-04-23T19:10:48.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efdb4e09640 1 -- 172.21.15.63:0/1450376979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb805d470 con 0x5570d2ece510 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570d2fc68d0 con 0x5570d2ece510 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.702+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d2fc9530 con 0x5570d2ece510 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.703+0000 7efdb4e09640 1 -- 172.21.15.63:0/1450376979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7efeb805d660 con 0x5570d2ece510 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.703+0000 7efdb4e09640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efdb0036e60 0x7efdb0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.703+0000 7efdb4e09640 1 -- 172.21.15.63:0/1450376979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7efeb808e480 con 0x5570d2ece510 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.703+0000 7efec7acb640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efdb0036e60 0x7efdb0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.703+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d2ecf290 0x5570d2fcd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.703+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d2fcd8c0 con 0x5570d2ecf290 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.703+0000 7efec8acd640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d2ecf290 0x5570d2fcd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.704+0000 7efec7acb640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efdb0036e60 0x7efdb0039320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efebc000f60 tx=0x7efebc0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.704+0000 7efec8acd640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d2ecf290 0x5570d2fcd2f0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.704+0000 7efec8acd640 1 -- 172.21.15.63:0/1450376979 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efec000a4a0 con 0x5570d2ecf290 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.704+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d2fca8c0 con 0x5570d2ecf290 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.704+0000 7efec8acd640 1 -- 172.21.15.63:0/1450376979 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7efec000a4a0 con 0x5570d2ecf290 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.704+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d2fcf730 con 0x5570d2ecf290 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.705+0000 7efec8acd640 1 -- 172.21.15.63:0/1450376979 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7efec000a4a0 con 0x5570d2ecf290 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.705+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d2fcea70 con 0x5570d2ecf290 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.705+0000 7efec8acd640 1 -- 172.21.15.63:0/1450376979 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7efec000a4a0 con 0x5570d2ecf290 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.705+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570d2fd0ae0 0x5570d2fd2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.705+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d2fd3450 con 0x5570d2fd0ae0 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.705+0000 7efec82cc640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570d2fd0ae0 0x5570d2fd2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.706+0000 7efec82cc640 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570d2fd0ae0 0x5570d2fd2ed0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.706+0000 7efec82cc640 1 -- 172.21.15.63:0/1450376979 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7efeb8078070 con 0x5570d2fd0ae0 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.706+0000 7efecc536800 1 Processor -- start 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.706+0000 7efecc536800 1 -- start start 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.706+0000 7efecc536800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2fe1870 0x5570d2fe1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.706+0000 7efecc536800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570d2dc0e30 con 0x5570d2fe1870 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efec8acd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2fe1870 0x5570d2fe1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efec8acd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2fe1870 0x5570d2fe1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35568/0 (socket says 172.21.15.63:35568) 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efec8acd640 1 -- 172.21.15.63:0/2598230590 learned_addr learned my addr 172.21.15.63:0/2598230590 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efec8acd640 1 -- 172.21.15.63:0/2598230590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d2fe21d0 con 0x5570d2fe1870 2024-04-23T19:10:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efec8acd640 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2fe1870 0x5570d2fe1c90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5570d2fcdca0 tx=0x7efec00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efdadffb640 1 -- 172.21.15.63:0/2598230590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec0013440 con 0x5570d2fe1870 2024-04-23T19:10:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efdadffb640 1 -- 172.21.15.63:0/2598230590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec00135e0 con 0x5570d2fe1870 2024-04-23T19:10:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efdadffb640 1 -- 172.21.15.63:0/2598230590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec00138c0 con 0x5570d2fe1870 2024-04-23T19:10:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570d2fe2c40 con 0x5570d2fe1870 2024-04-23T19:10:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.707+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d2fe3140 con 0x5570d2fe1870 2024-04-23T19:10:48.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.708+0000 7efdadffb640 1 -- 172.21.15.63:0/2598230590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7efec0011040 con 0x5570d2fe1870 2024-04-23T19:10:48.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.708+0000 7efdadffb640 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efda0036e60 0x7efda0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.708+0000 7efdadffb640 1 -- 172.21.15.63:0/2598230590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7efec001d070 con 0x5570d2fe1870 2024-04-23T19:10:48.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.708+0000 7efec82cc640 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efda0036e60 0x7efda0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efecc536800 1 Processor -- start 2024-04-23T19:10:48.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efec82cc640 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efda0036e60 0x7efda0039320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5570d2fd3830 tx=0x7efeb8066f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efecc536800 1 -- start start 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efecc536800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2feef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efecc536800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570d2fbef30 con 0x5570d2feeb70 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efec8acd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2feef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efec8acd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2feef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35582/0 (socket says 172.21.15.63:35582) 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.709+0000 7efec8acd640 1 -- 172.21.15.63:0/1575503304 learned_addr learned my addr 172.21.15.63:0/1575503304 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efec8acd640 1 -- 172.21.15.63:0/1575503304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d2fef560 con 0x5570d2feeb70 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efec8acd640 1 --2- 172.21.15.63:0/1575503304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2feef90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efec00129a0 tx=0x7efec0012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f07a2c3a0be6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efd9f7fe640 1 -- 172.21.15.63:0/1575503304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec002ad80 con 0x5570d2feeb70 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efd9f7fe640 1 -- 172.21.15.63:0/1575503304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec00014f0 con 0x5570d2feeb70 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efd9f7fe640 1 -- 172.21.15.63:0/1575503304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec00017d0 con 0x5570d2feeb70 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efecc536800 1 -- 172.21.15.63:0/1575503304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 msgr2=0x5570d2feef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efecc536800 1 --2- 172.21.15.63:0/1575503304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2feef90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efec00129a0 tx=0x7efec0012a90 comp rx=0 tx=0).stop 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efecc536800 1 -- 172.21.15.63:0/1575503304 shutdown_connections 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efecc536800 1 --2- 172.21.15.63:0/1575503304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2feef90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efecc536800 1 -- 172.21.15.63:0/1575503304 >> 172.21.15.63:0/1575503304 conn(0x5570d2feaed0 msgr2=0x5570d2fe8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efecc536800 1 -- 172.21.15.63:0/1575503304 shutdown_connections 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.710+0000 7efecc536800 1 -- 172.21.15.63:0/1575503304 wait complete. 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efecc536800 1 Processor -- start 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efecc536800 1 -- start start 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efecc536800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2ff6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efecc536800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570d2fefcd0 con 0x5570d2feeb70 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efec8acd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2ff6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efec8acd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2ff6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35592/0 (socket says 172.21.15.63:35592) 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 learned_addr learned my addr 172.21.15.63:0/797668669 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d2ff6670 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.711+0000 7efec8acd640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2ff6130 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efec004cba0 tx=0x7efec004cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.712+0000 7efd9dffb640 1 -- 172.21.15.63:0/797668669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec00125b0 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.712+0000 7efd9dffb640 1 -- 172.21.15.63:0/797668669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0030070 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.712+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570d2ff6900 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.712+0000 7efd9dffb640 1 -- 172.21.15.63:0/797668669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec004e110 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.712+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d2ff95b0 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.712+0000 7efd9dffb640 1 -- 172.21.15.63:0/797668669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7efec0047050 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efd9dffb640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efd90036e60 0x7efd90039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efd9dffb640 1 -- 172.21.15.63:0/797668669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7efec000e040 con 0x5570d2feeb70 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efec82cc640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efd90036e60 0x7efd90039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d3001e40 0x5570d3004300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d3004a50 con 0x5570d3001e40 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efec7acb640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d3001e40 0x5570d3004300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efec82cc640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efd90036e60 0x7efd90039320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efeb808c8e0 tx=0x7efeb8086000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.713+0000 7efec7acb640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d3001e40 0x5570d3004300 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efebc009050 con 0x5570d3001e40 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 realm 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d3001290 con 0x5570d3001e40 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efebc009050 con 0x5570d3001e40 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d3005530 con 0x5570d3001e40 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efebc009050 con 0x5570d3001e40 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d3005e00 con 0x5570d3001e40 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7efebc009050 con 0x5570d3001e40 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.714+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d30061a0 con 0x5570d3001e40 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7efebc009050 con 0x5570d3001e40 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efecc536800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efecc536800 20 searching for the correct realm 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5570d3008080 0x5570d300a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efec8acd640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5570d3008080 0x5570d300a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efec8acd640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5570d3008080 0x5570d300a470 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efec8acd640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7efec0025120 0x7efec0026a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.715+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efec0025120 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.716+0000 7efec82cc640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7efec0025120 0x7efec0026a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.716+0000 7efec82cc640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7efec0025120 0x7efec0026a60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.716+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.716+0000 7efec82cc640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efeb809b040 0x7efeb808e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.716+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.716+0000 7efec7acb640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efeb809b040 0x7efeb808e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.717+0000 7efec7acb640 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efeb809b040 0x7efeb808e860 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.717+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc009050 con 0x7efeb809b040 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.717+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.717+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc009050 con 0x5570d3001e40 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.717+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.717+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.717+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efec0025120 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.718+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.719+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.719+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.719+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.719+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.719+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.719+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efec0025120 2024-04-23T19:10:48.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.720+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.721+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.722+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.722+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.722+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.722+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efec0025120 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.722+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.722+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.723+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.723+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efec0025120 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.723+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.723+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.723+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.723+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efec0025120 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3008080 2024-04-23T19:10:48.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efeb809b040 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.724+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x5570d3001e40 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ab20 con 0x7efec0025120 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:48.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d3006b50 con 0x5570d3001e40 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d30073f0 con 0x5570d3001e40 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.725+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d3006790 con 0x7efeb809b040 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 zone default found 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 4 Realm: () 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 zonegroup default 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d300af80 con 0x7efec0025120 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d300d1b0 con 0x5570d3008080 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.726+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d300d550 con 0x5570d3008080 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.727+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.727+0000 7efecc536800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.727+0000 7efecc536800 20 started sync module instance, tier type = 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.727+0000 7efecc536800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.727+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300c690 con 0x7efec0025120 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.728+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.728+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93943064742528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300c2f0 con 0x7efec0025120 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.728+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300e410 con 0x5570d3001e40 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.729+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93943064742528] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.729+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.729+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93943064747872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300e380 con 0x5570d3001e40 2024-04-23T19:10:48.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.729+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300f2d0 con 0x5570d3008080 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.730+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93943064747872] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.730+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.730+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93943064750080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300f2d0 con 0x5570d3008080 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.730+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300fd40 con 0x7efec0025120 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.731+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.731+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93943064754224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d30105e0 con 0x7efec0025120 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.731+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d30110c0 con 0x5570d3001e40 2024-04-23T19:10:48.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.731+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93943064750080] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.732+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.732+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93943064759248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d30110c0 con 0x5570d3001e40 2024-04-23T19:10:48.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.732+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3011b30 con 0x7efeb809b040 2024-04-23T19:10:48.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.733+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93943064754224] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.733+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.734+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93943064761888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3011af0 con 0x7efeb809b040 2024-04-23T19:10:48.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.734+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93943064759248] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.734+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ee20 con 0x7efeb809b040 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.734+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93943064761888] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.735+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.735+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93943064756528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3012090 con 0x7efeb809b040 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.735+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3012a50 con 0x7efeb809b040 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93943064765856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3012e70 con 0x7efeb809b040 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 20 add_watcher() i=0 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 20 add_watcher() i=1 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 20 add_watcher() i=2 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 20 add_watcher() i=3 2024-04-23T19:10:48.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 20 add_watcher() i=4 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 20 add_watcher() i=5 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93943064756528] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.736+0000 7efecc536800 20 add_watcher() i=6 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.737+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93943064765856] v32'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.737+0000 7efecc536800 20 add_watcher() i=7 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.737+0000 7efecc536800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.737+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ee20 con 0x5570d3001e40 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.737+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.737+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93943064766912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ee20 con 0x5570d3001e40 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.738+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93943064766912] v32'168 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.738+0000 7efd7e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.738+0000 7efecc536800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:48.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.738+0000 7efecc536800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.738+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301e4c0 con 0x5570d3001e40 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.739+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'29 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.739+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301eab0 con 0x7efeb809b040 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.741+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.741+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301eeb0 con 0x5570d3008080 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.742+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.742+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301ed10 con 0x5570d3001e40 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.743+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'169 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.743+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301f390 con 0x7efec0025120 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.744+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'56 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.744+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301f900 con 0x5570d3008080 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.745+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'37 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.745+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301f900 con 0x5570d3001e40 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.746+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.746+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301fcc0 con 0x7efeb809b040 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.747+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'37 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.747+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301fcc0 con 0x7efec0025120 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.748+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.748+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.749+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.749+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efec0025120 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.750+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'57 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.750+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efeb809b040 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.751+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.751+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efec0025120 2024-04-23T19:10:48.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.751+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.752+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efeb809b040 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.753+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'43 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.753+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3001e40 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.754+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'170 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.754+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.755+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'51 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.755+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efec0025120 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.756+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'58 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.756+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.757+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.757+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3001e40 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.758+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'30 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.758+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3001e40 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.758+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'171 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.758+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.760+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'52 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.760+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efec0025120 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.761+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'32 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.761+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.762+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.762+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.763+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.763+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.764+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'18 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.764+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.765+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'38 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.765+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efeb809b040 2024-04-23T19:10:48.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.766+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.766+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.767+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.767+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efeb809b040 2024-04-23T19:10:48.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.768+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'38 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.768+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x5570d3008080 2024-04-23T19:10:48.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.769+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'53 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.769+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efeb809b040 2024-04-23T19:10:48.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.770+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'44 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.770+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d3020170 con 0x7efec0025120 2024-04-23T19:10:48.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.771+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.772+0000 7efd7d7e2640 20 reqs_thread_entry: start 2024-04-23T19:10:48.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.772+0000 7efecc536800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:48.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.772+0000 7efecc536800 20 Filter name: none 2024-04-23T19:10:48.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.772+0000 7efceeffd640 20 reqs_thread_entry: start 2024-04-23T19:10:48.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.772+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.772+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv28 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.772+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3001e40 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.773+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv28 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.773+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.773+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv28 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.773+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.773+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.773+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efeb809b040 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.774+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv164 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3001e40 2024-04-23T19:10:48.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv164 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv164 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.775+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.776+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.776+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efec0025120 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.776+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv51 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.776+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.776+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.776+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.777+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.777+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.777+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv36 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.777+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.777+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.777+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.778+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.778+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3001e40 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.778+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv23 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.778+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.778+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.778+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efeb809b040 2024-04-23T19:10:48.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv36 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.779+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efec0025120 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv23 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.780+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.781+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.781+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.781+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.781+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efec0025120 2024-04-23T19:10:48.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.781+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv53 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.781+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.782+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.782+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.782+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.782+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efeb809b040 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.782+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.782+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efec0025120 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.783+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.784+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.784+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efeb809b040 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.784+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.784+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv166 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3001e40 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv166 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv166 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.785+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 36 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv50 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 37 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.786+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efec0025120 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.787+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv55 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.787+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.787+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 34 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.787+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.787+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 38 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.787+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 39 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 40 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3001e40 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.788+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.789+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.789+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.789+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv162 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.789+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3001e40 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.789+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv162 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.789+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3001e40 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv162 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 41 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 42 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv46 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 43 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.790+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 35 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efec0025120 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 36 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv31 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 37 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 44 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.791+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.792+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 45 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.792+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.792+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 46 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.792+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.792+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 47 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.792+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 48 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 49 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 50 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 51 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv17 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.793+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 52 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.794+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.794+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 53 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.794+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.794+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 54 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.794+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.794+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 55 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.794+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.795+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.795+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efeb809b040 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.795+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.795+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.796+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.796+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.796+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 56 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.796+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.796+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 57 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.796+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.797+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 58 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.797+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.797+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.797+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efeb809b040 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.797+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.797+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 59 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x5570d3008080 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 60 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv48 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x5570d3008080 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 61 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.798+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.799+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.799+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efeb809b040 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.799+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 42 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.799+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efeb809b040 2024-04-23T19:10:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.799+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 43 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.799+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.800+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 38 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.800+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304ab80 con 0x7efec0025120 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.800+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 39 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.800+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570d304bb90 con 0x7efec0025120 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.800+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 40 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.801+0000 7efd7e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:48.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.801+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93943064766912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d301d000 con 0x5570d3001e40 2024-04-23T19:10:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.802+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 93943064766912] v32'172 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.802+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93943064742528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d30129c0 con 0x7efec0025120 2024-04-23T19:10:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.803+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 41 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 93943064742528] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.803+0000 7efecc536800 20 remove_watcher() i=0 2024-04-23T19:10:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.803+0000 7efecc536800 2 removed watcher, disabling cache 2024-04-23T19:10:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.803+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93943064747872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ca80 con 0x5570d3001e40 2024-04-23T19:10:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.804+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 93943064747872] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.804+0000 7efecc536800 20 remove_watcher() i=1 2024-04-23T19:10:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.804+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93943064750080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ca80 con 0x5570d3008080 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.806+0000 7efec8acd640 1 -- 172.21.15.63:0/797668669 <== osd.3 v2:172.21.15.63:6816/1289558434 62 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 93943064750080] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efec0029070 con 0x5570d3008080 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.806+0000 7efecc536800 20 remove_watcher() i=2 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.806+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93943064754224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300e800 con 0x7efec0025120 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.807+0000 7efec82cc640 1 -- 172.21.15.63:0/797668669 <== osd.2 v2:172.21.15.63:6808/753864785 42 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 93943064754224] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efeb807e050 con 0x7efec0025120 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.807+0000 7efecc536800 20 remove_watcher() i=3 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.807+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93943064759248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ca80 con 0x5570d3001e40 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.808+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 93943064759248] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x5570d3001e40 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.808+0000 7efecc536800 20 remove_watcher() i=4 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.808+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93943064761888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300e800 con 0x7efeb809b040 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.809+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 44 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 93943064761888] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.809+0000 7efecc536800 20 remove_watcher() i=5 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.809+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93943064756528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300ca80 con 0x7efeb809b040 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.810+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 45 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 93943064756528] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.810+0000 7efecc536800 20 remove_watcher() i=6 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.810+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93943064765856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5570d300e800 con 0x7efeb809b040 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 <== osd.0 v2:172.21.15.16:6800/2262294208 46 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 93943064765856] v32'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efebc02a040 con 0x7efeb809b040 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 20 remove_watcher() i=7 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efeb809b040 msgr2=0x7efeb808e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7efeb809b040 0x7efeb808e860 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d3001e40 msgr2=0x5570d3004300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d3001e40 0x5570d3004300 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7efec0025120 msgr2=0x7efec0026a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7efec0025120 0x7efec0026a60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5570d3008080 msgr2=0x5570d300a470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5570d3008080 0x5570d300a470 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efd90036e60 msgr2=0x7efd90039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efd90036e60 0x7efd90039320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efeb808c8e0 tx=0x7efeb8086000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 msgr2=0x5570d2ff6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2ff6130 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efec004cba0 tx=0x7efec004cbd0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efec7acb640 1 -- 172.21.15.63:0/797668669 reap_dead start 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 shutdown_connections 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/797668669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2feeb70 0x5570d2ff6130 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 >> 172.21.15.63:0/797668669 conn(0x5570d2feaed0 msgr2=0x5570d2fe8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 shutdown_connections 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/797668669 wait complete. 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efda0036e60 msgr2=0x7efda0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efda0036e60 0x7efda0039320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5570d2fd3830 tx=0x7efeb8066f50 comp rx=0 tx=0).stop 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2fe1870 msgr2=0x5570d2fe1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2fe1870 0x5570d2fe1c90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5570d2fcdca0 tx=0x7efec00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 shutdown_connections 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efda0036e60 0x7efda0039320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 --2- 172.21.15.63:0/2598230590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2fe1870 0x5570d2fe1c90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 >> 172.21.15.63:0/2598230590 conn(0x5570d2fd5910 msgr2=0x5570d2fd5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 shutdown_connections 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.811+0000 7efecc536800 1 -- 172.21.15.63:0/2598230590 wait complete. 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570d2fd0ae0 msgr2=0x5570d2fd2ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570d2fd0ae0 0x5570d2fd2ed0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d2ecf290 msgr2=0x5570d2fcd2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d2ecf290 0x5570d2fcd2f0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efdb0036e60 msgr2=0x7efdb0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efdb0036e60 0x7efdb0039320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efebc000f60 tx=0x7efebc0012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 msgr2=0x5570d2fc6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fc6100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efeb8060040 tx=0x7efeb8002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 shutdown_connections 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570d2fd0ae0 0x5570d2fd2ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570d2ecf290 0x5570d2fcd2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7efdb0036e60 0x7efdb0039320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 --2- 172.21.15.63:0/1450376979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d2ece510 0x5570d2fc6100 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 >> 172.21.15.63:0/1450376979 conn(0x5570d2ec8db0 msgr2=0x5570d2fb8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 shutdown_connections 2024-04-23T19:10:48.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.812+0000 7efecc536800 1 -- 172.21.15.63:0/1450376979 wait complete. 2024-04-23T19:10:48.826 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-04-23T19:10:48.827 DEBUG:tasks.util.rgw: json result: [] 2024-04-23T19:10:48.827 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-23T19:10:48.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-23T19:10:48.828 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-23T19:10:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.911+0000 7f8a1792d800 1 Processor -- start 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.911+0000 7f8a1792d800 1 -- start start 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.911+0000 7f8a1792d800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5571ebf7eba0 0x5571ebf80f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.911+0000 7f8a1792d800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5571ebe307f0 con 0x5571ebf814d0 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a1792d800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5571ebe1fe20 con 0x5571ebf7eba0 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a135ae640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5571ebf7eba0 0x5571ebf80f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a135ae640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5571ebf7eba0 0x5571ebf80f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35598/0 (socket says 172.21.15.63:35598) 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a135ae640 1 -- 172.21.15.63:0/2634916539 learned_addr learned my addr 172.21.15.63:0/2634916539 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a12dad640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5571ebf814d0 0x5571ebf838c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49668/0 (socket says 172.21.15.63:49668) 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a135ae640 1 -- 172.21.15.63:0/2634916539 >> v1:172.21.15.16:6789/0 conn(0x5571ebf814d0 legacy=0x5571ebf838c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a125ac640 1 -- 172.21.15.63:0/2634916539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073680118 0 0) 0x5571ebe307f0 con 0x5571ebf814d0 2024-04-23T19:10:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.912+0000 7f8a135ae640 1 -- 172.21.15.63:0/2634916539 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ebe3eb50 con 0x5571ebf7eba0 2024-04-23T19:10:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.914+0000 7f8a135ae640 1 --2- 172.21.15.63:0/2634916539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ebf80f90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400a520 tx=0x7f8a0401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4856505d16f10a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.914+0000 7f8a125ac640 1 -- 172.21.15.63:0/2634916539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04057070 con 0x5571ebf7eba0 2024-04-23T19:10:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.914+0000 7f8a125ac640 1 -- 172.21.15.63:0/2634916539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04052570 con 0x5571ebf7eba0 2024-04-23T19:10:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.914+0000 7f8a125ac640 1 -- 172.21.15.63:0/2634916539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0404ae60 con 0x5571ebf7eba0 2024-04-23T19:10:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.915+0000 7f8a1792d800 1 -- 172.21.15.63:0/2634916539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 msgr2=0x5571ebf80f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.915+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2634916539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ebf80f90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400a520 tx=0x7f8a0401d010 comp rx=0 tx=0).stop 2024-04-23T19:10:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.915+0000 7f8a1792d800 1 -- 172.21.15.63:0/2634916539 shutdown_connections 2024-04-23T19:10:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.915+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2634916539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ebf80f90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.915+0000 7f8a1792d800 1 -- 172.21.15.63:0/2634916539 >> 172.21.15.63:0/2634916539 conn(0x5571ebf78630 msgr2=0x5571ebf7aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.915+0000 7f8a1792d800 1 -- 172.21.15.63:0/2634916539 shutdown_connections 2024-04-23T19:10:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.915+0000 7f8a1792d800 1 -- 172.21.15.63:0/2634916539 wait complete. 2024-04-23T19:10:48.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.927+0000 7f8a1792d800 1 Processor -- start 2024-04-23T19:10:48.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.927+0000 7f8a1792d800 1 -- start start 2024-04-23T19:10:48.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.928+0000 7f8a1792d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.928+0000 7f8a1792d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571ebec96f0 con 0x5571ebf7eba0 2024-04-23T19:10:48.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.928+0000 7f8a135ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.928+0000 7f8a135ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35612/0 (socket says 172.21.15.63:35612) 2024-04-23T19:10:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.928+0000 7f8a135ae640 1 -- 172.21.15.63:0/158993263 learned_addr learned my addr 172.21.15.63:0/158993263 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.928+0000 7f8a135ae640 1 -- 172.21.15.63:0/158993263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ec06f6d0 con 0x5571ebf7eba0 2024-04-23T19:10:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.928+0000 7f8a135ae640 1 --2- 172.21.15.63:0/158993263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec06f100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8a04055040 tx=0x7f8a0404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecc33aa457a9366 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f890160a640 1 -- 172.21.15.63:0/158993263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04062070 con 0x5571ebf7eba0 2024-04-23T19:10:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f890160a640 1 -- 172.21.15.63:0/158993263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04052720 con 0x5571ebf7eba0 2024-04-23T19:10:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f890160a640 1 -- 172.21.15.63:0/158993263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04057040 con 0x5571ebf7eba0 2024-04-23T19:10:48.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f8a1792d800 1 -- 172.21.15.63:0/158993263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 msgr2=0x5571ec06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f8a1792d800 1 --2- 172.21.15.63:0/158993263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec06f100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8a04055040 tx=0x7f8a0404c000 comp rx=0 tx=0).stop 2024-04-23T19:10:48.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f8a1792d800 1 -- 172.21.15.63:0/158993263 shutdown_connections 2024-04-23T19:10:48.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f8a1792d800 1 --2- 172.21.15.63:0/158993263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec06f100 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f8a1792d800 1 -- 172.21.15.63:0/158993263 >> 172.21.15.63:0/158993263 conn(0x5571ebf78630 msgr2=0x5571ec06aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f8a1792d800 1 -- 172.21.15.63:0/158993263 shutdown_connections 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.929+0000 7f8a1792d800 1 -- 172.21.15.63:0/158993263 wait complete. 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a1792d800 1 Processor -- start 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a1792d800 1 -- start start 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a1792d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec076b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a1792d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571ebf852e0 con 0x5571ebf7eba0 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a135ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec076b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a135ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec076b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35624/0 (socket says 172.21.15.63:35624) 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a135ae640 1 -- 172.21.15.63:0/2442720019 learned_addr learned my addr 172.21.15.63:0/2442720019 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a135ae640 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ec0770d0 con 0x5571ebf7eba0 2024-04-23T19:10:48.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.930+0000 7f8a135ae640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec076b90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005400 tx=0x7f8a04002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.931+0000 7f88ffe07640 1 -- 172.21.15.63:0/2442720019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04057050 con 0x5571ebf7eba0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.931+0000 7f88ffe07640 1 -- 172.21.15.63:0/2442720019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04067450 con 0x5571ebf7eba0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.931+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5571ec077360 con 0x5571ebf7eba0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.931+0000 7f88ffe07640 1 -- 172.21.15.63:0/2442720019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04062040 con 0x5571ebf7eba0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.931+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ec07a010 con 0x5571ebf7eba0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.931+0000 7f88ffe07640 1 -- 172.21.15.63:0/2442720019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8a04067730 con 0x5571ebf7eba0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f88ffe07640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88f8036eb0 0x7f88f8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f88ffe07640 1 -- 172.21.15.63:0/2442720019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f8a0408e970 con 0x5571ebf7eba0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f8a12dad640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88f8036eb0 0x7f88f8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ebf814d0 0x5571ec07ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec07e310 con 0x5571ebf814d0 2024-04-23T19:10:48.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f8a13daf640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ebf814d0 0x5571ec07ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f8a12dad640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88f8036eb0 0x7f88f8039370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a08000c00 tx=0x7f8a080013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.932+0000 7f8a13daf640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ebf814d0 0x5571ec07ddd0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a13daf640 1 -- 172.21.15.63:0/2442720019 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8a0c00a4a0 con 0x5571ebf814d0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec07b380 con 0x5571ebf814d0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a13daf640 1 -- 172.21.15.63:0/2442720019 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8a0c00a4a0 con 0x5571ebf814d0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec080140 con 0x5571ebf814d0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a13daf640 1 -- 172.21.15.63:0/2442720019 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8a0c00a4a0 con 0x5571ebf814d0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec07f480 con 0x5571ebf814d0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a13daf640 1 -- 172.21.15.63:0/2442720019 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8a0c00a4a0 con 0x5571ebf814d0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5571ec0814f0 0x5571ec0838e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.933+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec083ea0 con 0x5571ec0814f0 2024-04-23T19:10:48.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.934+0000 7f8a135ae640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5571ec0814f0 0x5571ec0838e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.934+0000 7f8a135ae640 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5571ec0814f0 0x5571ec0838e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.934+0000 7f8a135ae640 1 -- 172.21.15.63:0/2442720019 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8a04064070 con 0x5571ec0814f0 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a1792d800 1 Processor -- start 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a1792d800 1 -- start start 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a1792d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec0922b0 0x5571ec0926d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a1792d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571ebe6fe30 con 0x5571ec0922b0 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a13daf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec0922b0 0x5571ec0926d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a13daf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec0922b0 0x5571ec0926d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35640/0 (socket says 172.21.15.63:35640) 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a13daf640 1 -- 172.21.15.63:0/3076698686 learned_addr learned my addr 172.21.15.63:0/3076698686 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a13daf640 1 -- 172.21.15.63:0/3076698686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ec092c10 con 0x5571ec0922b0 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.935+0000 7f8a13daf640 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec0922b0 0x5571ec0926d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x5571ec077f90 tx=0x7f8a0c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.936+0000 7f88fd602640 1 -- 172.21.15.63:0/3076698686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c013440 con 0x5571ec0922b0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.936+0000 7f88fd602640 1 -- 172.21.15.63:0/3076698686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c0135e0 con 0x5571ec0922b0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.936+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5571ec093680 con 0x5571ec0922b0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.936+0000 7f88fd602640 1 -- 172.21.15.63:0/3076698686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c0138c0 con 0x5571ec0922b0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.936+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ec093b80 con 0x5571ec0922b0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.936+0000 7f88fd602640 1 -- 172.21.15.63:0/3076698686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8a0c011040 con 0x5571ec0922b0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f88fd602640 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88ec036e60 0x7f88ec039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f88fd602640 1 -- 172.21.15.63:0/3076698686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f8a0c01d070 con 0x5571ec0922b0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a135ae640 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88ec036e60 0x7f88ec039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a1792d800 1 Processor -- start 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a1792d800 1 -- start start 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a135ae640 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88ec036e60 0x7f88ec039320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8a0405b500 tx=0x7f8a04066f10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a1792d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec09fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a1792d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571ec06f990 con 0x5571ec09f670 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a13daf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec09fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a13daf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec09fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35654/0 (socket says 172.21.15.63:35654) 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a13daf640 1 -- 172.21.15.63:0/1284164167 learned_addr learned my addr 172.21.15.63:0/1284164167 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.937+0000 7f8a13daf640 1 -- 172.21.15.63:0/1284164167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ec0a0060 con 0x5571ec09f670 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a13daf640 1 --2- 172.21.15.63:0/1284164167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec09fa90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c04cca0 tx=0x7f8a0c012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fcecee1c5d10b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f88f6ffd640 1 -- 172.21.15.63:0/1284164167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c042030 con 0x5571ec09f670 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f88f6ffd640 1 -- 172.21.15.63:0/1284164167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c001310 con 0x5571ec09f670 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f88f6ffd640 1 -- 172.21.15.63:0/1284164167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c027040 con 0x5571ec09f670 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a1792d800 1 -- 172.21.15.63:0/1284164167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 msgr2=0x5571ec09fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a1792d800 1 --2- 172.21.15.63:0/1284164167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec09fa90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c04cca0 tx=0x7f8a0c012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:48.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a1792d800 1 -- 172.21.15.63:0/1284164167 shutdown_connections 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a1792d800 1 --2- 172.21.15.63:0/1284164167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec09fa90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a1792d800 1 -- 172.21.15.63:0/1284164167 >> 172.21.15.63:0/1284164167 conn(0x5571ec09b8f0 msgr2=0x5571ec099950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a1792d800 1 -- 172.21.15.63:0/1284164167 shutdown_connections 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.938+0000 7f8a1792d800 1 -- 172.21.15.63:0/1284164167 wait complete. 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a1792d800 1 Processor -- start 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a1792d800 1 -- start start 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a1792d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec0a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a1792d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571ec0a0730 con 0x5571ec09f670 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a13daf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec0a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a13daf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec0a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35670/0 (socket says 172.21.15.63:35670) 2024-04-23T19:10:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 learned_addr learned my addr 172.21.15.63:0/3390612423 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ec0a7070 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.939+0000 7f8a13daf640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec0a6b30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c001df0 tx=0x7f8a0c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.940+0000 7f88f57fa640 1 -- 172.21.15.63:0/3390612423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c027030 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.940+0000 7f88f57fa640 1 -- 172.21.15.63:0/3390612423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c02a350 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.940+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5571ec0a7300 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.940+0000 7f88f57fa640 1 -- 172.21.15.63:0/3390612423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c02a650 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.940+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ec0a9fb0 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.940+0000 7f88f57fa640 1 -- 172.21.15.63:0/3390612423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8a0c045070 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.940+0000 7f88f57fa640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88dc036e60 0x7f88dc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.941+0000 7f88f57fa640 1 -- 172.21.15.63:0/3390612423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f8a0c043070 con 0x5571ec09f670 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.941+0000 7f8a135ae640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88dc036e60 0x7f88dc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.941+0000 7f8a135ae640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88dc036e60 0x7f88dc039320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a0408e010 tx=0x7f8a0408a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.941+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.941+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ec0b2810 0x5571ec0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.941+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0b5420 con 0x5571ec0b2810 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.941+0000 7f8a12dad640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ec0b2810 0x5571ec0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a12dad640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ec0b2810 0x5571ec0b4cd0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 realm 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0b1c60 con 0x5571ec0b2810 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0b5f10 con 0x5571ec0b2810 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0b67e0 con 0x5571ec0b2810 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.942+0000 7f8a1792d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0b6b80 con 0x5571ec0b2810 2024-04-23T19:10:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a1792d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a1792d800 20 searching for the correct realm 2024-04-23T19:10:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5571ec0b8ab0 0x5571ec0baec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x5571ec0b8ab0 2024-04-23T19:10:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a13daf640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5571ec0b8ab0 0x5571ec0baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.943+0000 7f8a13daf640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5571ec0b8ab0 0x5571ec0baec0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a13daf640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8a0c0123a0 0x7f8a0c046790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x7f8a0c0123a0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a135ae640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8a0c0123a0 0x7f8a0c046790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a135ae640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8a0c0123a0 0x7f8a0c046790 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a135ae640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8a0408e0b0 0x7f8a0408b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.944+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x7f8a0408e0b0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.945+0000 7f8a12dad640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8a0408e0b0 0x7f8a0408b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.945+0000 7f8a12dad640 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8a0408e0b0 0x7f8a0408b180 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.945+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.945+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x5571ec0b2810 2024-04-23T19:10:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.945+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.945+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x5571ec0b2810 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x7f8a0408e0b0 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x7f8a0408e0b0 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x7f8a0c0123a0 2024-04-23T19:10:48.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.946+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x5571ec0b8ab0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.947+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.947+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x5571ec0b2810 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.947+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.947+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x7f8a0408e0b0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.947+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.947+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x5571ec0b2810 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bb590 con 0x7f8a0c0123a0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b8ab0 2024-04-23T19:10:48.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b8ab0 2024-04-23T19:10:48.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.948+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0408e0b0 2024-04-23T19:10:48.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.949+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.949+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b8ab0 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.949+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.949+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b8ab0 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b2810 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b2810 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0408e0b0 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.950+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b8ab0 2024-04-23T19:10:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0c0123a0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b2810 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0c0123a0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.951+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b8ab0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0408e0b0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0c0123a0 2024-04-23T19:10:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b8ab0 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.952+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0408e0b0 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x5571ec0b2810 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f8a040a8060 con 0x7f8a0c0123a0 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:48.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0b7550 con 0x5571ec0b2810 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.953+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0b7df0 con 0x5571ec0b2810 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bba10 con 0x7f8a0408e0b0 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 zone default found 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 4 Realm: () 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 zonegroup default 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bbac0 con 0x7f8a0c0123a0 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.954+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bdd90 con 0x5571ec0b8ab0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a1792d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0be130 con 0x5571ec0b8ab0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a1792d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a1792d800 20 started sync module instance, tier type = 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a1792d800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.955+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bd220 con 0x7f8a0c0123a0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.956+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.956+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93947779864080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bce80 con 0x7f8a0c0123a0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.956+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bef10 con 0x5571ec0b2810 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.957+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93947779864080] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.957+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.957+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93947779868016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bf300 con 0x5571ec0b2810 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.957+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bfd20 con 0x5571ec0b8ab0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.958+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93947779868016] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.959+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.959+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93947779872784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c0110 con 0x5571ec0b8ab0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.959+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c0b70 con 0x7f8a0c0123a0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.961+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93947779872784] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.961+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.961+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93947779869072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c0b70 con 0x7f8a0c0123a0 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.961+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c15f0 con 0x5571ec0b2810 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.962+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.962+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93947779879168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c15f0 con 0x5571ec0b2810 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.962+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c2610 con 0x7f8a0408e0b0 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.962+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93947779869072] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.963+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93947779879168] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.963+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.963+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93947779883232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c2610 con 0x7f8a0408e0b0 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.963+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c2a40 con 0x7f8a0408e0b0 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.965+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93947779883232] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.965+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.965+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93947779881408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c2a00 con 0x7f8a0408e0b0 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.965+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c3440 con 0x7f8a0408e0b0 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93947779886864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c3400 con 0x7f8a0408e0b0 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 20 add_watcher() i=0 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 20 add_watcher() i=1 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 20 add_watcher() i=2 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 20 add_watcher() i=3 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 20 add_watcher() i=4 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 20 add_watcher() i=5 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93947779881408] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.966+0000 7f8a1792d800 20 add_watcher() i=6 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.967+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93947779886864] v32'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.967+0000 7f8a1792d800 20 add_watcher() i=7 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.967+0000 7f8a1792d800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.967+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c0650 con 0x5571ec0b2810 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.967+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.967+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93947779889120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c0650 con 0x5571ec0b2810 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.968+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93947779889120] v32'173 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.968+0000 7f88c9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.968+0000 7f8a1792d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.968+0000 7f8a1792d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.968+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0cf240 con 0x5571ec0b2810 2024-04-23T19:10:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.970+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'30 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.970+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0cf910 con 0x7f8a0408e0b0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.971+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.971+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0cfd30 con 0x5571ec0b8ab0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.972+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.972+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0cfcf0 con 0x5571ec0b2810 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.973+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'174 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.973+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0280 con 0x7f8a0c0123a0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.974+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'59 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.974+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0730 con 0x5571ec0b8ab0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.975+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'39 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.975+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0730 con 0x5571ec0b2810 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.976+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.976+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0730 con 0x7f8a0408e0b0 2024-04-23T19:10:48.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.977+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'39 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.977+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0730 con 0x7f8a0c0123a0 2024-04-23T19:10:48.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.978+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.978+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d07c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.979+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.979+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d07c0 con 0x7f8a0c0123a0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.980+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'60 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.980+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0850 con 0x7f8a0408e0b0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.981+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.981+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0850 con 0x7f8a0c0123a0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.982+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.982+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0850 con 0x7f8a0408e0b0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.983+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'45 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.983+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0850 con 0x5571ec0b2810 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.984+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'175 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.984+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0850 con 0x5571ec0b8ab0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.985+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'54 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.985+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d0850 con 0x7f8a0c0123a0 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.986+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'61 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.986+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.987+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.987+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b2810 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.988+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'31 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.988+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b2810 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.989+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'176 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.989+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.990+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'55 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.990+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x7f8a0c0123a0 2024-04-23T19:10:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.991+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'33 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.991+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.992+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.992+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.993+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.993+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.994+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'19 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.994+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.995+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'40 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.995+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.996+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.996+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.997+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.997+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.998+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'40 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:48.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.998+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x5571ec0b8ab0 2024-04-23T19:10:48.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.999+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'56 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:49.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:48.999+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x7f8a0408e0b0 2024-04-23T19:10:49.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.000+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'46 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:49.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.000+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0d08c0 con 0x7f8a0c0123a0 2024-04-23T19:10:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.001+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.001+0000 7f88c8fe1640 20 reqs_thread_entry: start 2024-04-23T19:10:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.001+0000 7f8a1792d800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 20 Filter name: none 2024-04-23T19:10:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f88427fc640 20 reqs_thread_entry: start 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0fbac0 con 0x5571ec0b2810 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0fdf10 con 0x7f8a0c0123a0 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.002+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:49.004 INFO:teuthology.orchestra.run.smithi063.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-23T19:10:49.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.003+0000 7f88c9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.003+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93947779889120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0ce0b0 con 0x5571ec0b2810 2024-04-23T19:10:49.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.004+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 93947779889120] v32'177 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:49.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.004+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93947779864080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0cdc00 con 0x7f8a0c0123a0 2024-04-23T19:10:49.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.005+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 93947779864080] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:49.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.005+0000 7f8a1792d800 20 remove_watcher() i=0 2024-04-23T19:10:49.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.005+0000 7f8a1792d800 2 removed watcher, disabling cache 2024-04-23T19:10:49.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.005+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93947779868016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bd610 con 0x5571ec0b2810 2024-04-23T19:10:49.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.007+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 93947779868016] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:49.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.007+0000 7f8a1792d800 20 remove_watcher() i=1 2024-04-23T19:10:49.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.007+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93947779872784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0be570 con 0x5571ec0b8ab0 2024-04-23T19:10:49.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.007+0000 7f8a13daf640 1 -- 172.21.15.63:0/3390612423 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 93947779872784] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0c029070 con 0x5571ec0b8ab0 2024-04-23T19:10:49.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.007+0000 7f8a1792d800 20 remove_watcher() i=2 2024-04-23T19:10:49.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.007+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93947779869072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0be570 con 0x7f8a0c0123a0 2024-04-23T19:10:49.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.008+0000 7f8a135ae640 1 -- 172.21.15.63:0/3390612423 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 93947779869072] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a04099070 con 0x7f8a0c0123a0 2024-04-23T19:10:49.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.008+0000 7f8a1792d800 20 remove_watcher() i=3 2024-04-23T19:10:49.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.008+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93947779879168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bf810 con 0x5571ec0b2810 2024-04-23T19:10:49.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.009+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 93947779879168] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x5571ec0b2810 2024-04-23T19:10:49.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.009+0000 7f8a1792d800 20 remove_watcher() i=4 2024-04-23T19:10:49.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.009+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93947779883232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bf810 con 0x7f8a0408e0b0 2024-04-23T19:10:49.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.010+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 93947779883232] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:49.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.010+0000 7f8a1792d800 20 remove_watcher() i=5 2024-04-23T19:10:49.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.010+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93947779881408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0bf810 con 0x7f8a0408e0b0 2024-04-23T19:10:49.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.011+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 93947779881408] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:49.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.011+0000 7f8a1792d800 20 remove_watcher() i=6 2024-04-23T19:10:49.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.011+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93947779886864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5571ec0c19c0 con 0x7f8a0408e0b0 2024-04-23T19:10:49.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 93947779886864] v32'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8a0800f5f0 con 0x7f8a0408e0b0 2024-04-23T19:10:49.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 20 remove_watcher() i=7 2024-04-23T19:10:49.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8a0408e0b0 msgr2=0x7f8a0408b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f8a0408e0b0 0x7f8a0408b180 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ec0b2810 msgr2=0x5571ec0b4cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ec0b2810 0x5571ec0b4cd0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8a0c0123a0 msgr2=0x7f8a0c046790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f8a0c0123a0 0x7f8a0c046790 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5571ec0b8ab0 msgr2=0x5571ec0baec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.012+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5571ec0b8ab0 0x5571ec0baec0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88dc036e60 msgr2=0x7f88dc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88dc036e60 0x7f88dc039320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a0408e010 tx=0x7f8a0408a000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 msgr2=0x5571ec0a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec0a6b30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c001df0 tx=0x7f8a0c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a12dad640 1 -- 172.21.15.63:0/3390612423 reap_dead start 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 shutdown_connections 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3390612423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec09f670 0x5571ec0a6b30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 >> 172.21.15.63:0/3390612423 conn(0x5571ec09b8f0 msgr2=0x5571ec098f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 shutdown_connections 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3390612423 wait complete. 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88ec036e60 msgr2=0x7f88ec039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88ec036e60 0x7f88ec039320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8a0405b500 tx=0x7f8a04066f10 comp rx=0 tx=0).stop 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec0922b0 msgr2=0x5571ec0926d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec0922b0 0x5571ec0926d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x5571ec077f90 tx=0x7f8a0c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 shutdown_connections 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88ec036e60 0x7f88ec039320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/3076698686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ec0922b0 0x5571ec0926d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 >> 172.21.15.63:0/3076698686 conn(0x5571ec086350 msgr2=0x5571ec086750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 shutdown_connections 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/3076698686 wait complete. 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5571ec0814f0 msgr2=0x5571ec0838e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5571ec0814f0 0x5571ec0838e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ebf814d0 msgr2=0x5571ec07ddd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.013+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ebf814d0 0x5571ec07ddd0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88f8036eb0 msgr2=0x7f88f8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88f8036eb0 0x7f88f8039370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a08000c00 tx=0x7f8a080013b0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 msgr2=0x5571ec076b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec076b90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005400 tx=0x7f8a04002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 shutdown_connections 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5571ec0814f0 0x5571ec0838e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5571ebf814d0 0x5571ec07ddd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f88f8036eb0 0x7f88f8039370 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 --2- 172.21.15.63:0/2442720019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571ebf7eba0 0x5571ec076b90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 >> 172.21.15.63:0/2442720019 conn(0x5571ebf78630 msgr2=0x5571ec069220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 shutdown_connections 2024-04-23T19:10:49.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.014+0000 7f8a1792d800 1 -- 172.21.15.63:0/2442720019 wait complete. 2024-04-23T19:10:49.022 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-23T19:10:49.031 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-04-23T19:10:49.036 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:49.068 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:49.082 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:49.127 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-23T19:10:49.132 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:49.132 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-23T19:10:49.132 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-23T19:10:49.132 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-23T19:10:49.177 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:49.177 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:49.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.177+0000 7f5fbd015800 1 Processor -- start 2024-04-23T19:10:49.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.177+0000 7f5fbd015800 1 -- start start 2024-04-23T19:10:49.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fbd015800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55c927718400 0x55c927718820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fbd015800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c9275cc7f0 con 0x55c92771b3f0 2024-04-23T19:10:49.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fbd015800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55c9275bbe20 con 0x55c927718400 2024-04-23T19:10:49.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fb85dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55c927718400 0x55c927718820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fb7ddb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55c92771b3f0 0x55c92771b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49682/0 (socket says 172.21.15.63:49682) 2024-04-23T19:10:49.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/2392517157 learned_addr learned my addr 172.21.15.63:0/2392517157 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:49.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fb85dc640 1 -- 172.21.15.63:0/2392517157 >> v1:172.21.15.16:6789/0 conn(0x55c92771b3f0 legacy=0x55c92771b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.178+0000 7f5fb85dc640 1 -- 172.21.15.63:0/2392517157 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9275dab50 con 0x55c927718400 2024-04-23T19:10:49.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.180+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/2392517157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927718820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800a470 tx=0x7f5fa801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2813082552abf619 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.180+0000 7f5fb75da640 1 -- 172.21.15.63:0/2392517157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8057070 con 0x55c927718400 2024-04-23T19:10:49.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.180+0000 7f5fb75da640 1 -- 172.21.15.63:0/2392517157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa804ad80 con 0x55c927718400 2024-04-23T19:10:49.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.180+0000 7f5fb75da640 1 -- 172.21.15.63:0/2392517157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8052590 con 0x55c927718400 2024-04-23T19:10:49.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.181+0000 7f5fbd015800 1 -- 172.21.15.63:0/2392517157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 msgr2=0x55c927718820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.181+0000 7f5fbd015800 1 --2- 172.21.15.63:0/2392517157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927718820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800a470 tx=0x7f5fa801ce50 comp rx=0 tx=0).stop 2024-04-23T19:10:49.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.181+0000 7f5fbd015800 1 -- 172.21.15.63:0/2392517157 shutdown_connections 2024-04-23T19:10:49.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.181+0000 7f5fbd015800 1 --2- 172.21.15.63:0/2392517157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927718820 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.181+0000 7f5fbd015800 1 -- 172.21.15.63:0/2392517157 >> 172.21.15.63:0/2392517157 conn(0x55c927713db0 msgr2=0x55c9277161d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.181+0000 7f5fbd015800 1 -- 172.21.15.63:0/2392517157 shutdown_connections 2024-04-23T19:10:49.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.181+0000 7f5fbd015800 1 -- 172.21.15.63:0/2392517157 wait complete. 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.193+0000 7f5fbd015800 1 Processor -- start 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fbd015800 1 -- start start 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fbd015800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c92771ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fbd015800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9276656f0 con 0x55c927718400 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fb85dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c92771ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fb85dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c92771ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35692/0 (socket says 172.21.15.63:35692) 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fb85dc640 1 -- 172.21.15.63:0/4012433123 learned_addr learned my addr 172.21.15.63:0/4012433123 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fb85dc640 1 -- 172.21.15.63:0/4012433123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9277192f0 con 0x55c927718400 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.194+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/4012433123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c92771ad30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8055040 tx=0x7f5fa804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75dea5450126754 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5ea660c640 1 -- 172.21.15.63:0/4012433123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8057040 con 0x55c927718400 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5ea660c640 1 -- 172.21.15.63:0/4012433123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa8063070 con 0x55c927718400 2024-04-23T19:10:49.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5ea660c640 1 -- 172.21.15.63:0/4012433123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa80527e0 con 0x55c927718400 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5fbd015800 1 -- 172.21.15.63:0/4012433123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 msgr2=0x55c92771ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5fbd015800 1 --2- 172.21.15.63:0/4012433123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c92771ad30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8055040 tx=0x7f5fa804c000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5fbd015800 1 -- 172.21.15.63:0/4012433123 shutdown_connections 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5fbd015800 1 --2- 172.21.15.63:0/4012433123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c92771ad30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5fbd015800 1 -- 172.21.15.63:0/4012433123 >> 172.21.15.63:0/4012433123 conn(0x55c927713db0 msgr2=0x55c927807240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5fbd015800 1 -- 172.21.15.63:0/4012433123 shutdown_connections 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.195+0000 7f5fbd015800 1 -- 172.21.15.63:0/4012433123 wait complete. 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fbd015800 1 Processor -- start 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fbd015800 1 -- start start 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fbd015800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927815550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fbd015800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9277238f0 con 0x55c927718400 2024-04-23T19:10:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fb85dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927815550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fb85dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927815550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35696/0 (socket says 172.21.15.63:35696) 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3437591391 learned_addr learned my addr 172.21.15.63:0/3437591391 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.196+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c927815a90 con 0x55c927718400 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.197+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927815550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8060040 tx=0x7f5fa8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.197+0000 7f5ea4e09640 1 -- 172.21.15.63:0/3437591391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8057050 con 0x55c927718400 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.197+0000 7f5ea4e09640 1 -- 172.21.15.63:0/3437591391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa8052d10 con 0x55c927718400 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.197+0000 7f5ea4e09640 1 -- 172.21.15.63:0/3437591391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa805d470 con 0x55c927718400 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.197+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c927815d20 con 0x55c927718400 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.197+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9278189d0 con 0x55c927718400 2024-04-23T19:10:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5ea4e09640 1 -- 172.21.15.63:0/3437591391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5fa805d610 con 0x55c927718400 2024-04-23T19:10:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5ea4e09640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5ea0036eb0 0x7f5ea0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5ea4e09640 1 -- 172.21.15.63:0/3437591391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f5fa808e4a0 con 0x55c927718400 2024-04-23T19:10:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5fb7ddb640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5ea0036eb0 0x7f5ea0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c92771b3f0 0x55c92781c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92781cd10 con 0x55c92771b3f0 2024-04-23T19:10:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c92771b3f0 0x55c92781c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5fb7ddb640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5ea0036eb0 0x7f5ea0039370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000f60 tx=0x7f5fac0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.198+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c92771b3f0 0x55c92781c790 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.199+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3437591391 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5fb000a4a0 con 0x55c92771b3f0 2024-04-23T19:10:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.199+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927819d60 con 0x55c92771b3f0 2024-04-23T19:10:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.199+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3437591391 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5fb000a4a0 con 0x55c92771b3f0 2024-04-23T19:10:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.199+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92781eb10 con 0x55c92771b3f0 2024-04-23T19:10:49.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.199+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3437591391 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5fb000a4a0 con 0x55c92771b3f0 2024-04-23T19:10:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.199+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92781de50 con 0x55c92771b3f0 2024-04-23T19:10:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.200+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3437591391 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5fb000a4a0 con 0x55c92771b3f0 2024-04-23T19:10:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.200+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c92781fec0 0x55c9278222b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.200+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927822830 con 0x55c92781fec0 2024-04-23T19:10:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.200+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c92781fec0 0x55c9278222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.200+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c92781fec0 0x55c9278222b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3437591391 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5fa8078070 con 0x55c92781fec0 2024-04-23T19:10:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fbd015800 1 Processor -- start 2024-04-23T19:10:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fbd015800 1 -- start start 2024-04-23T19:10:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fbd015800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927830c50 0x55c927831070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fbd015800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c92760be30 con 0x55c927830c50 2024-04-23T19:10:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fb8ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927830c50 0x55c927831070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fb8ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927830c50 0x55c927831070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35710/0 (socket says 172.21.15.63:35710) 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.201+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/2801631821 learned_addr learned my addr 172.21.15.63:0/2801631821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.202+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/2801631821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9278315b0 con 0x55c927830c50 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.202+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927830c50 0x55c927831070 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55c927816950 tx=0x7f5fb00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.202+0000 7f5e9dffb640 1 -- 172.21.15.63:0/2801631821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0013440 con 0x55c927830c50 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.202+0000 7f5e9dffb640 1 -- 172.21.15.63:0/2801631821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb00135e0 con 0x55c927830c50 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.202+0000 7f5e9dffb640 1 -- 172.21.15.63:0/2801631821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb00138c0 con 0x55c927830c50 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.202+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c927832020 con 0x55c927830c50 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.202+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c927832520 con 0x55c927830c50 2024-04-23T19:10:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5e9dffb640 1 -- 172.21.15.63:0/2801631821 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5fb0011040 con 0x55c927830c50 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5e9dffb640 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e90036e60 0x7f5e90039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5e9dffb640 1 -- 172.21.15.63:0/2801631821 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f5fb001d070 con 0x55c927830c50 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e90036e60 0x7f5e90039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5fbd015800 1 Processor -- start 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5fbd015800 1 -- start start 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e90036e60 0x7f5e90039320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5fa808ca40 tx=0x7f5fa805c8e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5fbd015800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c92783e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.203+0000 7f5fbd015800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9277195b0 con 0x55c92783dfb0 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5fb8ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c92783e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5fb8ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c92783e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35726/0 (socket says 172.21.15.63:35726) 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/525984739 learned_addr learned my addr 172.21.15.63:0/525984739 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/525984739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92783e9a0 con 0x55c92783dfb0 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/525984739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c92783e3d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5fb004ccb0 tx=0x7f5fb0012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f708376d1cc09c09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5e8f7fe640 1 -- 172.21.15.63:0/525984739 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0012df0 con 0x55c92783dfb0 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5e8f7fe640 1 -- 172.21.15.63:0/525984739 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb0001310 con 0x55c92783dfb0 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.204+0000 7f5e8f7fe640 1 -- 172.21.15.63:0/525984739 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0001630 con 0x55c92783dfb0 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 -- 172.21.15.63:0/525984739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 msgr2=0x55c92783e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 --2- 172.21.15.63:0/525984739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c92783e3d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5fb004ccb0 tx=0x7f5fb0012910 comp rx=0 tx=0).stop 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 -- 172.21.15.63:0/525984739 shutdown_connections 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 --2- 172.21.15.63:0/525984739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c92783e3d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 -- 172.21.15.63:0/525984739 >> 172.21.15.63:0/525984739 conn(0x55c92783a2d0 msgr2=0x55c927838330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 -- 172.21.15.63:0/525984739 shutdown_connections 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 -- 172.21.15.63:0/525984739 wait complete. 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 Processor -- start 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 -- start start 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c9278455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fbd015800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c92783f120 con 0x55c92783dfb0 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fb8ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c9278455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fb8ddd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c9278455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35728/0 (socket says 172.21.15.63:35728) 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.205+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 learned_addr learned my addr 172.21.15.63:0/3443041108 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.206+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c927845ae0 con 0x55c92783dfb0 2024-04-23T19:10:49.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.206+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c9278455a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5fb004bb80 tx=0x7f5fb004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.206+0000 7f5e8dffb640 1 -- 172.21.15.63:0/3443041108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0001c30 con 0x55c92783dfb0 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.206+0000 7f5e8dffb640 1 -- 172.21.15.63:0/3443041108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb002ae30 con 0x55c92783dfb0 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.206+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c927845d70 con 0x55c92783dfb0 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.206+0000 7f5e8dffb640 1 -- 172.21.15.63:0/3443041108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0021db0 con 0x55c92783dfb0 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.206+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c927848a20 con 0x55c92783dfb0 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5e8dffb640 1 -- 172.21.15.63:0/3443041108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5fb0040040 con 0x55c92783dfb0 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5e8dffb640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e80036eb0 0x7f5e80039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5e8dffb640 1 -- 172.21.15.63:0/3443041108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f5fb0032070 con 0x55c92783dfb0 2024-04-23T19:10:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e80036eb0 0x7f5e80039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e80036eb0 0x7f5e80039370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5fa805c970 tx=0x7f5fa8086000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c9278511d0 0x55c927853690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.207+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927853de0 con 0x55c9278511d0 2024-04-23T19:10:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fb7ddb640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c9278511d0 0x55c927853690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fb7ddb640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c9278511d0 0x55c927853690 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5fac009050 con 0x55c9278511d0 2024-04-23T19:10:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 20 realm 2024-04-23T19:10:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927850620 con 0x55c9278511d0 2024-04-23T19:10:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5fac009050 con 0x55c9278511d0 2024-04-23T19:10:49.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:49.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c9278548d0 con 0x55c9278511d0 2024-04-23T19:10:49.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.208+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5fac009050 con 0x55c9278511d0 2024-04-23T19:10:49.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c9278551a0 con 0x55c9278511d0 2024-04-23T19:10:49.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5fac009050 con 0x55c9278511d0 2024-04-23T19:10:49.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927855540 con 0x55c9278511d0 2024-04-23T19:10:49.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5fac009050 con 0x55c9278511d0 2024-04-23T19:10:49.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:49.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 20 searching for the correct realm 2024-04-23T19:10:49.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c927857420 0x55c927859810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.209+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c927857420 0x55c927859810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.210+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c927857420 0x55c927859810 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.210+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.210+0000 7f5fb8ddd640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5fb00123a0 0x7f5fb002b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.210+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fb00123a0 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.210+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5fb00123a0 0x7f5fb002b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.210+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5fb00123a0 0x7f5fb002b840 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.211+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.211+0000 7f5fb85dc640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5fa809b040 0x7f5fa8066380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.211+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.211+0000 7f5fb7ddb640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5fa809b040 0x7f5fa8066380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.211+0000 7f5fb7ddb640 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5fa809b040 0x7f5fa8066380 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac009050 con 0x7f5fa809b040 2024-04-23T19:10:49.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac009050 con 0x55c9278511d0 2024-04-23T19:10:49.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.212+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fb00123a0 2024-04-23T19:10:49.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.213+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fb00123a0 2024-04-23T19:10:49.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.214+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.215+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.215+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.215+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.215+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.215+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.215+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.215+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.216+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fb00123a0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.217+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.217+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.217+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.217+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fb00123a0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.217+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.217+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.217+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.218+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.218+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.218+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fb00123a0 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.218+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.218+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c927857420 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.219+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.219+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fa809b040 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.219+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.219+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x55c9278511d0 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.219+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.219+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55c927859ec0 con 0x7f5fb00123a0 2024-04-23T19:10:49.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927855ec0 con 0x55c9278511d0 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927856760 con 0x55c9278511d0 2024-04-23T19:10:49.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92785a340 con 0x7f5fa809b040 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.220+0000 7f5fbd015800 20 zone default found 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 4 Realm: () 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 zonegroup default 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92785a7f0 con 0x7f5fb00123a0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92785c570 con 0x55c927857420 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92785ca50 con 0x55c927857420 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 started sync module instance, tier type = 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.221+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785ba10 con 0x7f5fb00123a0 2024-04-23T19:10:49.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.222+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.222+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94322439863808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785d360 con 0x7f5fb00123a0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.222+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785dbb0 con 0x55c9278511d0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.223+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94322439863808] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.224+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.224+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94322439868080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785dc70 con 0x55c9278511d0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.224+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785e760 con 0x55c927857420 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.225+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94322439868080] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.225+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.225+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94322439873072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785e720 con 0x55c927857420 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.225+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785f1e0 con 0x7f5fb00123a0 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.226+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94322439873072] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.226+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.226+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94322439870208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785f580 con 0x7f5fb00123a0 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.226+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785ffb0 con 0x55c9278511d0 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.227+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94322439870208] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.227+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.228+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94322439879360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92785ffb0 con 0x55c9278511d0 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.228+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c927860a70 con 0x7f5fa809b040 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.228+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94322439879360] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.229+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.229+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94322439882048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c927860a30 con 0x7f5fa809b040 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.229+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c927861450 con 0x7f5fa809b040 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.230+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94322439882048] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.230+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.230+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94322439884608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c927861410 con 0x7f5fa809b040 2024-04-23T19:10:49.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.230+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c927861e50 con 0x7f5fa809b040 2024-04-23T19:10:49.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94322439884608] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94322439887136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c927861e10 con 0x7f5fa809b040 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 20 add_watcher() i=0 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 20 add_watcher() i=1 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 20 add_watcher() i=2 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 20 add_watcher() i=3 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 20 add_watcher() i=4 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 20 add_watcher() i=5 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.231+0000 7f5fbd015800 20 add_watcher() i=6 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.232+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94322439887136] v32'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.232+0000 7f5fbd015800 20 add_watcher() i=7 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.232+0000 7f5fbd015800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.232+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927865240 con 0x55c9278511d0 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.232+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.232+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94322439875824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c927862760 con 0x55c9278511d0 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.233+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94322439875824] v32'178 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.233+0000 7f5e6e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.233+0000 7f5fbd015800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.233+0000 7f5fbd015800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5fbd015800 5 note: GC not initialized 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5e6d7e2640 20 reqs_thread_entry: start 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5fbd015800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5fbd015800 20 Filter name: none 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5ddeffd640 20 reqs_thread_entry: start 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5fbd015800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.234+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c927898d60 con 0x7f5fb00123a0 2024-04-23T19:10:49.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 : miss 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.8 8:17d7809a:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92789a820 con 0x55c9278511d0 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv11 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:49.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 info.flags=0x17 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 to cache LRU end 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 20 get_obj_state: octx=0x7f5fac00e510 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55c92789cec8 s->prefetch_data=0 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.235+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 3.6s0 3:63673d0d:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x55c9278511d0 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.236+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=849b,stat out=16b] v0'0 uv4 ondisk = 0) ==== 262+0+865 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.236+0000 7f5fbd015800 10 manifest: total_size = 5 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.236+0000 7f5fbd015800 20 get_obj_state: setting s->obj_tag to _0l9pknC8GLF7Q6fY1xaM1MuaUOAbayG 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.237+0000 7f5e6e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.237+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94322439875824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x55c9278511d0 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.238+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94322439875824] v32'179 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.238+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94322439863808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x7f5fb00123a0 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.239+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94322439863808] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.239+0000 7f5fbd015800 20 remove_watcher() i=0 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.239+0000 7f5fbd015800 2 removed watcher, disabling cache 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.239+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94322439868080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x55c9278511d0 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.240+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94322439868080] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.240+0000 7f5fbd015800 20 remove_watcher() i=1 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.240+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94322439873072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x55c927857420 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.241+0000 7f5fb8ddd640 1 -- 172.21.15.63:0/3443041108 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94322439873072] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fb0029070 con 0x55c927857420 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.241+0000 7f5fbd015800 20 remove_watcher() i=2 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.241+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94322439870208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x7f5fb00123a0 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.242+0000 7f5fb85dc640 1 -- 172.21.15.63:0/3443041108 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94322439870208] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fa807e040 con 0x7f5fb00123a0 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.242+0000 7f5fbd015800 20 remove_watcher() i=3 2024-04-23T19:10:49.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.242+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94322439879360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x55c9278511d0 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.243+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94322439879360] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x55c9278511d0 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.243+0000 7f5fbd015800 20 remove_watcher() i=4 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.243+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94322439882048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x7f5fa809b040 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.244+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94322439882048] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.244+0000 7f5fbd015800 20 remove_watcher() i=5 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.244+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94322439884608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x7f5fa809b040 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.245+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94322439884608] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.245+0000 7f5fbd015800 20 remove_watcher() i=6 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.245+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94322439887136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55c92789dfc0 con 0x7f5fa809b040 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94322439887136] v32'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5fac02a040 con 0x7f5fa809b040 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 20 remove_watcher() i=7 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5fa809b040 msgr2=0x7f5fa8066380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f5fa809b040 0x7f5fa8066380 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c9278511d0 msgr2=0x55c927853690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c9278511d0 0x55c927853690 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5fb00123a0 msgr2=0x7f5fb002b840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f5fb00123a0 0x7f5fb002b840 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c927857420 msgr2=0x55c927859810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.246+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55c927857420 0x55c927859810 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e80036eb0 msgr2=0x7f5e80039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e80036eb0 0x7f5e80039370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5fa805c970 tx=0x7f5fa8086000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 msgr2=0x55c9278455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fb7ddb640 1 -- 172.21.15.63:0/3443041108 reap_dead start 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c9278455a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5fb004bb80 tx=0x7f5fb004cc40 comp rx=0 tx=0).stop 2024-04-23T19:10:49.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 shutdown_connections 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3443041108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c92783dfb0 0x55c9278455a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 >> 172.21.15.63:0/3443041108 conn(0x55c92783a2d0 msgr2=0x55c927837930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 shutdown_connections 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3443041108 wait complete. 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e90036e60 msgr2=0x7f5e90039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e90036e60 0x7f5e90039320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5fa808ca40 tx=0x7f5fa805c8e0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927830c50 msgr2=0x55c927831070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927830c50 0x55c927831070 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55c927816950 tx=0x7f5fb00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 shutdown_connections 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e90036e60 0x7f5e90039320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/2801631821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927830c50 0x55c927831070 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 >> 172.21.15.63:0/2801631821 conn(0x55c927824cf0 msgr2=0x55c9278250f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 shutdown_connections 2024-04-23T19:10:49.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/2801631821 wait complete. 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c92781fec0 msgr2=0x55c9278222b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c92781fec0 0x55c9278222b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c92771b3f0 msgr2=0x55c92781c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.247+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c92771b3f0 0x55c92781c790 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5ea0036eb0 msgr2=0x7f5ea0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5ea0036eb0 0x7f5ea0039370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000f60 tx=0x7f5fac0012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 msgr2=0x55c927815550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927815550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8060040 tx=0x7f5fa8002fb0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 shutdown_connections 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55c92781fec0 0x55c9278222b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55c92771b3f0 0x55c92781c790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5ea0036eb0 0x7f5ea0039370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 --2- 172.21.15.63:0/3437591391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c927718400 0x55c927815550 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 >> 172.21.15.63:0/3437591391 conn(0x55c927713db0 msgr2=0x55c927719b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 shutdown_connections 2024-04-23T19:10:49.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.248+0000 7f5fbd015800 1 -- 172.21.15.63:0/3437591391 wait complete. 2024-04-23T19:10:49.273 INFO:teuthology.orchestra.run.smithi063.stdout:fooFoofooFooFULL_CONTROL 2024-04-23T19:10:49.274 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-04-23T19:10:49.276 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-04-23T19:10:49.281 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:49.281 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-23T19:10:49.281 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-23T19:10:49.281 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-23T19:10:49.328 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:49.328 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69eb613800 1 Processor -- start 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69eb613800 1 -- start start 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69eb613800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x556a73aa04a0 0x556a73aa2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69eb613800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556a7394f7f0 con 0x556a73a9c010 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69eb613800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x556a7393ee20 con 0x556a73aa04a0 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69e6413640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x556a73aa04a0 0x556a73aa2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69e6c14640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x556a73a9c010 0x556a73a9c430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49696/0 (socket says 172.21.15.63:49696) 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69e6c14640 1 -- 172.21.15.63:0/2188971055 learned_addr learned my addr 172.21.15.63:0/2188971055 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69e6413640 1 --2- 172.21.15.63:0/2188971055 >> v2:172.21.15.16:3300/0 conn(0x556a73aa04a0 0x556a73aa2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35730/0 (socket says 172.21.15.63:35730) 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69e6413640 1 -- 172.21.15.63:0/2188971055 >> v1:172.21.15.16:6789/0 conn(0x556a73a9c010 legacy=0x556a73a9c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69e5c12640 1 -- 172.21.15.63:0/2188971055 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1141221472 0 0) 0x556a7394f7f0 con 0x556a73a9c010 2024-04-23T19:10:49.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.329+0000 7f69e6413640 1 -- 172.21.15.63:0/2188971055 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7395db50 con 0x556a73aa04a0 2024-04-23T19:10:49.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.331+0000 7f69e6413640 1 --2- 172.21.15.63:0/2188971055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73aa04a0 0x556a73aa2890 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00a470 tx=0x7f69dc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e6c67aa2476c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69e5c12640 1 -- 172.21.15.63:0/2188971055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69dc057070 con 0x556a73aa04a0 2024-04-23T19:10:49.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69e5c12640 1 -- 172.21.15.63:0/2188971055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69dc052570 con 0x556a73aa04a0 2024-04-23T19:10:49.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69e5c12640 1 -- 172.21.15.63:0/2188971055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69dc052870 con 0x556a73aa04a0 2024-04-23T19:10:49.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69eb613800 1 -- 172.21.15.63:0/2188971055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73aa04a0 msgr2=0x556a73aa2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69eb613800 1 --2- 172.21.15.63:0/2188971055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73aa04a0 0x556a73aa2890 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00a470 tx=0x7f69dc01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:49.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69eb613800 1 -- 172.21.15.63:0/2188971055 shutdown_connections 2024-04-23T19:10:49.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69eb613800 1 --2- 172.21.15.63:0/2188971055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73aa04a0 0x556a73aa2890 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69eb613800 1 -- 172.21.15.63:0/2188971055 >> 172.21.15.63:0/2188971055 conn(0x556a73a97630 msgr2=0x556a73a99a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69eb613800 1 -- 172.21.15.63:0/2188971055 shutdown_connections 2024-04-23T19:10:49.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.332+0000 7f69eb613800 1 -- 172.21.15.63:0/2188971055 wait complete. 2024-04-23T19:10:49.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69eb613800 1 Processor -- start 2024-04-23T19:10:49.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69eb613800 1 -- start start 2024-04-23T19:10:49.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69eb613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73b8d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69eb613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a739e86f0 con 0x556a73a9c010 2024-04-23T19:10:49.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69e6c14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73b8d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69e6c14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73b8d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35732/0 (socket says 172.21.15.63:35732) 2024-04-23T19:10:49.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69e6c14640 1 -- 172.21.15.63:0/1525577306 learned_addr learned my addr 172.21.15.63:0/1525577306 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.345+0000 7f69e6c14640 1 -- 172.21.15.63:0/1525577306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a73b8d630 con 0x556a73a9c010 2024-04-23T19:10:49.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.346+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1525577306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73b8d060 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69d800b240 tx=0x7f69d800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b37358e9e5f0c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.346+0000 7f68d4e09640 1 -- 172.21.15.63:0/1525577306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d8008070 con 0x556a73a9c010 2024-04-23T19:10:49.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.346+0000 7f68d4e09640 1 -- 172.21.15.63:0/1525577306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d800d500 con 0x556a73a9c010 2024-04-23T19:10:49.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.346+0000 7f68d4e09640 1 -- 172.21.15.63:0/1525577306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d800d860 con 0x556a73a9c010 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.346+0000 7f69eb613800 1 -- 172.21.15.63:0/1525577306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 msgr2=0x556a73b8d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.346+0000 7f69eb613800 1 --2- 172.21.15.63:0/1525577306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73b8d060 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69d800b240 tx=0x7f69d800b700 comp rx=0 tx=0).stop 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 -- 172.21.15.63:0/1525577306 shutdown_connections 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 --2- 172.21.15.63:0/1525577306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73b8d060 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 -- 172.21.15.63:0/1525577306 >> 172.21.15.63:0/1525577306 conn(0x556a73a97630 msgr2=0x556a73b88980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 -- 172.21.15.63:0/1525577306 shutdown_connections 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 -- 172.21.15.63:0/1525577306 wait complete. 2024-04-23T19:10:49.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 Processor -- start 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 -- start start 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73a10100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69eb613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a73aa3160 con 0x556a73a9c010 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69e6c14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73a10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69e6c14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73a10100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35744/0 (socket says 172.21.15.63:35744) 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69e6c14640 1 -- 172.21.15.63:0/71109412 learned_addr learned my addr 172.21.15.63:0/71109412 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.347+0000 7f69e6c14640 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a73b96820 con 0x556a73a9c010 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.348+0000 7f69e6c14640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73a10100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000f60 tx=0x7f69d8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.348+0000 7f68d3606640 1 -- 172.21.15.63:0/71109412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d8019870 con 0x556a73a9c010 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.348+0000 7f68d3606640 1 -- 172.21.15.63:0/71109412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d8019a10 con 0x556a73a9c010 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.348+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556a73b9ae30 con 0x556a73a9c010 2024-04-23T19:10:49.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.348+0000 7f68d3606640 1 -- 172.21.15.63:0/71109412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d8019d10 con 0x556a73a9c010 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.348+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a73b9b350 con 0x556a73a9c010 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f68d3606640 1 -- 172.21.15.63:0/71109412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f69d8017040 con 0x556a73a9c010 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f68d3606640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68cc036eb0 0x7f68cc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f68d3606640 1 -- 172.21.15.63:0/71109412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f69d8015020 con 0x556a73a9c010 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f69e6413640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68cc036eb0 0x7f68cc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73aa04a0 0x556a73b9f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73b9f740 con 0x556a73aa04a0 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f69e7415640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73aa04a0 0x556a73b9f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f69e6413640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68cc036eb0 0x7f68cc039370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f69dc054000 tx=0x7f69dc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.349+0000 7f69e7415640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73aa04a0 0x556a73b9f1c0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.350+0000 7f69e7415640 1 -- 172.21.15.63:0/71109412 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f69e000a4a0 con 0x556a73aa04a0 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.350+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73b9c790 con 0x556a73aa04a0 2024-04-23T19:10:49.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.350+0000 7f69e7415640 1 -- 172.21.15.63:0/71109412 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f69e000a4a0 con 0x556a73aa04a0 2024-04-23T19:10:49.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.350+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73ba1540 con 0x556a73aa04a0 2024-04-23T19:10:49.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.350+0000 7f69e7415640 1 -- 172.21.15.63:0/71109412 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f69e000a4a0 con 0x556a73aa04a0 2024-04-23T19:10:49.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.350+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73ba0880 con 0x556a73aa04a0 2024-04-23T19:10:49.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.351+0000 7f69e7415640 1 -- 172.21.15.63:0/71109412 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f69e000a4a0 con 0x556a73aa04a0 2024-04-23T19:10:49.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.351+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556a73ba28f0 0x556a73ba4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.351+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73ba5260 con 0x556a73ba28f0 2024-04-23T19:10:49.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.351+0000 7f69e6c14640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556a73ba28f0 0x556a73ba4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.351+0000 7f69e6c14640 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556a73ba28f0 0x556a73ba4ce0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.351+0000 7f69e6c14640 1 -- 172.21.15.63:0/71109412 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f69d802d070 con 0x556a73ba28f0 2024-04-23T19:10:49.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69eb613800 1 Processor -- start 2024-04-23T19:10:49.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69eb613800 1 -- start start 2024-04-23T19:10:49.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69eb613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bb3610 0x556a73bb3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69eb613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a7398ee30 con 0x556a73bb3610 2024-04-23T19:10:49.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69e7415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bb3610 0x556a73bb3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69e7415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bb3610 0x556a73bb3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35758/0 (socket says 172.21.15.63:35758) 2024-04-23T19:10:49.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69e7415640 1 -- 172.21.15.63:0/1356624332 learned_addr learned my addr 172.21.15.63:0/1356624332 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.352+0000 7f69e7415640 1 -- 172.21.15.63:0/1356624332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a73bb3f70 con 0x556a73bb3610 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.353+0000 7f69e7415640 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bb3610 0x556a73bb3a30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x556a73b9fb00 tx=0x7f69e00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.353+0000 7f68d0e01640 1 -- 172.21.15.63:0/1356624332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69e0013440 con 0x556a73bb3610 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.353+0000 7f68d0e01640 1 -- 172.21.15.63:0/1356624332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e00135e0 con 0x556a73bb3610 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.353+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556a73bb49e0 con 0x556a73bb3610 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.353+0000 7f68d0e01640 1 -- 172.21.15.63:0/1356624332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69e00138c0 con 0x556a73bb3610 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.353+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a73bb4ee0 con 0x556a73bb3610 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f68d0e01640 1 -- 172.21.15.63:0/1356624332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f69e0011040 con 0x556a73bb3610 2024-04-23T19:10:49.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f68d0e01640 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68c0036e60 0x7f68c0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f68d0e01640 1 -- 172.21.15.63:0/1356624332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f69e001d070 con 0x556a73bb3610 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68c0036e60 0x7f68c0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 Processor -- start 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 -- start start 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68c0036e60 0x7f68c0039320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x556a73ba5640 tx=0x7f69d80015f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a73b8d9b0 con 0x556a73bc0980 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69e7415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69e7415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35766/0 (socket says 172.21.15.63:35766) 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69e7415640 1 -- 172.21.15.63:0/3603550202 learned_addr learned my addr 172.21.15.63:0/3603550202 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69e7415640 1 -- 172.21.15.63:0/3603550202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a73bc1370 con 0x556a73bc0980 2024-04-23T19:10:49.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69e7415640 1 --2- 172.21.15.63:0/3603550202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc0da0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f69e00013e0 tx=0x7f69e0012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afac7a598a79ebc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f68ca7fc640 1 -- 172.21.15.63:0/3603550202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69e0043070 con 0x556a73bc0980 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f68ca7fc640 1 -- 172.21.15.63:0/3603550202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e002ac50 con 0x556a73bc0980 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f68ca7fc640 1 -- 172.21.15.63:0/3603550202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69e00014f0 con 0x556a73bc0980 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 -- 172.21.15.63:0/3603550202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 msgr2=0x556a73bc0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 --2- 172.21.15.63:0/3603550202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc0da0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f69e00013e0 tx=0x7f69e0012c90 comp rx=0 tx=0).stop 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 -- 172.21.15.63:0/3603550202 shutdown_connections 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 --2- 172.21.15.63:0/3603550202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc0da0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.354+0000 7f69eb613800 1 -- 172.21.15.63:0/3603550202 >> 172.21.15.63:0/3603550202 conn(0x556a73bbcc90 msgr2=0x556a73bbacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69eb613800 1 -- 172.21.15.63:0/3603550202 shutdown_connections 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69eb613800 1 -- 172.21.15.63:0/3603550202 wait complete. 2024-04-23T19:10:49.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69eb613800 1 Processor -- start 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69eb613800 1 -- start start 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69eb613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69eb613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a73bc1ac0 con 0x556a73bc0980 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69e7415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69e7415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35770/0 (socket says 172.21.15.63:35770) 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 learned_addr learned my addr 172.21.15.63:0/1969191888 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.355+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a73bc8400 con 0x556a73bc0980 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.356+0000 7f69e7415640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc7ec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f69e004cb80 tx=0x7f69e004cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.356+0000 7f68c8ff9640 1 -- 172.21.15.63:0/1969191888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69e0043030 con 0x556a73bc0980 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.356+0000 7f68c8ff9640 1 -- 172.21.15.63:0/1969191888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e002ae00 con 0x556a73bc0980 2024-04-23T19:10:49.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.356+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556a73bc8690 con 0x556a73bc0980 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.356+0000 7f68c8ff9640 1 -- 172.21.15.63:0/1969191888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69e00126e0 con 0x556a73bc0980 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.356+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a73bcb340 con 0x556a73bc0980 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f68c8ff9640 1 -- 172.21.15.63:0/1969191888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f69e004e110 con 0x556a73bc0980 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f68c8ff9640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68b0036df0 0x7f68b00392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f68c8ff9640 1 -- 172.21.15.63:0/1969191888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f69e0044070 con 0x556a73bc0980 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68b0036df0 0x7f68b00392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68b0036df0 0x7f68b00392b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69d804d010 tx=0x7f69d8035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73bd3c20 0x556a73bd60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.357+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bd6830 con 0x556a73bd3c20 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69e6413640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73bd3c20 0x556a73bd60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69e6413640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73bd3c20 0x556a73bd60e0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 20 realm 2024-04-23T19:10:49.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bd6c00 con 0x556a73bd3c20 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.358+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bd74d0 con 0x556a73bd3c20 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bd7870 con 0x556a73bd3c20 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bd80c0 con 0x556a73bd3c20 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 20 searching for the correct realm 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556a73bd9e30 0x556a73bdc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd9e30 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.359+0000 7f69e7415640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556a73bd9e30 0x556a73bdc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.360+0000 7f69e7415640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556a73bd9e30 0x556a73bdc220 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.360+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.360+0000 7f69e7415640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69e002bc00 0x7f69e0022420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.360+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x7f69e002bc00 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.360+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69e002bc00 0x7f69e0022420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.360+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69e002bc00 0x7f69e0022420 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.361+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.361+0000 7f69e6c14640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69d8059040 0x7f69d804d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.361+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x7f69d8059040 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.361+0000 7f69e6413640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69d8059040 0x7f69d804d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.361+0000 7f69e6413640 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69d8059040 0x7f69d804d850 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd3c20 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd3c20 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x7f69d8059040 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.362+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x7f69d8059040 2024-04-23T19:10:49.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x7f69e002bc00 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd9e30 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd3c20 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.363+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x7f69d8059040 2024-04-23T19:10:49.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd3c20 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x7f69e002bc00 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd9e30 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.364+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdc7b0 con 0x556a73bd9e30 2024-04-23T19:10:49.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.365+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.365+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69d8059040 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.365+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.365+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd9e30 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.365+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.365+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd9e30 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.366+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.366+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd3c20 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.366+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.366+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd3c20 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.366+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.366+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69d8059040 2024-04-23T19:10:49.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd9e30 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69e002bc00 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd3c20 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.367+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69e002bc00 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.368+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.368+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd9e30 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.368+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.368+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69d8059040 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.368+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.368+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69e002bc00 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd9e30 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69d8059040 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x556a73bd3c20 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.369+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f69e0062040 con 0x7f69e002bc00 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bd8a70 con 0x556a73bd3c20 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bd9310 con 0x556a73bd3c20 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdcc30 con 0x7f69d8059040 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.370+0000 7f69eb613800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 zone default found 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 4 Realm: () 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 zonegroup default 2024-04-23T19:10:49.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdd0e0 con 0x7f69e002bc00 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdee60 con 0x556a73bd9e30 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdf2e0 con 0x556a73bd9e30 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 started sync module instance, tier type = 2024-04-23T19:10:49.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.371+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73bde300 con 0x7f69e002bc00 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.372+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.372+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93915696719600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73bdfbf0 con 0x7f69e002bc00 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.373+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be0440 con 0x556a73bd3c20 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.373+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93915696719600] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.374+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.374+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93915696725904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be0860 con 0x556a73bd3c20 2024-04-23T19:10:49.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.374+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be10b0 con 0x556a73bd9e30 2024-04-23T19:10:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.375+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.375+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93915696728640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be1540 con 0x556a73bd9e30 2024-04-23T19:10:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.375+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be2060 con 0x7f69e002bc00 2024-04-23T19:10:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.375+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93915696725904] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.376+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93915696728640] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.376+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.376+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93915696732976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be0f60 con 0x7f69e002bc00 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.376+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be2a70 con 0x556a73bd3c20 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.377+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.377+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93915696735680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be2e90 con 0x556a73bd3c20 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.377+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be3940 con 0x7f69d8059040 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.377+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93915696732976] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.378+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93915696735680] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.378+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.378+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93915696726960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be29e0 con 0x7f69d8059040 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.378+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be41e0 con 0x7f69d8059040 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.379+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(56 notify.6 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.379+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93915696741520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be4560 con 0x7f69d8059040 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.379+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be4db0 con 0x7f69d8059040 2024-04-23T19:10:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.379+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93915696726960] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93915696741520] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93915696742576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be4980 con 0x7f69d8059040 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 20 add_watcher() i=0 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 20 add_watcher() i=1 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 20 add_watcher() i=2 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 20 add_watcher() i=3 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 20 add_watcher() i=4 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 20 add_watcher() i=5 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.380+0000 7f69eb613800 20 add_watcher() i=6 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.381+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93915696742576] v32'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.381+0000 7f69eb613800 20 add_watcher() i=7 2024-04-23T19:10:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.381+0000 7f69eb613800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.381+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73be7b30 con 0x556a73bd3c20 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.381+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.381+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93915696756080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73be4980 con 0x556a73bd3c20 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.382+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93915696756080] v32'180 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f689d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 5 note: GC not initialized 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f6817fff640 20 reqs_thread_entry: start 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 20 Filter name: none 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f6815ffb640 20 reqs_thread_entry: start 2024-04-23T19:10:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.383+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73c1b5c0 con 0x7f69e002bc00 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 : miss 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.8 8:17d7809a:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73c1d080 con 0x556a73bd3c20 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv11 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 info.flags=0x17 2024-04-23T19:10:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 to cache LRU end 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 20 get_obj_state: octx=0x7f69dc052c80 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x556a73c1f728 s->prefetch_data=0 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.384+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 3.6s0 3:63673d0d:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x556a73bd3c20 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.385+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv5 ondisk = 0) ==== 262+0+927 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.385+0000 7f69eb613800 10 manifest: total_size = 5 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.385+0000 7f69eb613800 20 get_obj_state: setting s->obj_tag to _lOjU1Q9PFfWXRed9M6t5z9VcOazeTO1 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.385+0000 7f689d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.385+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93915696756080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x556a73bd3c20 2024-04-23T19:10:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.386+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93915696756080] v32'181 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.386+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93915696719600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x7f69e002bc00 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.387+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93915696719600] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.387+0000 7f69eb613800 20 remove_watcher() i=0 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.387+0000 7f69eb613800 2 removed watcher, disabling cache 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.387+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93915696725904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x556a73bd3c20 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.389+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93915696725904] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.389+0000 7f69eb613800 20 remove_watcher() i=1 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.389+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93915696728640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x556a73bd9e30 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.390+0000 7f69e7415640 1 -- 172.21.15.63:0/1969191888 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93915696728640] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69e0029070 con 0x556a73bd9e30 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.390+0000 7f69eb613800 20 remove_watcher() i=2 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.390+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93915696732976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x7f69e002bc00 2024-04-23T19:10:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.391+0000 7f69e6c14640 1 -- 172.21.15.63:0/1969191888 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93915696732976] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69d802d070 con 0x7f69e002bc00 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.391+0000 7f69eb613800 20 remove_watcher() i=3 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.391+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93915696735680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x556a73bd3c20 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.392+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93915696735680] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x556a73bd3c20 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.392+0000 7f69eb613800 20 remove_watcher() i=4 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.392+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93915696726960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x7f69d8059040 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.393+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93915696726960] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.393+0000 7f69eb613800 20 remove_watcher() i=5 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.393+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93915696741520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x7f69d8059040 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.394+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93915696741520] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.394+0000 7f69eb613800 20 remove_watcher() i=6 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.394+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93915696742576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x556a73c20860 con 0x7f69d8059040 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93915696742576] v32'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f69dc05a040 con 0x7f69d8059040 2024-04-23T19:10:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 20 remove_watcher() i=7 2024-04-23T19:10:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69d8059040 msgr2=0x7f69d804d850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f69d8059040 0x7f69d804d850 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73bd3c20 msgr2=0x556a73bd60e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73bd3c20 0x556a73bd60e0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69e002bc00 msgr2=0x7f69e0022420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f69e002bc00 0x7f69e0022420 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556a73bd9e30 msgr2=0x556a73bdc220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x556a73bd9e30 0x556a73bdc220 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68b0036df0 msgr2=0x7f68b00392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68b0036df0 0x7f68b00392b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69d804d010 tx=0x7f69d8035000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 msgr2=0x556a73bc7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69e6413640 1 -- 172.21.15.63:0/1969191888 reap_dead start 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc7ec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f69e004cb80 tx=0x7f69e004cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 shutdown_connections 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 --2- 172.21.15.63:0/1969191888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bc0980 0x556a73bc7ec0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.395+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 >> 172.21.15.63:0/1969191888 conn(0x556a73bbcc90 msgr2=0x556a73bba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 shutdown_connections 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1969191888 wait complete. 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68c0036e60 msgr2=0x7f68c0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68c0036e60 0x7f68c0039320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x556a73ba5640 tx=0x7f69d80015f0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bb3610 msgr2=0x556a73bb3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bb3610 0x556a73bb3a30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x556a73b9fb00 tx=0x7f69e00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 shutdown_connections 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68c0036e60 0x7f68c0039320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/1356624332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73bb3610 0x556a73bb3a30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 >> 172.21.15.63:0/1356624332 conn(0x556a73ba76b0 msgr2=0x556a73ba7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 shutdown_connections 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/1356624332 wait complete. 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556a73ba28f0 msgr2=0x556a73ba4ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556a73ba28f0 0x556a73ba4ce0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73aa04a0 msgr2=0x556a73b9f1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73aa04a0 0x556a73b9f1c0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68cc036eb0 msgr2=0x7f68cc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68cc036eb0 0x7f68cc039370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f69dc054000 tx=0x7f69dc04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 msgr2=0x556a73a10100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.396+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73a10100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000f60 tx=0x7f69d8000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 shutdown_connections 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x556a73ba28f0 0x556a73ba4ce0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x556a73aa04a0 0x556a73b9f1c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f68cc036eb0 0x7f68cc039370 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 --2- 172.21.15.63:0/71109412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a73a9c010 0x556a73a10100 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 >> 172.21.15.63:0/71109412 conn(0x556a73a97630 msgr2=0x556a73b871a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 shutdown_connections 2024-04-23T19:10:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.397+0000 7f69eb613800 1 -- 172.21.15.63:0/71109412 wait complete. 2024-04-23T19:10:49.420 INFO:teuthology.orchestra.run.smithi063.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-23T19:10:49.420 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-23T19:10:49.423 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-04-23T19:10:49.435 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:49.531 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-23T19:10:49.531 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-23T19:10:49.531 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-23T19:10:49.531 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-23T19:10:49.531 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-23T19:10:49.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-23T19:10:49.532 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-04-23T19:10:49.577 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:49.577 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:49.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.576+0000 7f35e932c800 1 Processor -- start 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.576+0000 7f35e932c800 1 -- start start 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.576+0000 7f35e932c800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e051222580 0x55e05122b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.576+0000 7f35e932c800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e0510d47f0 con 0x55e051220a40 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.576+0000 7f35e932c800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55e0510c3e20 con 0x55e051222580 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.576+0000 7f35e4863640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e051222580 0x55e05122b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.577+0000 7f35e5064640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55e051220a40 0x55e051220e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49700/0 (socket says 172.21.15.63:49700) 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.577+0000 7f35e5064640 1 -- 172.21.15.63:0/2542123082 learned_addr learned my addr 172.21.15.63:0/2542123082 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.577+0000 7f35e4863640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e051222580 0x55e05122b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35776/0 (socket says 172.21.15.63:35776) 2024-04-23T19:10:49.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.577+0000 7f35e4863640 1 -- 172.21.15.63:0/2542123082 >> v1:172.21.15.16:6789/0 conn(0x55e051220a40 legacy=0x55e051220e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.577+0000 7f35e4863640 1 -- 172.21.15.63:0/2542123082 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0510e2b50 con 0x55e051222580 2024-04-23T19:10:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e4863640 1 --2- 172.21.15.63:0/2542123082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051222580 0x55e05122b080 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00a470 tx=0x7f35dc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa6a09e185e03e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35d7fff640 1 -- 172.21.15.63:0/2542123082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35dc057070 con 0x55e051222580 2024-04-23T19:10:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35d7fff640 1 -- 172.21.15.63:0/2542123082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35dc052440 con 0x55e051222580 2024-04-23T19:10:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35d7fff640 1 -- 172.21.15.63:0/2542123082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35dc052740 con 0x55e051222580 2024-04-23T19:10:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e932c800 1 -- 172.21.15.63:0/2542123082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051222580 msgr2=0x55e05122b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e932c800 1 --2- 172.21.15.63:0/2542123082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051222580 0x55e05122b080 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00a470 tx=0x7f35dc01ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e932c800 1 -- 172.21.15.63:0/2542123082 shutdown_connections 2024-04-23T19:10:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e932c800 1 --2- 172.21.15.63:0/2542123082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051222580 0x55e05122b080 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e932c800 1 -- 172.21.15.63:0/2542123082 >> 172.21.15.63:0/2542123082 conn(0x55e05121c600 msgr2=0x55e05121ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e932c800 1 -- 172.21.15.63:0/2542123082 shutdown_connections 2024-04-23T19:10:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.579+0000 7f35e932c800 1 -- 172.21.15.63:0/2542123082 wait complete. 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.592+0000 7f35e932c800 1 Processor -- start 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.592+0000 7f35e932c800 1 -- start start 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.592+0000 7f35e932c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e051315790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.592+0000 7f35e932c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e05116d6f0 con 0x55e051220a40 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f35e5064640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e051315790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f35e5064640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e051315790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35780/0 (socket says 172.21.15.63:35780) 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f35e5064640 1 -- 172.21.15.63:0/2458447057 learned_addr learned my addr 172.21.15.63:0/2458447057 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f35e5064640 1 -- 172.21.15.63:0/2458447057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e051315d60 con 0x55e051220a40 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f35e5064640 1 --2- 172.21.15.63:0/2458447057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e051315790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f35d8001de0 tx=0x7f35d800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a8c27bf2517c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f34d3606640 1 -- 172.21.15.63:0/2458447057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d8012020 con 0x55e051220a40 2024-04-23T19:10:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f34d3606640 1 -- 172.21.15.63:0/2458447057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d800e5a0 con 0x55e051220a40 2024-04-23T19:10:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.593+0000 7f34d3606640 1 -- 172.21.15.63:0/2458447057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d800e8c0 con 0x55e051220a40 2024-04-23T19:10:49.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.594+0000 7f35e932c800 1 -- 172.21.15.63:0/2458447057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 msgr2=0x55e051315790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.594+0000 7f35e932c800 1 --2- 172.21.15.63:0/2458447057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e051315790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f35d8001de0 tx=0x7f35d800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.594+0000 7f35e932c800 1 -- 172.21.15.63:0/2458447057 shutdown_connections 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.594+0000 7f35e932c800 1 --2- 172.21.15.63:0/2458447057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e051315790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.594+0000 7f35e932c800 1 -- 172.21.15.63:0/2458447057 >> 172.21.15.63:0/2458447057 conn(0x55e05121c600 msgr2=0x55e0513110b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.594+0000 7f35e932c800 1 -- 172.21.15.63:0/2458447057 shutdown_connections 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.594+0000 7f35e932c800 1 -- 172.21.15.63:0/2458447057 wait complete. 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e932c800 1 Processor -- start 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e932c800 1 -- start start 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e932c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e05131d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e932c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e05122d8a0 con 0x55e051220a40 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e5064640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e05131d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e5064640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e05131d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35792/0 (socket says 172.21.15.63:35792) 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e5064640 1 -- 172.21.15.63:0/102785006 learned_addr learned my addr 172.21.15.63:0/102785006 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e5064640 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05131d840 con 0x55e051220a40 2024-04-23T19:10:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.595+0000 7f35e5064640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e05131d300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c010 tx=0x7f35d8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.596+0000 7f34d1e03640 1 -- 172.21.15.63:0/102785006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d8012070 con 0x55e051220a40 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.596+0000 7f34d1e03640 1 -- 172.21.15.63:0/102785006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d801a7a0 con 0x55e051220a40 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.596+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e05131dad0 con 0x55e051220a40 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.596+0000 7f34d1e03640 1 -- 172.21.15.63:0/102785006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35d801aa80 con 0x55e051220a40 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.596+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e051320780 con 0x55e051220a40 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.596+0000 7f34d1e03640 1 -- 172.21.15.63:0/102785006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f35d8015020 con 0x55e051220a40 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f34d1e03640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34cc036eb0 0x7f34cc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f34d1e03640 1 -- 172.21.15.63:0/102785006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f35d8020070 con 0x55e051220a40 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f35e4863640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34cc036eb0 0x7f34cc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051222580 0x55e051324540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051324a80 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f35e5865640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051222580 0x55e051324540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f35e4863640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34cc036eb0 0x7f34cc039370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f35dc054000 tx=0x7f35dc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.597+0000 7f35e5865640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051222580 0x55e051324540 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.598+0000 7f35e5865640 1 -- 172.21.15.63:0/102785006 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f35e000a4a0 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.598+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051321b10 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.598+0000 7f35e5865640 1 -- 172.21.15.63:0/102785006 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f35e000a4a0 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.598+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513268c0 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.598+0000 7f35e5865640 1 -- 172.21.15.63:0/102785006 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f35e000a4a0 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.598+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051325c00 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.599+0000 7f35e5865640 1 -- 172.21.15.63:0/102785006 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f35e000a4a0 con 0x55e051222580 2024-04-23T19:10:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.599+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e051327c70 0x55e05132a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.599+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05132a620 con 0x55e051327c70 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.599+0000 7f35e5064640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e051327c70 0x55e05132a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.599+0000 7f35e5064640 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e051327c70 0x55e05132a060 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.599+0000 7f35e5064640 1 -- 172.21.15.63:0/102785006 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f35d802b070 con 0x55e051327c70 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e932c800 1 Processor -- start 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e932c800 1 -- start start 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e932c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051338a70 0x55e051338e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e932c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e051113e30 con 0x55e051338a70 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e5865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051338a70 0x55e051338e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e5865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051338a70 0x55e051338e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35794/0 (socket says 172.21.15.63:35794) 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e5865640 1 -- 172.21.15.63:0/3215663089 learned_addr learned my addr 172.21.15.63:0/3215663089 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.600+0000 7f35e5865640 1 -- 172.21.15.63:0/3215663089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0513393d0 con 0x55e051338a70 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.601+0000 7f35e5865640 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051338a70 0x55e051338e90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55e05131e700 tx=0x7f35e00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.601+0000 7f34caffd640 1 -- 172.21.15.63:0/3215663089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e0013440 con 0x55e051338a70 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.601+0000 7f34caffd640 1 -- 172.21.15.63:0/3215663089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e00135e0 con 0x55e051338a70 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.601+0000 7f34caffd640 1 -- 172.21.15.63:0/3215663089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e00138c0 con 0x55e051338a70 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.601+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e051339e40 con 0x55e051338a70 2024-04-23T19:10:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.601+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05133a340 con 0x55e051338a70 2024-04-23T19:10:49.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f34caffd640 1 -- 172.21.15.63:0/3215663089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f35e0011040 con 0x55e051338a70 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f34caffd640 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34bc036e60 0x7f34bc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f34caffd640 1 -- 172.21.15.63:0/3215663089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f35e001d070 con 0x55e051338a70 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f35e5064640 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34bc036e60 0x7f34bc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f35e932c800 1 Processor -- start 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f35e5064640 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34bc036e60 0x7f34bc039320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35d804b0b0 tx=0x7f35d804af80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f35e932c800 1 -- start start 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f35e932c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e0513461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f35e932c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e0513160e0 con 0x55e051345db0 2024-04-23T19:10:49.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.602+0000 7f35e5865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e0513461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e5865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e0513461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35806/0 (socket says 172.21.15.63:35806) 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e5865640 1 -- 172.21.15.63:0/1982793484 learned_addr learned my addr 172.21.15.63:0/1982793484 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e5865640 1 -- 172.21.15.63:0/1982793484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0513467a0 con 0x55e051345db0 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e5865640 1 --2- 172.21.15.63:0/1982793484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e0513461d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f35e004cca0 tx=0x7f35e0012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e870fb0704e7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f34c8ff9640 1 -- 172.21.15.63:0/1982793484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e0042030 con 0x55e051345db0 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f34c8ff9640 1 -- 172.21.15.63:0/1982793484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0001310 con 0x55e051345db0 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f34c8ff9640 1 -- 172.21.15.63:0/1982793484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e0027040 con 0x55e051345db0 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e932c800 1 -- 172.21.15.63:0/1982793484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 msgr2=0x55e0513461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e932c800 1 --2- 172.21.15.63:0/1982793484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e0513461d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f35e004cca0 tx=0x7f35e0012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e932c800 1 -- 172.21.15.63:0/1982793484 shutdown_connections 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e932c800 1 --2- 172.21.15.63:0/1982793484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e0513461d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.603+0000 7f35e932c800 1 -- 172.21.15.63:0/1982793484 >> 172.21.15.63:0/1982793484 conn(0x55e051342080 msgr2=0x55e0513400e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e932c800 1 -- 172.21.15.63:0/1982793484 shutdown_connections 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e932c800 1 -- 172.21.15.63:0/1982793484 wait complete. 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e932c800 1 Processor -- start 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e932c800 1 -- start start 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e932c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e05134d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e932c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e051346f30 con 0x55e051345db0 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e5865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e05134d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e5865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e05134d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35822/0 (socket says 172.21.15.63:35822) 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 learned_addr learned my addr 172.21.15.63:0/74722147 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05134d800 con 0x55e051345db0 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.604+0000 7f35e5865640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e05134d2c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f35e0001df0 tx=0x7f35e004cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.605+0000 7f34baffd640 1 -- 172.21.15.63:0/74722147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e0027030 con 0x55e051345db0 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.605+0000 7f34baffd640 1 -- 172.21.15.63:0/74722147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e002a350 con 0x55e051345db0 2024-04-23T19:10:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.605+0000 7f34baffd640 1 -- 172.21.15.63:0/74722147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e002a650 con 0x55e051345db0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.605+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e05134da90 con 0x55e051345db0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.605+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e051350740 con 0x55e051345db0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f34baffd640 1 -- 172.21.15.63:0/74722147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f35e0045070 con 0x55e051345db0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f34baffd640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34b0036eb0 0x7f34b0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f34baffd640 1 -- 172.21.15.63:0/74722147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f35e0043070 con 0x55e051345db0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f35e5064640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34b0036eb0 0x7f34b0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051358fa0 0x55e05135b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05135bbb0 con 0x55e051358fa0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f35e5064640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34b0036eb0 0x7f34b0039370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55e05132aa00 tx=0x7f35d8048000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.606+0000 7f35e4863640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051358fa0 0x55e05135b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e4863640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051358fa0 0x55e05135b460 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 20 realm 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05135bf80 con 0x55e051358fa0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05135c850 con 0x55e051358fa0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.607+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05135cbf0 con 0x55e051358fa0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e932c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05135d440 con 0x55e051358fa0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e932c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e932c800 20 searching for the correct realm 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e05135f1b0 0x55e0513615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e5865640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e05135f1b0 0x55e0513615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.608+0000 7f35e5865640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e05135f1b0 0x55e0513615a0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.609+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.609+0000 7f35e5865640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f35e00123a0 0x7f35e0046790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.609+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35e00123a0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.609+0000 7f35e5064640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f35e00123a0 0x7f35e0046790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.609+0000 7f35e5064640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f35e00123a0 0x7f35e0046790 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.610+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.610+0000 7f35e5064640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f35d8017050 0x7f35d8019000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.610+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.610+0000 7f35e4863640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f35d8017050 0x7f35d8019000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.610+0000 7f35e4863640 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f35d8017050 0x7f35d8019000 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.611+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35e00123a0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.612+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35e00123a0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.613+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.614+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.615+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.615+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.615+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.615+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.616+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.616+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.616+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.616+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35e00123a0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.616+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.616+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35e00123a0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e0029070 con 0x55e05135f1b0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.617+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35e00123a0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e05135f1b0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35d8017050 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x55e051358fa0 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.618+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e051361b30 con 0x7f35e00123a0 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05135ddf0 con 0x55e051358fa0 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05135e690 con 0x55e051358fa0 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.619+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051361fb0 con 0x7f35d8017050 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 20 zone default found 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 4 Realm: () 2024-04-23T19:10:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 20 zonegroup default 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051362460 con 0x7f35e00123a0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513641e0 con 0x55e05135f1b0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.620+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513646f0 con 0x55e05135f1b0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e932c800 20 started sync module instance, tier type = 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e932c800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.621+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051363680 con 0x7f35e00123a0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.622+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.622+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94421923543664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513632e0 con 0x7f35e00123a0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.622+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051365620 con 0x55e051358fa0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.623+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94421923543664] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.623+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.623+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94421923549552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051365a40 con 0x55e051358fa0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.623+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051366400 con 0x55e05135f1b0 2024-04-23T19:10:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.625+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94421923549552] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.625+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.625+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94421923550608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513659b0 con 0x55e05135f1b0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.625+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051367270 con 0x7f35e00123a0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.626+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94421923550608] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.626+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.626+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94421923556800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051367660 con 0x7f35e00123a0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.626+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051368060 con 0x55e051358fa0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.627+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94421923556800] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.627+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.627+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94421923555232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051368020 con 0x55e051358fa0 2024-04-23T19:10:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.627+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051368ac0 con 0x7f35d8017050 2024-04-23T19:10:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.628+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94421923555232] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.628+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.628+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94421923562944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051368020 con 0x7f35d8017050 2024-04-23T19:10:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.628+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051369510 con 0x7f35d8017050 2024-04-23T19:10:49.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.629+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94421923562944] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.629+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.629+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94421923565600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051368020 con 0x7f35d8017050 2024-04-23T19:10:49.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.629+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051369f20 con 0x7f35d8017050 2024-04-23T19:10:49.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94421923565600] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94421923560784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051369f20 con 0x7f35d8017050 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 20 add_watcher() i=0 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 20 add_watcher() i=1 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 20 add_watcher() i=2 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 20 add_watcher() i=3 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 20 add_watcher() i=4 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 20 add_watcher() i=5 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.630+0000 7f35e932c800 20 add_watcher() i=6 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.631+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94421923560784] v32'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.631+0000 7f35e932c800 20 add_watcher() i=7 2024-04-23T19:10:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.631+0000 7f35e932c800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.631+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05136d2d0 con 0x55e051358fa0 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.631+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.632+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94421923581056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05136a090 con 0x55e051358fa0 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.632+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94421923581056] v32'182 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.633+0000 7f349b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.633+0000 7f35e932c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.633+0000 7f35e932c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.633+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051375bf0 con 0x55e051358fa0 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.634+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'31 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.634+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513762c0 con 0x7f35d8017050 2024-04-23T19:10:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.635+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.635+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513766e0 con 0x55e05135f1b0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.636+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.636+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051376680 con 0x55e051358fa0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.637+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'183 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.637+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051376c90 con 0x7f35e00123a0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.638+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'62 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.638+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377200 con 0x55e05135f1b0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.639+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'41 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.639+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377200 con 0x55e051358fa0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.640+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.640+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377200 con 0x7f35d8017050 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.642+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'41 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.642+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377200 con 0x7f35e00123a0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.642+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.643+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513770d0 con 0x55e05135f1b0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.643+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.644+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513770d0 con 0x7f35e00123a0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.644+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'63 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.644+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377070 con 0x7f35d8017050 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.645+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.646+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377070 con 0x7f35e00123a0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.647+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.647+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051376fe0 con 0x7f35d8017050 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.648+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'47 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.648+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051376fe0 con 0x55e051358fa0 2024-04-23T19:10:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.649+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'184 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.649+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.650+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'57 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.650+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x7f35e00123a0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.651+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'64 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.651+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.652+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.652+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e051358fa0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.653+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'32 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.653+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e051358fa0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.654+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'185 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.654+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.655+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'58 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.655+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x7f35e00123a0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.656+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'34 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.656+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.657+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.657+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.658+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.658+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.659+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'20 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.659+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.660+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'42 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.660+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x7f35d8017050 2024-04-23T19:10:49.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.661+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.661+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.662+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.662+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x7f35d8017050 2024-04-23T19:10:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.664+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'42 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.664+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x55e05135f1b0 2024-04-23T19:10:49.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.665+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'59 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.665+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x7f35d8017050 2024-04-23T19:10:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.666+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'48 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.666+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051377020 con 0x7f35e00123a0 2024-04-23T19:10:49.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.667+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.667+0000 7f349a7e4640 20 reqs_thread_entry: start 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.667+0000 7f35e932c800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.667+0000 7f35e932c800 20 Filter name: none 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f340bfff640 20 reqs_thread_entry: start 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a25a0 con 0x7f35e00123a0 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 : miss 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 8.8 8:17d7809a:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a4060 con 0x55e051358fa0 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(95 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv11 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 info.flags=0x17 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 to cache LRU end 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.668+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 4.35 4:aeba30d1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x55e051358fa0 2024-04-23T19:10:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.669+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(96 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.669+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 4.28 4:168cd691:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x55e051358fa0 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.669+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(97 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.669+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:98 4.1c 4:3a466642:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x7f35d8017050 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.669+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(98 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2 [call out=426b] v0'0 uv3 ondisk = 0) ==== 194+0+426 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.669+0000 7f35e932c800 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.670+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 4.2c 4:376f84e1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x55e051358fa0 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.670+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(99 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.670+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:100 4.3d 4:bf9f1895:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x55e05135f1b0 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.670+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(100 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.670+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 4.9 4:92d14093:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x55e051358fa0 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.670+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(101 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.670+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 4.37 4:efd41996:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x7f35d8017050 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.671+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(102 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6 [call out=425b] v0'0 uv3 ondisk = 0) ==== 194+0+425 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.671+0000 7f35e932c800 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.671+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 4.2f 4:f65464c1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x55e051358fa0 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.671+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(103 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.671+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:104 4.3e 4:7efb3cf9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x7f35e00123a0 2024-04-23T19:10:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(104 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8 [call out=425b] v0'0 uv13 ondisk = 0) ==== 194+0+425 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e932c800 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 4.f 4:f0cd61bc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x55e051358fa0 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(105 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:106 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421890 con 0x7f35d8017050 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(106 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [call out=698b] v0'0 uv9 ondisk = 0) ==== 195+0+698 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e932c800 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e932c800 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.672+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 3.20s0 3:06b275e3:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05141f590 con 0x7f35e00123a0 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.673+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(107 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.673+0000 7f35e932c800 10 manifest: total_size = 6 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.673+0000 7f35e932c800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.13138391416895305264 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.673+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.673+0000 7f35e932c800 20 prepare_atomic_modification: state is not atomic. state=0x55e051421098 2024-04-23T19:10:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.673+0000 7f35e932c800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2 2024-04-23T19:10:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.673+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:108 4.1c 4:3a466642:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9640 con 0x7f35d8017050 2024-04-23T19:10:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.674+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(108 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2 [stat,call,call] v32'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.674+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:109 3.20s0 3:06b275e3:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a96a0 con 0x7f35e00123a0 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.675+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(109 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_9c8454ddf7aa50116496bac348d7550d [call] v32'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.675+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:110 4.1c 4:3a466642:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f70 con 0x7f35d8017050 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.675+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.675+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:111 3.9s0 3:91741bc2:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f70 con 0x55e05135f1b0 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(111 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+893 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e932c800 10 manifest: total_size = 3 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e932c800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5717904624321691417 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e932c800 20 prepare_atomic_modification: state is not atomic. state=0x55e051421098 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e932c800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:112 4.37 4:efd41996:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f70 con 0x7f35d8017050 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(110 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2 [stat,call,call] v32'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.676+0000 7f34c8ff9640 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.677+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(112 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6 [stat,call,call] v32'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.677+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:113 3.9s0 3:91741bc2:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f70 con 0x55e05135f1b0 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.678+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(113 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_b1b9a972ccd8c962a473909b97007eb4 [call] v32'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.678+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:114 4.37 4:efd41996:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f40 con 0x7f35d8017050 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.678+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.678+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:115 3.35s0 3:af1e2ee4:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f40 con 0x55e051358fa0 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(115 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+893 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e932c800 10 manifest: total_size = 4 2024-04-23T19:10:49.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e932c800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.1653708970684476328 2024-04-23T19:10:49.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e932c800 20 prepare_atomic_modification: state is not atomic. state=0x55e051421098 2024-04-23T19:10:49.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e932c800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8 2024-04-23T19:10:49.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:116 4.3e 4:7efb3cf9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f40 con 0x7f35e00123a0 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(114 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6 [stat,call,call] v32'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.679+0000 7f34bbfff640 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.680+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(116 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8 [stat,call,call] v32'14 uv14 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.680+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:117 3.35s0 3:af1e2ee4:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f20 con 0x55e051358fa0 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.681+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(117 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v32'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.681+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:118 4.3e 4:7efb3cf9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f20 con 0x7f35e00123a0 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.681+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.681+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:119 3.29s0 3:94d94f71:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a9f20 con 0x7f35d8017050 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.681+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(119 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f35e932c800 10 manifest: total_size = 5 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f35e932c800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.15153335742391952907 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f35e932c800 20 prepare_atomic_modification: state is not atomic. state=0x55e051421098 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f35e932c800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:120 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513afd00 con 0x7f35d8017050 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(118 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8 [stat,call,call] v32'15 uv15 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.682+0000 7f34c8ff9640 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.683+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(120 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [stat,call,call] v32'10 uv10 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.683+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:121 3.29s0 3:94d94f71:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513aecf0 con 0x7f35d8017050 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== osd_op_reply(121 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_24d27c169c2c881eb09a065116f2aa5c [call] v32'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:122 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a5ef0 con 0x7f35d8017050 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:123 3.6s0 3:63673d0d:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a6a80 con 0x55e051358fa0 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(123 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv5 ondisk = 0) ==== 262+0+927 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 10 manifest: total_size = 5 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 20 get_obj_state: setting s->obj_tag to _lOjU1Q9PFfWXRed9M6t5z9VcOazeTO1 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 20 get_obj_state: octx=0x7f35dc052c20 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55e051421098 s->prefetch_data=0 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 20 prepare_atomic_modification: state is not atomic. state=0x55e051421098 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.684+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:124 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513adc40 con 0x7f35d8017050 2024-04-23T19:10:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.685+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(122 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [stat,call,call] v32'11 uv11 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.685+0000 7f34bbfff640 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-04-23T19:10:49.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.686+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== osd_op_reply(124 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [stat,call,call] v32'12 uv12 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.686+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:125 3.6s0 3:63673d0d:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a6a80 con 0x55e051358fa0 2024-04-23T19:10:49.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(125 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5_bb3aec0fdcdbc2974890f805c585d432 [call] v32'6 uv6 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:126 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a6a80 con 0x7f35d8017050 2024-04-23T19:10:49.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 20 list_objects_ordered: starting attempt 1 2024-04-23T19:10:49.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-23T19:10:49.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:127 4.35 4:aeba30d1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a6a80 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:128 4.28 4:168cd691:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051420eb0 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:129 4.1c 4:3a466642:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513b2d30 con 0x7f35d8017050 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:130 4.2c 4:376f84e1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513b5640 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:131 4.3d 4:bf9f1895:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513b81a0 con 0x55e05135f1b0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:132 4.9 4:92d14093:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513baea0 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:133 4.37 4:efd41996:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x7f35d8017050 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:134 4.2f 4:f65464c1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0aa0 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:135 4.3e 4:7efb3cf9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c38d0 con 0x7f35e00123a0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:136 4.f 4:f0cd61bc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513b5680 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:137 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c8270 con 0x7f35d8017050 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.687+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(131 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(127 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== osd_op_reply(128 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(130 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(135 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8 [call out=124b] v0'0 uv15 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(129 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(133 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(132 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== osd_op_reply(134 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(136 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(126 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [stat,call,call] v32'13 uv13 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.688+0000 7f34c8ff9640 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(137 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [call out=124b] v0'0 uv13 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-23T19:10:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5])"; uploads.size()=0; is_truncated=0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:138 4.35 4:aeba30d1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0aa0 con 0x55e051358fa0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:139 4.28 4:168cd691:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e051358fa0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:140 4.1c 4:3a466642:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0aa0 con 0x7f35d8017050 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:141 4.2c 4:376f84e1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e051358fa0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:142 4.3d 4:bf9f1895:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0aa0 con 0x55e05135f1b0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:143 4.9 4:92d14093:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e051358fa0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:144 4.37 4:efd41996:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0aa0 con 0x7f35d8017050 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(138 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(142 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(139 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:145 4.2f 4:f65464c1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e051358fa0 2024-04-23T19:10:49.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:146 4.3e 4:7efb3cf9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0aa0 con 0x7f35e00123a0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(141 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(143 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:147 4.f 4:f0cd61bc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c32e0 con 0x55e051358fa0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:148 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0aa0 con 0x7f35d8017050 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(140 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.689+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(144 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.690+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(146 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8 [call out=124b] v0'0 uv15 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.690+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 49 ==== osd_op_reply(147 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.690+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 50 ==== osd_op_reply(145 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.690+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== osd_op_reply(148 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [call out=124b] v0'0 uv13 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.690+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:149 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x7f35e00123a0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.691+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(149 foo.buckets [call] v32'37 uv37 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.691+0000 7f35e932c800 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x11) 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.691+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:150 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x7f35e00123a0 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(150 myfoo [call,call,delete] v32'38 uv38 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e932c800 10 removing default.rgw.meta+root+myfoo from cache 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e932c800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:151 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94421923816560 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x7f35d8017050 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 42 ==== watch-notify(notify (1) cookie 94421923565600 notify 137438953495 ret 0) ==== 191+0+0 (crc 0 0 0) 0x55e0513a2070 con 0x7f35d8017050 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f34bbfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953495 cookie 94421923565600 notifier 4646 bl.length()=149 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f34bbfff640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:152 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f34180032c0 con 0x7f35d8017050 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 43 ==== osd_op_reply(151 notify.6 [notify cookie 94421923816560 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 44 ==== osd_op_reply(152 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.692+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 45 ==== watch-notify(notify_complete (2) cookie 94421923816560 notify 137438953495 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e051350740 con 0x7f35d8017050 2024-04-23T19:10:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.693+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:153 8.8 8:17d7809a:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e051358fa0 2024-04-23T19:10:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 51 ==== osd_op_reply(153 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 [call,call,delete] v32'12 uv12 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 from cache 2024-04-23T19:10:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5, ofs0, ns] 2024-04-23T19:10:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:154 7.3 7:c609908c:::notify.5:head [notify cookie 94421923816560 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x7f35d8017050 2024-04-23T19:10:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 46 ==== watch-notify(notify (1) cookie 94421923562944 notify 137438953496 ret 0) ==== 248+0+0 (crc 0 0 0) 0x55e051350740 con 0x7f35d8017050 2024-04-23T19:10:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f34c8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953496 cookie 94421923562944 notifier 4646 bl.length()=206 2024-04-23T19:10:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 47 ==== osd_op_reply(154 notify.5 [notify cookie 94421923816560 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f34c8ff9640 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:155 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f34a8003d60 con 0x7f35d8017050 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 48 ==== osd_op_reply(155 notify.5 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 49 ==== watch-notify(notify_complete (2) cookie 94421923816560 notify 137438953496 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e051375d80 con 0x7f35d8017050 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:156 4.35 4:aeba30d1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e051358fa0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:157 4.28 4:168cd691:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a4700 con 0x55e051358fa0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:158 4.1c 4:3a466642:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051421300 con 0x7f35d8017050 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:159 4.2c 4:376f84e1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0f70 con 0x55e051358fa0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:160 4.3d 4:bf9f1895:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e05135f1b0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:161 4.9 4:92d14093:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a4700 con 0x55e051358fa0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:162 4.37 4:efd41996:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a5390 con 0x7f35d8017050 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:163 4.2f 4:f65464c1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0f70 con 0x55e051358fa0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:164 4.3e 4:7efb3cf9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a4700 con 0x7f35e00123a0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.694+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:165 4.f 4:f0cd61bc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513c0f70 con 0x55e051358fa0 2024-04-23T19:10:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.695+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:166 4.23 4:c7173711:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513a5390 con 0x7f35d8017050 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.696+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 52 ==== osd_op_reply(157 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.1 [delete] v32'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.696+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 53 ==== osd_op_reply(156 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.0 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.696+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 54 ==== osd_op_reply(161 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.5 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.696+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 55 ==== osd_op_reply(159 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.3 [delete] v32'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.696+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 50 ==== osd_op_reply(158 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.2 [delete] v32'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.696+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 51 ==== osd_op_reply(162 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.6 [delete] v32'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 52 ==== osd_op_reply(166 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.10 [delete] v32'14 uv14 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 56 ==== osd_op_reply(163 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.7 [delete] v32'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(164 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.8 [delete] v32'16 uv16 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(160 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.4 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 57 ==== osd_op_reply(165 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5.9 [delete] v32'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e932c800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 : miss 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e932c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.697+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:167 6.1 6:84116265:::pubsub..bucket.myfoo%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e051421300 con 0x55e051358fa0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.698+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 58 ==== osd_op_reply(167 pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.698+0000 7f35e932c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.698+0000 7f35e932c800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 info.flags=0x0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.698+0000 7f35e932c800 10 adding default.rgw.log++pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 to cache LRU end 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.698+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:168 6.1 6:84116265:::pubsub..bucket.myfoo%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513bdc70 con 0x55e051358fa0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.700+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 59 ==== osd_op_reply(168 pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5 [delete] v32'15 uv14 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.700+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:169 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05141f590 con 0x7f35e00123a0 2024-04-23T19:10:49.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.702+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(169 foo.buckets [call] v32'39 uv39 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.702+0000 7f349b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.702+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:170 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94421923581056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05141f590 con 0x55e051358fa0 2024-04-23T19:10:49.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.703+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 60 ==== osd_op_reply(170 data_loggenerations_metadata [watch unwatch cookie 94421923581056] v32'186 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.703+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:171 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94421923543664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05136cc80 con 0x7f35e00123a0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.704+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== osd_op_reply(171 notify.0 [watch unwatch cookie 94421923543664] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.704+0000 7f35e932c800 20 remove_watcher() i=0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.704+0000 7f35e932c800 2 removed watcher, disabling cache 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.704+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:172 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94421923549552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05136cc80 con 0x55e051358fa0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.706+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 61 ==== osd_op_reply(172 notify.1 [watch unwatch cookie 94421923549552] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.706+0000 7f35e932c800 20 remove_watcher() i=1 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.706+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:173 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94421923550608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051365170 con 0x55e05135f1b0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.707+0000 7f35e5865640 1 -- 172.21.15.63:0/74722147 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(173 notify.2 [watch unwatch cookie 94421923550608] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35e006a040 con 0x55e05135f1b0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.707+0000 7f35e932c800 20 remove_watcher() i=2 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.707+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:174 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94421923556800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05136cc80 con 0x7f35e00123a0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.708+0000 7f35e5064640 1 -- 172.21.15.63:0/74722147 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== osd_op_reply(174 notify.3 [watch unwatch cookie 94421923556800] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35d802b070 con 0x7f35e00123a0 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.708+0000 7f35e932c800 20 remove_watcher() i=3 2024-04-23T19:10:49.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.708+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:175 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94421923555232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e051366dc0 con 0x55e051358fa0 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.709+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.1 v2:172.21.15.63:6800/3216879695 62 ==== osd_op_reply(175 notify.4 [watch unwatch cookie 94421923555232] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x55e051358fa0 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.709+0000 7f35e932c800 20 remove_watcher() i=4 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.709+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:176 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94421923562944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05136cc80 con 0x7f35d8017050 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.710+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 53 ==== osd_op_reply(176 notify.5 [watch unwatch cookie 94421923562944] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.710+0000 7f35e932c800 20 remove_watcher() i=5 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.710+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:177 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94421923565600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513667a0 con 0x7f35d8017050 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.711+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 54 ==== osd_op_reply(177 notify.6 [watch unwatch cookie 94421923565600] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.711+0000 7f35e932c800 20 remove_watcher() i=6 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.711+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:178 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94421923560784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0513685c0 con 0x7f35d8017050 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 <== osd.0 v2:172.21.15.16:6800/2262294208 55 ==== osd_op_reply(178 notify.7 [watch unwatch cookie 94421923560784] v32'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f35dc05a040 con 0x7f35d8017050 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 20 remove_watcher() i=7 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f35d8017050 msgr2=0x7f35d8019000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f35d8017050 0x7f35d8019000 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051358fa0 msgr2=0x55e05135b460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051358fa0 0x55e05135b460 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f35e00123a0 msgr2=0x7f35e0046790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f35e00123a0 0x7f35e0046790 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e05135f1b0 msgr2=0x55e0513615a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e05135f1b0 0x55e0513615a0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34b0036eb0 msgr2=0x7f34b0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34b0036eb0 0x7f34b0039370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55e05132aa00 tx=0x7f35d8048000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 msgr2=0x55e05134d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e4863640 1 -- 172.21.15.63:0/74722147 reap_dead start 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e05134d2c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f35e0001df0 tx=0x7f35e004cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 shutdown_connections 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 --2- 172.21.15.63:0/74722147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051345db0 0x55e05134d2c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.712+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 >> 172.21.15.63:0/74722147 conn(0x55e051342080 msgr2=0x55e05133f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 shutdown_connections 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/74722147 wait complete. 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34bc036e60 msgr2=0x7f34bc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34bc036e60 0x7f34bc039320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35d804b0b0 tx=0x7f35d804af80 comp rx=0 tx=0).stop 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051338a70 msgr2=0x55e051338e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051338a70 0x55e051338e90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55e05131e700 tx=0x7f35e00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 shutdown_connections 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34bc036e60 0x7f34bc039320 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/3215663089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051338a70 0x55e051338e90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 >> 172.21.15.63:0/3215663089 conn(0x55e05132cb10 msgr2=0x55e05132cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 shutdown_connections 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/3215663089 wait complete. 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e051327c70 msgr2=0x55e05132a060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e051327c70 0x55e05132a060 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051222580 msgr2=0x55e051324540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051222580 0x55e051324540 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34cc036eb0 msgr2=0x7f34cc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34cc036eb0 0x7f34cc039370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f35dc054000 tx=0x7f35dc04e000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 msgr2=0x55e05131d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.713+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e05131d300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c010 tx=0x7f35d8000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 shutdown_connections 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e051327c70 0x55e05132a060 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e051222580 0x55e051324540 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f34cc036eb0 0x7f34cc039370 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 --2- 172.21.15.63:0/102785006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e051220a40 0x55e05131d300 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 >> 172.21.15.63:0/102785006 conn(0x55e05121c600 msgr2=0x55e05130f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 shutdown_connections 2024-04-23T19:10:49.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.714+0000 7f35e932c800 1 -- 172.21.15.63:0/102785006 wait complete. 2024-04-23T19:10:49.723 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-23T19:10:49.723 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-23T19:10:49.723 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-04-23T19:10:49.770 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:49.770 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:49.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.769+0000 7f50bd610800 1 Processor -- start 2024-04-23T19:10:49.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.769+0000 7f50bd610800 1 -- start start 2024-04-23T19:10:49.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.769+0000 7f50bd610800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559a7e113ba0 0x559a7e115f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.769+0000 7f50bd610800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559a7dfc57f0 con 0x559a7e1164d0 2024-04-23T19:10:49.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.769+0000 7f50bd610800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x559a7dfb4e20 con 0x559a7e113ba0 2024-04-23T19:10:49.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.770+0000 7f50b8d5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559a7e113ba0 0x559a7e115f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.770+0000 7f50b855b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x559a7e1164d0 0x559a7e1188c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49708/0 (socket says 172.21.15.63:49708) 2024-04-23T19:10:49.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.770+0000 7f50b855b640 1 -- 172.21.15.63:0/3057495520 learned_addr learned my addr 172.21.15.63:0/3057495520 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:49.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.770+0000 7f50b8d5c640 1 -- 172.21.15.63:0/3057495520 >> v1:172.21.15.16:6789/0 conn(0x559a7e1164d0 legacy=0x559a7e1188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.770+0000 7f50b8d5c640 1 -- 172.21.15.63:0/3057495520 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7dfd3b50 con 0x559a7e113ba0 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/3057495520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e115f90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f50a800a470 tx=0x7f50a801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3fe6498951f363 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50b7d5a640 1 -- 172.21.15.63:0/3057495520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a8057070 con 0x559a7e113ba0 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50b7d5a640 1 -- 172.21.15.63:0/3057495520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a8052570 con 0x559a7e113ba0 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50b7d5a640 1 -- 172.21.15.63:0/3057495520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a8052870 con 0x559a7e113ba0 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50bd610800 1 -- 172.21.15.63:0/3057495520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 msgr2=0x559a7e115f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50bd610800 1 --2- 172.21.15.63:0/3057495520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e115f90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f50a800a470 tx=0x7f50a801ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50bd610800 1 -- 172.21.15.63:0/3057495520 shutdown_connections 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50bd610800 1 --2- 172.21.15.63:0/3057495520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e115f90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.772+0000 7f50bd610800 1 -- 172.21.15.63:0/3057495520 >> 172.21.15.63:0/3057495520 conn(0x559a7e10d630 msgr2=0x559a7e10fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.773+0000 7f50bd610800 1 -- 172.21.15.63:0/3057495520 shutdown_connections 2024-04-23T19:10:49.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.773+0000 7f50bd610800 1 -- 172.21.15.63:0/3057495520 wait complete. 2024-04-23T19:10:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.785+0000 7f50bd610800 1 Processor -- start 2024-04-23T19:10:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.785+0000 7f50bd610800 1 -- start start 2024-04-23T19:10:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.785+0000 7f50bd610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e1134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.785+0000 7f50bd610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a7e05e6f0 con 0x559a7e113ba0 2024-04-23T19:10:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f50b8d5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e1134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f50b8d5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e1134e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35834/0 (socket says 172.21.15.63:35834) 2024-04-23T19:10:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f50b8d5c640 1 -- 172.21.15.63:0/2647105797 learned_addr learned my addr 172.21.15.63:0/2647105797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f50b8d5c640 1 -- 172.21.15.63:0/2647105797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7e111aa0 con 0x559a7e113ba0 2024-04-23T19:10:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/2647105797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e1134e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f50a8055040 tx=0x7f50a804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9529b1d774e4e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f4fa6e0d640 1 -- 172.21.15.63:0/2647105797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a8057040 con 0x559a7e113ba0 2024-04-23T19:10:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f4fa6e0d640 1 -- 172.21.15.63:0/2647105797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a8052990 con 0x559a7e113ba0 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.786+0000 7f4fa6e0d640 1 -- 172.21.15.63:0/2647105797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a8052c70 con 0x559a7e113ba0 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 -- 172.21.15.63:0/2647105797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 msgr2=0x559a7e1134e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 --2- 172.21.15.63:0/2647105797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e1134e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f50a8055040 tx=0x7f50a804c000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 -- 172.21.15.63:0/2647105797 shutdown_connections 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 --2- 172.21.15.63:0/2647105797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e1134e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 -- 172.21.15.63:0/2647105797 >> 172.21.15.63:0/2647105797 conn(0x559a7e10d630 msgr2=0x559a7e1ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 -- 172.21.15.63:0/2647105797 shutdown_connections 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 -- 172.21.15.63:0/2647105797 wait complete. 2024-04-23T19:10:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 Processor -- start 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.787+0000 7f50bd610800 1 -- start start 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.788+0000 7f50bd610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e20e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.788+0000 7f50bd610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a7e11a280 con 0x559a7e113ba0 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.788+0000 7f50b8d5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e20e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.788+0000 7f50b8d5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e20e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35846/0 (socket says 172.21.15.63:35846) 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.788+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1297303320 learned_addr learned my addr 172.21.15.63:0/1297303320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.788+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7e20e5b0 con 0x559a7e113ba0 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.789+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e20e070 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f50a800a470 tx=0x7f50a8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.789+0000 7f4fa560a640 1 -- 172.21.15.63:0/1297303320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a8073070 con 0x559a7e113ba0 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.789+0000 7f4fa560a640 1 -- 172.21.15.63:0/1297303320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a805d650 con 0x559a7e113ba0 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.789+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a7e20e840 con 0x559a7e113ba0 2024-04-23T19:10:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.789+0000 7f4fa560a640 1 -- 172.21.15.63:0/1297303320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a8057070 con 0x559a7e113ba0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.789+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7e2114f0 con 0x559a7e113ba0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f4fa560a640 1 -- 172.21.15.63:0/1297303320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f50a8076070 con 0x559a7e113ba0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f4fa560a640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4fa0036eb0 0x7f4fa0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f4fa560a640 1 -- 172.21.15.63:0/1297303320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f50a808e6a0 con 0x559a7e113ba0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f50b855b640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4fa0036eb0 0x7f4fa0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e1164d0 0x559a7e2152b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e215830 con 0x559a7e1164d0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f50b955d640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e1164d0 0x559a7e2152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f50b855b640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4fa0036eb0 0x7f4fa0039370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000f60 tx=0x7f50a00012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.790+0000 7f50b955d640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e1164d0 0x559a7e2152b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.791+0000 7f50b955d640 1 -- 172.21.15.63:0/1297303320 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f50b000a4a0 con 0x559a7e1164d0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.791+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e212880 con 0x559a7e1164d0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.791+0000 7f50b955d640 1 -- 172.21.15.63:0/1297303320 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f50b000a4a0 con 0x559a7e1164d0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.791+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e217630 con 0x559a7e1164d0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.791+0000 7f50b955d640 1 -- 172.21.15.63:0/1297303320 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f50b000a4a0 con 0x559a7e1164d0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.791+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e216970 con 0x559a7e1164d0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.792+0000 7f50b955d640 1 -- 172.21.15.63:0/1297303320 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f50b000a4a0 con 0x559a7e1164d0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.792+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a7e2189e0 0x559a7e21add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.792+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e21b350 con 0x559a7e2189e0 2024-04-23T19:10:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.792+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a7e2189e0 0x559a7e21add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.792+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a7e2189e0 0x559a7e21add0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1297303320 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f50a8075030 con 0x559a7e2189e0 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50bd610800 1 Processor -- start 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50bd610800 1 -- start start 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50bd610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e229770 0x559a7e229b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50bd610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a7e004e30 con 0x559a7e229770 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50b955d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e229770 0x559a7e229b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50b955d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e229770 0x559a7e229b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35852/0 (socket says 172.21.15.63:35852) 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.793+0000 7f50b955d640 1 -- 172.21.15.63:0/3349332237 learned_addr learned my addr 172.21.15.63:0/3349332237 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.794+0000 7f50b955d640 1 -- 172.21.15.63:0/3349332237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7e22a0d0 con 0x559a7e229770 2024-04-23T19:10:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.794+0000 7f50b955d640 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e229770 0x559a7e229b90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559a7e20f470 tx=0x7f50b00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.794+0000 7f4f9e7fc640 1 -- 172.21.15.63:0/3349332237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b0013440 con 0x559a7e229770 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.794+0000 7f4f9e7fc640 1 -- 172.21.15.63:0/3349332237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b00135e0 con 0x559a7e229770 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.794+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a7e22ab40 con 0x559a7e229770 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.794+0000 7f4f9e7fc640 1 -- 172.21.15.63:0/3349332237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b00138c0 con 0x559a7e229770 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.794+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7e22b040 con 0x559a7e229770 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f4f9e7fc640 1 -- 172.21.15.63:0/3349332237 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f50b0011040 con 0x559a7e229770 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f4f9e7fc640 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f90036eb0 0x7f4f90039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f4f9e7fc640 1 -- 172.21.15.63:0/3349332237 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f50b001d070 con 0x559a7e229770 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f90036eb0 0x7f4f90039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f50bd610800 1 Processor -- start 2024-04-23T19:10:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f50bd610800 1 -- start start 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f90036eb0 0x7f4f90039370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f50a8052d70 tx=0x7f50a8005240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f50bd610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e236eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.795+0000 7f50bd610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a7e111d60 con 0x559a7e236a90 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f50b955d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e236eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f50b955d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e236eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35860/0 (socket says 172.21.15.63:35860) 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f50b955d640 1 -- 172.21.15.63:0/116340147 learned_addr learned my addr 172.21.15.63:0/116340147 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f50b955d640 1 -- 172.21.15.63:0/116340147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7e237480 con 0x559a7e236a90 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f50b955d640 1 --2- 172.21.15.63:0/116340147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e236eb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f50b004cca0 tx=0x7f50b0012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd73430d7ea4be0b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f4f8ffff640 1 -- 172.21.15.63:0/116340147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b0042030 con 0x559a7e236a90 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f4f8ffff640 1 -- 172.21.15.63:0/116340147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b0001310 con 0x559a7e236a90 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.796+0000 7f4f8ffff640 1 -- 172.21.15.63:0/116340147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b0027040 con 0x559a7e236a90 2024-04-23T19:10:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 -- 172.21.15.63:0/116340147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 msgr2=0x559a7e236eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 --2- 172.21.15.63:0/116340147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e236eb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f50b004cca0 tx=0x7f50b0012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 -- 172.21.15.63:0/116340147 shutdown_connections 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 --2- 172.21.15.63:0/116340147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e236eb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 -- 172.21.15.63:0/116340147 >> 172.21.15.63:0/116340147 conn(0x559a7e232db0 msgr2=0x559a7e230e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 -- 172.21.15.63:0/116340147 shutdown_connections 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 -- 172.21.15.63:0/116340147 wait complete. 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 Processor -- start 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 -- start start 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e23dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.797+0000 7f50bd610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a7e237ba0 con 0x559a7e236a90 2024-04-23T19:10:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f50b955d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e23dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f50b955d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e23dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35872/0 (socket says 172.21.15.63:35872) 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 learned_addr learned my addr 172.21.15.63:0/1031062570 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7e23e520 con 0x559a7e236a90 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f50b955d640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e23dfe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f50b0001df0 tx=0x7f50b004cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f4f8e7fc640 1 -- 172.21.15.63:0/1031062570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b0027030 con 0x559a7e236a90 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f4f8e7fc640 1 -- 172.21.15.63:0/1031062570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b002a350 con 0x559a7e236a90 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a7e23e7b0 con 0x559a7e236a90 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f4f8e7fc640 1 -- 172.21.15.63:0/1031062570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b002a650 con 0x559a7e236a90 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.798+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7e241460 con 0x559a7e236a90 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.799+0000 7f4f8e7fc640 1 -- 172.21.15.63:0/1031062570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f50b0045070 con 0x559a7e236a90 2024-04-23T19:10:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.799+0000 7f4f8e7fc640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f84036eb0 0x7f4f84039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.799+0000 7f4f8e7fc640 1 -- 172.21.15.63:0/1031062570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f50b0043070 con 0x559a7e236a90 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.799+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f84036eb0 0x7f4f84039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.800+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f84036eb0 0x7f4f84039370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x559a7e21b730 tx=0x7f50a8086000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.800+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.800+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e249cc0 0x559a7e24c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.800+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e24c8d0 con 0x559a7e249cc0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.800+0000 7f50b855b640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e249cc0 0x559a7e24c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.800+0000 7f50b855b640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e249cc0 0x559a7e24c180 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f50a0009050 con 0x559a7e249cc0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 realm 2024-04-23T19:10:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e249110 con 0x559a7e249cc0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f50a0009050 con 0x559a7e249cc0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e24d3c0 con 0x559a7e249cc0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f50a0009050 con 0x559a7e249cc0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e24dc90 con 0x559a7e249cc0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f50a0009050 con 0x559a7e249cc0 2024-04-23T19:10:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e24e030 con 0x559a7e249cc0 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f50a0009050 con 0x559a7e249cc0 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 20 searching for the correct realm 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.801+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a7e24ff10 0x559a7e252300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.802+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.802+0000 7f50b955d640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a7e24ff10 0x559a7e252300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.802+0000 7f50b955d640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a7e24ff10 0x559a7e252300 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.802+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.802+0000 7f50b955d640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f50b00123a0 0x7f50b0046790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.802+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50b00123a0 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.802+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f50b00123a0 0x7f50b0046790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.803+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f50b00123a0 0x7f50b0046790 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.803+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.803+0000 7f50b8d5c640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f50a8078060 0x7f50a805ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.803+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.803+0000 7f50b855b640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f50a8078060 0x7f50a805ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.804+0000 7f50b855b640 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f50a8078060 0x7f50a805ca60 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.804+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a0009050 con 0x7f50a8078060 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.804+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.804+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a0009050 con 0x559a7e249cc0 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.804+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.804+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.804+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50b00123a0 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.805+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50b00123a0 2024-04-23T19:10:49.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.806+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.807+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.808+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50b00123a0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50b00123a0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.809+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.810+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.810+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.810+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.810+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50b00123a0 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.810+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.810+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e24ff10 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.811+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.811+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50a8078060 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.811+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.811+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x559a7e249cc0 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.811+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.811+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2529b0 con 0x7f50b00123a0 2024-04-23T19:10:49.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.811+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e24e9b0 con 0x559a7e249cc0 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e24f250 con 0x559a7e249cc0 2024-04-23T19:10:49.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e252e30 con 0x7f50a8078060 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 zone default found 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 4 Realm: () 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 zonegroup default 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.812+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2532e0 con 0x7f50b00123a0 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e255060 con 0x559a7e24ff10 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e255520 con 0x559a7e24ff10 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 20 started sync module instance, tier type = 2024-04-23T19:10:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.813+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e254500 con 0x7f50b00123a0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.814+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.814+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94122029697264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e255e30 con 0x7f50b00123a0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.814+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e256680 con 0x559a7e249cc0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.815+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94122029697264] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.816+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.816+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94122029701504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2566e0 con 0x559a7e249cc0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.816+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2571d0 con 0x559a7e24ff10 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.817+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94122029701504] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.817+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.817+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94122029706400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e257570 con 0x559a7e24ff10 2024-04-23T19:10:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.817+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e258030 con 0x7f50b00123a0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.818+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94122029706400] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.818+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.818+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94122029710080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e258420 con 0x7f50b00123a0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.818+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e258e60 con 0x559a7e249cc0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.819+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94122029710080] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.819+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.819+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94122029713712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e259250 con 0x559a7e249cc0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.819+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e259c70 con 0x7f50a8078060 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.820+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94122029713712] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.820+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.820+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94122029717344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e259c30 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.820+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e25a670 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.822+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94122029717344] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.822+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.822+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94122029719872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e25a630 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.822+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e25b050 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94122029719872] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94122029722432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e25b010 con 0x7f50a8078060 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 20 add_watcher() i=0 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 20 add_watcher() i=1 2024-04-23T19:10:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 20 add_watcher() i=2 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 20 add_watcher() i=3 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 20 add_watcher() i=4 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 20 add_watcher() i=5 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.823+0000 7f50bd610800 20 add_watcher() i=6 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.824+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94122029722432] v32'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.824+0000 7f50bd610800 20 add_watcher() i=7 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.824+0000 7f50bd610800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.824+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e25e4a0 con 0x559a7e249cc0 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.825+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.825+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94122029703632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e25b9c0 con 0x559a7e249cc0 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94122029703632] v32'187 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f4f6efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 5 note: GC not initialized 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f4f6dfe3640 20 reqs_thread_entry: start 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 20 Filter name: none 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f4edf7fe640 20 reqs_thread_entry: start 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.826+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2919f0 con 0x559a7e249cc0 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.827+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.827+0000 7f50bd610800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.827+0000 7f50bd610800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.827+0000 7f50bd610800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.827+0000 7f50bd610800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.827+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~403 in=403b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2978b0 con 0x559a7e249cc0 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~403] v32'14 uv14 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50bd610800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50bd610800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50bd610800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94122029970208 in=591b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559a7e293310 con 0x559a7e249cc0 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94122029713712 notify 137438953490 ret 0) ==== 621+0+0 (crc 0 0 0) 0x559a7dfc57f0 con 0x559a7e249cc0 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f4f8ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953490 cookie 94122029713712 notifier 4653 bl.length()=579 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f4f8ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f4f8ffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f4f8ffff640 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f4f7c003ea0 con 0x559a7e249cc0 2024-04-23T19:10:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(64 notify.4 [notify cookie 94122029970208 out=8b] v0'0 uv180 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.828+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.829+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94122029970208 notify 137438953490 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f50a003fa00 con 0x559a7e249cc0 2024-04-23T19:10:49.844 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[{"type":"user","perm":"read"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.829+0000 7f4f6efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.829+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94122029703632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e290920 con 0x559a7e249cc0 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.830+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94122029703632] v32'188 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.830+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94122029697264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2561d0 con 0x7f50b00123a0 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.831+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94122029697264] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.832+0000 7f50bd610800 20 remove_watcher() i=0 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.832+0000 7f50bd610800 2 removed watcher, disabling cache 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.832+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94122029701504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2548f0 con 0x559a7e249cc0 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.833+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94122029701504] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.833+0000 7f50bd610800 20 remove_watcher() i=1 2024-04-23T19:10:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.833+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94122029706400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2548f0 con 0x559a7e24ff10 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.834+0000 7f50b955d640 1 -- 172.21.15.63:0/1031062570 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94122029706400] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50b0029070 con 0x559a7e24ff10 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.834+0000 7f50bd610800 20 remove_watcher() i=2 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.834+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94122029710080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2548f0 con 0x7f50b00123a0 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.835+0000 7f50b8d5c640 1 -- 172.21.15.63:0/1031062570 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94122029710080] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a8075030 con 0x7f50b00123a0 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.835+0000 7f50bd610800 20 remove_watcher() i=3 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.835+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94122029713712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e257b00 con 0x559a7e249cc0 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.836+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94122029713712] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x559a7e249cc0 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.836+0000 7f50bd610800 20 remove_watcher() i=4 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.836+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94122029717344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2548f0 con 0x7f50a8078060 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.837+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94122029717344] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.837+0000 7f50bd610800 20 remove_watcher() i=5 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.837+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94122029719872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2548f0 con 0x7f50a8078060 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.838+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94122029719872] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.838+0000 7f50bd610800 20 remove_watcher() i=6 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.838+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94122029722432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559a7e2548f0 con 0x7f50a8078060 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94122029722432] v32'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f50a002a040 con 0x7f50a8078060 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 20 remove_watcher() i=7 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f50a8078060 msgr2=0x7f50a805ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f50a8078060 0x7f50a805ca60 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e249cc0 msgr2=0x559a7e24c180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e249cc0 0x559a7e24c180 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f50b00123a0 msgr2=0x7f50b0046790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f50b00123a0 0x7f50b0046790 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a7e24ff10 msgr2=0x559a7e252300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559a7e24ff10 0x559a7e252300 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f84036eb0 msgr2=0x7f4f84039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f84036eb0 0x7f4f84039370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x559a7e21b730 tx=0x7f50a8086000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 msgr2=0x559a7e23dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/1031062570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e236a90 0x559a7e23dfe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f50b0001df0 tx=0x7f50b004cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 reap_dead start 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50b855b640 1 -- 172.21.15.63:0/1031062570 reap_dead start 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 shutdown_connections 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 >> 172.21.15.63:0/1031062570 conn(0x559a7e232db0 msgr2=0x559a7e230410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 shutdown_connections 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/1031062570 wait complete. 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f90036eb0 msgr2=0x7f4f90039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f90036eb0 0x7f4f90039370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f50a8052d70 tx=0x7f50a8005240 comp rx=0 tx=0).stop 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e229770 msgr2=0x559a7e229b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e229770 0x559a7e229b90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559a7e20f470 tx=0x7f50b00178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 shutdown_connections 2024-04-23T19:10:49.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4f90036eb0 0x7f4f90039370 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 --2- 172.21.15.63:0/3349332237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e229770 0x559a7e229b90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 >> 172.21.15.63:0/3349332237 conn(0x559a7e21d810 msgr2=0x559a7e21dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 shutdown_connections 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.839+0000 7f50bd610800 1 -- 172.21.15.63:0/3349332237 wait complete. 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a7e2189e0 msgr2=0x559a7e21add0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a7e2189e0 0x559a7e21add0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e1164d0 msgr2=0x559a7e2152b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e1164d0 0x559a7e2152b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4fa0036eb0 msgr2=0x7f4fa0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4fa0036eb0 0x7f4fa0039370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000f60 tx=0x7f50a00012b0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 msgr2=0x559a7e20e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e20e070 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f50a800a470 tx=0x7f50a8003020 comp rx=0 tx=0).stop 2024-04-23T19:10:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 shutdown_connections 2024-04-23T19:10:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559a7e2189e0 0x559a7e21add0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559a7e1164d0 0x559a7e2152b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4fa0036eb0 0x7f4fa0039370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 --2- 172.21.15.63:0/1297303320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a7e113ba0 0x559a7e20e070 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 >> 172.21.15.63:0/1297303320 conn(0x559a7e10d630 msgr2=0x559a7e1122c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 shutdown_connections 2024-04-23T19:10:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.840+0000 7f50bd610800 1 -- 172.21.15.63:0/1297303320 wait complete. 2024-04-23T19:10:49.855 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:49.855 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-23T19:10:49.855 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-23T19:10:49.856 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-04-23T19:10:49.901 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:49.902 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.901+0000 7fe43290c800 1 Processor -- start 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.901+0000 7fe43290c800 1 -- start start 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.901+0000 7fe43290c800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55a05cd444a0 0x55a05cd46890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.901+0000 7fe43290c800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a05cbf37f0 con 0x55a05cd40010 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.901+0000 7fe43290c800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55a05cbe2e20 con 0x55a05cd444a0 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.901+0000 7fe42fc0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55a05cd444a0 0x55a05cd46890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.902+0000 7fe43040b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55a05cd40010 0x55a05cd40430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49714/0 (socket says 172.21.15.63:49714) 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.902+0000 7fe43040b640 1 -- 172.21.15.63:0/3485247622 learned_addr learned my addr 172.21.15.63:0/3485247622 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:49.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.902+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3485247622 >> v1:172.21.15.16:6789/0 conn(0x55a05cd40010 legacy=0x55a05cd40430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.902+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3485247622 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05cc01b50 con 0x55a05cd444a0 2024-04-23T19:10:49.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.904+0000 7fe42fc0a640 1 --2- 172.21.15.63:0/3485247622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd444a0 0x55a05cd46890 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe42400a470 tx=0x7fe42401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b881c989bfaed1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.904+0000 7fe42f409640 1 -- 172.21.15.63:0/3485247622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe424057070 con 0x55a05cd444a0 2024-04-23T19:10:49.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.904+0000 7fe42f409640 1 -- 172.21.15.63:0/3485247622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe424052570 con 0x55a05cd444a0 2024-04-23T19:10:49.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.904+0000 7fe42f409640 1 -- 172.21.15.63:0/3485247622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe424052870 con 0x55a05cd444a0 2024-04-23T19:10:49.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.905+0000 7fe43290c800 1 -- 172.21.15.63:0/3485247622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd444a0 msgr2=0x55a05cd46890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.905+0000 7fe43290c800 1 --2- 172.21.15.63:0/3485247622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd444a0 0x55a05cd46890 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe42400a470 tx=0x7fe42401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:49.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.905+0000 7fe43290c800 1 -- 172.21.15.63:0/3485247622 shutdown_connections 2024-04-23T19:10:49.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.905+0000 7fe43290c800 1 --2- 172.21.15.63:0/3485247622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd444a0 0x55a05cd46890 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.905+0000 7fe43290c800 1 -- 172.21.15.63:0/3485247622 >> 172.21.15.63:0/3485247622 conn(0x55a05cd3b630 msgr2=0x55a05cd3da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.905+0000 7fe43290c800 1 -- 172.21.15.63:0/3485247622 shutdown_connections 2024-04-23T19:10:49.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.905+0000 7fe43290c800 1 -- 172.21.15.63:0/3485247622 wait complete. 2024-04-23T19:10:49.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.917+0000 7fe43290c800 1 Processor -- start 2024-04-23T19:10:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.917+0000 7fe43290c800 1 -- start start 2024-04-23T19:10:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.918+0000 7fe43290c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ccb40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.918+0000 7fe43290c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a05cc8c6f0 con 0x55a05cd40010 2024-04-23T19:10:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.918+0000 7fe43040b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ccb40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.918+0000 7fe43040b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ccb40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35882/0 (socket says 172.21.15.63:35882) 2024-04-23T19:10:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.918+0000 7fe43040b640 1 -- 172.21.15.63:0/1443045664 learned_addr learned my addr 172.21.15.63:0/1443045664 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.918+0000 7fe43040b640 1 -- 172.21.15.63:0/1443045664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05ce350c0 con 0x55a05cd40010 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.919+0000 7fe43040b640 1 --2- 172.21.15.63:0/1443045664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ccb40e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe420001e10 tx=0x7fe42000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660ffa56adba5170 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.919+0000 7fe31e60c640 1 -- 172.21.15.63:0/1443045664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420012020 con 0x55a05cd40010 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.919+0000 7fe31e60c640 1 -- 172.21.15.63:0/1443045664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42000e5a0 con 0x55a05cd40010 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.919+0000 7fe31e60c640 1 -- 172.21.15.63:0/1443045664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe42000e8e0 con 0x55a05cd40010 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.919+0000 7fe43290c800 1 -- 172.21.15.63:0/1443045664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 msgr2=0x55a05ccb40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.919+0000 7fe43290c800 1 --2- 172.21.15.63:0/1443045664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ccb40e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe420001e10 tx=0x7fe42000b780 comp rx=0 tx=0).stop 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 -- 172.21.15.63:0/1443045664 shutdown_connections 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 --2- 172.21.15.63:0/1443045664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ccb40e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 -- 172.21.15.63:0/1443045664 >> 172.21.15.63:0/1443045664 conn(0x55a05cd3b630 msgr2=0x55a05ce2cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 -- 172.21.15.63:0/1443045664 shutdown_connections 2024-04-23T19:10:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 -- 172.21.15.63:0/1443045664 wait complete. 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 Processor -- start 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 -- start start 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ce329a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43290c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a05cd47160 con 0x55a05cd40010 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43040b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ce329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43040b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ce329a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35892/0 (socket says 172.21.15.63:35892) 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.920+0000 7fe43040b640 1 -- 172.21.15.63:0/2337359354 learned_addr learned my addr 172.21.15.63:0/2337359354 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.921+0000 7fe43040b640 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05ce32ee0 con 0x55a05cd40010 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.921+0000 7fe43040b640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ce329a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe42000b7c0 tx=0x7fe420000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.921+0000 7fe31ce09640 1 -- 172.21.15.63:0/2337359354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420012030 con 0x55a05cd40010 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.921+0000 7fe31ce09640 1 -- 172.21.15.63:0/2337359354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe420001470 con 0x55a05cd40010 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.921+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a05ce310e0 con 0x55a05cd40010 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.921+0000 7fe31ce09640 1 -- 172.21.15.63:0/2337359354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe42001a8d0 con 0x55a05cd40010 2024-04-23T19:10:49.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.921+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05ce31600 con 0x55a05cd40010 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe31ce09640 1 -- 172.21.15.63:0/2337359354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe420015020 con 0x55a05cd40010 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe31ce09640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe318036eb0 0x7fe318039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe31ce09640 1 -- 172.21.15.63:0/2337359354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe420020030 con 0x55a05cd40010 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe42fc0a640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe318036eb0 0x7fe318039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05cd444a0 0x55a05ce43390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce43910 con 0x55a05cd444a0 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe430c0c640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05cd444a0 0x55a05ce43390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.922+0000 7fe42fc0a640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe318036eb0 0x7fe318039370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe424054000 tx=0x7fe42404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.923+0000 7fe430c0c640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05cd444a0 0x55a05ce43390 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.923+0000 7fe430c0c640 1 -- 172.21.15.63:0/2337359354 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe42800a4a0 con 0x55a05cd444a0 2024-04-23T19:10:49.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.923+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce31b00 con 0x55a05cd444a0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.923+0000 7fe430c0c640 1 -- 172.21.15.63:0/2337359354 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe42800a4a0 con 0x55a05cd444a0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.923+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce45710 con 0x55a05cd444a0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.923+0000 7fe430c0c640 1 -- 172.21.15.63:0/2337359354 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe42800a4a0 con 0x55a05cd444a0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.924+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce44a50 con 0x55a05cd444a0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.924+0000 7fe430c0c640 1 -- 172.21.15.63:0/2337359354 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe42800a4a0 con 0x55a05cd444a0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.924+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a05ce46ac0 0x55a05ce48eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.924+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce49430 con 0x55a05ce46ac0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.924+0000 7fe43040b640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a05ce46ac0 0x55a05ce48eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.925+0000 7fe43040b640 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a05ce46ac0 0x55a05ce48eb0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.925+0000 7fe43040b640 1 -- 172.21.15.63:0/2337359354 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe420026d90 con 0x55a05ce46ac0 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.925+0000 7fe43290c800 1 Processor -- start 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.925+0000 7fe43290c800 1 -- start start 2024-04-23T19:10:49.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.925+0000 7fe43290c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce577e0 0x55a05ce57c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.925+0000 7fe43290c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a05cc32e30 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe430c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce577e0 0x55a05ce57c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe430c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce577e0 0x55a05ce57c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35896/0 (socket says 172.21.15.63:35896) 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe430c0c640 1 -- 172.21.15.63:0/972995167 learned_addr learned my addr 172.21.15.63:0/972995167 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe430c0c640 1 -- 172.21.15.63:0/972995167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05ce58140 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe430c0c640 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce577e0 0x55a05ce57c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55a05ce43cd0 tx=0x7fe4280178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe315ffb640 1 -- 172.21.15.63:0/972995167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe428013440 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe315ffb640 1 -- 172.21.15.63:0/972995167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4280135e0 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a05ce58bb0 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe315ffb640 1 -- 172.21.15.63:0/972995167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4280138c0 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.926+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05ce590b0 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.927+0000 7fe315ffb640 1 -- 172.21.15.63:0/972995167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe428011040 con 0x55a05ce577e0 2024-04-23T19:10:49.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.927+0000 7fe315ffb640 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe308036e60 0x7fe308039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.927+0000 7fe315ffb640 1 -- 172.21.15.63:0/972995167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe42801d070 con 0x55a05ce577e0 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.927+0000 7fe43040b640 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe308036e60 0x7fe308039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.927+0000 7fe43290c800 1 Processor -- start 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe43290c800 1 -- start start 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe43040b640 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe308036e60 0x7fe308039320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe42004b2a0 tx=0x7fe42002df20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe43290c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce64f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe43290c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a05ce35410 con 0x55a05ce64ae0 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe430c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce64f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe430c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce64f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35912/0 (socket says 172.21.15.63:35912) 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe430c0c640 1 -- 172.21.15.63:0/4079445502 learned_addr learned my addr 172.21.15.63:0/4079445502 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe430c0c640 1 -- 172.21.15.63:0/4079445502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05ce654d0 con 0x55a05ce64ae0 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe430c0c640 1 --2- 172.21.15.63:0/4079445502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce64f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe42804cc50 tx=0x7fe428001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8ebc166983ad17 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe30f7fe640 1 -- 172.21.15.63:0/4079445502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe428043070 con 0x55a05ce64ae0 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe30f7fe640 1 -- 172.21.15.63:0/4079445502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe428015ad0 con 0x55a05ce64ae0 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.928+0000 7fe30f7fe640 1 -- 172.21.15.63:0/4079445502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe42802ad80 con 0x55a05ce64ae0 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 -- 172.21.15.63:0/4079445502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 msgr2=0x55a05ce64f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 --2- 172.21.15.63:0/4079445502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce64f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe42804cc50 tx=0x7fe428001440 comp rx=0 tx=0).stop 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 -- 172.21.15.63:0/4079445502 shutdown_connections 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 --2- 172.21.15.63:0/4079445502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce64f00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 -- 172.21.15.63:0/4079445502 >> 172.21.15.63:0/4079445502 conn(0x55a05ce60e40 msgr2=0x55a05ce5eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 -- 172.21.15.63:0/4079445502 shutdown_connections 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 -- 172.21.15.63:0/4079445502 wait complete. 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 Processor -- start 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 -- start start 2024-04-23T19:10:49.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.929+0000 7fe43290c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce6c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe43290c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a05ce65c40 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe430c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce6c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe430c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce6c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35914/0 (socket says 172.21.15.63:35914) 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 learned_addr learned my addr 172.21.15.63:0/3969677640 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05ce6c5b0 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe430c0c640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce6c070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe42804baf0 tx=0x7fe42804cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe30dffb640 1 -- 172.21.15.63:0/3969677640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe428043030 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe30dffb640 1 -- 172.21.15.63:0/3969677640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4280123a0 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a05ce6c840 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe30dffb640 1 -- 172.21.15.63:0/3969677640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe428012680 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.930+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05ce6f4f0 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.931+0000 7fe30dffb640 1 -- 172.21.15.63:0/3969677640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe42804e110 con 0x55a05ce64ae0 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.931+0000 7fe30dffb640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe300036eb0 0x7fe300039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.931+0000 7fe30dffb640 1 -- 172.21.15.63:0/3969677640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe428044070 con 0x55a05ce64ae0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.931+0000 7fe43040b640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe300036eb0 0x7fe300039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.932+0000 7fe43040b640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe300036eb0 0x7fe300039370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe420016570 tx=0x7fe420034000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.932+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.932+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05ce77d50 0x55a05ce7a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.932+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce7a960 con 0x55a05ce77d50 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.932+0000 7fe42fc0a640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05ce77d50 0x55a05ce7a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.932+0000 7fe42fc0a640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05ce77d50 0x55a05ce7a210 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.932+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 realm 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce7ad30 con 0x55a05ce77d50 2024-04-23T19:10:49.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce7b600 con 0x55a05ce77d50 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce7b9a0 con 0x55a05ce77d50 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce7c1f0 con 0x55a05ce77d50 2024-04-23T19:10:49.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.933+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe43290c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe43290c800 20 searching for the correct realm 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a05ce7df60 0x55a05ce80350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe430c0c640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a05ce7df60 0x55a05ce80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe430c0c640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a05ce7df60 0x55a05ce80350 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe430c0c640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe42803e190 0x7fe42802c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe42803e190 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.934+0000 7fe43040b640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe42803e190 0x7fe42802c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.935+0000 7fe43040b640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe42803e190 0x7fe42802c450 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.935+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.935+0000 7fe43040b640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe420037070 0x7fe42004b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:49.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.935+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.935+0000 7fe42fc0a640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe420037070 0x7fe42004b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:49.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.936+0000 7fe42fc0a640 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe420037070 0x7fe42004b5f0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:49.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.936+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.936+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.936+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.936+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe42803e190 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.937+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.938+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe42803e190 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.939+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.940+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe42803e190 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.941+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe42803e190 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe42803e190 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.942+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce7df60 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.943+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.943+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe420037070 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.943+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.943+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x55a05ce77d50 2024-04-23T19:10:49.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.943+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.943+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce808e0 con 0x7fe42803e190 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce7cb70 con 0x55a05ce77d50 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce7d410 con 0x55a05ce77d50 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce80d60 con 0x7fe420037070 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 zone default found 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 4 Realm: () 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 zonegroup default 2024-04-23T19:10:49.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.944+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce81210 con 0x7fe42803e190 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce82f90 con 0x55a05ce7df60 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce83450 con 0x55a05ce7df60 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 20 started sync module instance, tier type = 2024-04-23T19:10:49.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.945+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce82430 con 0x7fe42803e190 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.946+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.946+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94147241846816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce82090 con 0x7fe42803e190 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.946+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce84380 con 0x55a05ce77d50 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.947+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.947+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94147241846816] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.947+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94147241852624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce847a0 con 0x55a05ce77d50 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.948+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce85160 con 0x55a05ce7df60 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.948+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94147241852624] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.949+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.949+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94147241853680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce84710 con 0x55a05ce7df60 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.949+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce86020 con 0x7fe42803e190 2024-04-23T19:10:49.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.950+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94147241853680] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.950+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.950+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94147241859952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce863c0 con 0x7fe42803e190 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.950+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce86e00 con 0x55a05ce77d50 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.951+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.951+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94147241863376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce876a0 con 0x55a05ce77d50 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.951+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce881c0 con 0x7fe420037070 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.951+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94147241859952] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.952+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94147241863376] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.952+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.952+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94147241868432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce87160 con 0x7fe420037070 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.952+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce88a60 con 0x7fe420037070 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.953+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94147241868432] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.953+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.953+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94147241858384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce87160 con 0x7fe420037070 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.953+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce89030 con 0x7fe420037070 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94147241858384] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94147241872160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce88ff0 con 0x7fe420037070 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 20 add_watcher() i=0 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 20 add_watcher() i=1 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 20 add_watcher() i=2 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 20 add_watcher() i=3 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 20 add_watcher() i=4 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 20 add_watcher() i=5 2024-04-23T19:10:49.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.954+0000 7fe43290c800 20 add_watcher() i=6 2024-04-23T19:10:49.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.955+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94147241872160] v32'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.955+0000 7fe43290c800 20 add_watcher() i=7 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.955+0000 7fe43290c800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.955+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce8c450 con 0x55a05ce77d50 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.956+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.956+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94147241870512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce89990 con 0x55a05ce77d50 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94147241870512] v32'189 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe2e67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe43290c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe43290c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe43290c800 5 note: GC not initialized 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe2e57e2640 20 reqs_thread_entry: start 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe43290c800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:49.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.957+0000 7fe43290c800 20 Filter name: none 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe256ffd640 20 reqs_thread_entry: start 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 20 rados->read ofs=0 len=0 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05cebf8d0 con 0x55a05ce77d50 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~403 out=403b,getxattrs out=75b] v0'0 uv14 ondisk = 0) ==== 231+0+526 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 20 rados_obj.operate() r=0 bl.length=403 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.958+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05cec57e0 con 0x55a05ce77d50 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v32'15 uv15 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe43290c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:49.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe43290c800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe43290c800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94147242119888 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a05cec11f0 con 0x55a05ce77d50 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify (1) cookie 94147241863376 notify 137438953491 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55a05cc01b50 con 0x55a05ce77d50 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe30f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953491 cookie 94147241863376 notifier 4660 bl.length()=567 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe30f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.959+0000 7fe30f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.960+0000 7fe30f7fe640 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fe2f4003ea0 con 0x55a05ce77d50 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.960+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(64 notify.4 [notify cookie 94147242119888 out=8b] v0'0 uv183 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.960+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.960+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== watch-notify(notify_complete (2) cookie 94147242119888 notify 137438953491 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fe42408f4f0 con 0x55a05ce77d50 2024-04-23T19:10:49.973 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-23T19:10:26.328391Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:10:26.328370Z","tags":[],"group_ids":[]} 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.961+0000 7fe2e67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.961+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94147241870512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce8bfb0 con 0x55a05ce77d50 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.962+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94147241870512] v32'190 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.962+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94147241846816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce884b0 con 0x7fe42803e190 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.963+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94147241846816] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.963+0000 7fe43290c800 20 remove_watcher() i=0 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.963+0000 7fe43290c800 2 removed watcher, disabling cache 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.963+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94147241852624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce884b0 con 0x55a05ce77d50 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.964+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94147241852624] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.964+0000 7fe43290c800 20 remove_watcher() i=1 2024-04-23T19:10:49.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.964+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94147241853680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce83ed0 con 0x55a05ce7df60 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.965+0000 7fe430c0c640 1 -- 172.21.15.63:0/3969677640 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94147241853680] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe428029070 con 0x55a05ce7df60 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.965+0000 7fe43290c800 20 remove_watcher() i=2 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.965+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94147241859952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce884b0 con 0x7fe42803e190 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.966+0000 7fe43040b640 1 -- 172.21.15.63:0/3969677640 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94147241859952] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe420018430 con 0x7fe42803e190 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.966+0000 7fe43290c800 20 remove_watcher() i=3 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.966+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94147241863376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce85b70 con 0x55a05ce77d50 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.967+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94147241863376] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x55a05ce77d50 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.967+0000 7fe43290c800 20 remove_watcher() i=4 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.967+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94147241868432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce884b0 con 0x7fe420037070 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.968+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94147241868432] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.968+0000 7fe43290c800 20 remove_watcher() i=5 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.968+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94147241858384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce87c90 con 0x7fe420037070 2024-04-23T19:10:49.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.969+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94147241858384] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.969+0000 7fe43290c800 20 remove_watcher() i=6 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.969+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94147241872160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a05ce884b0 con 0x7fe420037070 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94147241872160] v32'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe42405a040 con 0x7fe420037070 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 20 remove_watcher() i=7 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe420037070 msgr2=0x7fe42004b5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe420037070 0x7fe42004b5f0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05ce77d50 msgr2=0x55a05ce7a210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05ce77d50 0x55a05ce7a210 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe42803e190 msgr2=0x7fe42802c450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe42803e190 0x7fe42802c450 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a05ce7df60 msgr2=0x55a05ce80350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a05ce7df60 0x55a05ce80350 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe300036eb0 msgr2=0x7fe300039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe300036eb0 0x7fe300039370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe420016570 tx=0x7fe420034000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 msgr2=0x55a05ce6c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 --2- 172.21.15.63:0/3969677640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce64ae0 0x55a05ce6c070 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe42804baf0 tx=0x7fe42804cbb0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 reap_dead start 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe42fc0a640 1 -- 172.21.15.63:0/3969677640 reap_dead start 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 shutdown_connections 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 >> 172.21.15.63:0/3969677640 conn(0x55a05ce60e40 msgr2=0x55a05ce5e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.970+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 shutdown_connections 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/3969677640 wait complete. 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe308036e60 msgr2=0x7fe308039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe308036e60 0x7fe308039320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe42004b2a0 tx=0x7fe42002df20 comp rx=0 tx=0).stop 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce577e0 msgr2=0x55a05ce57c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce577e0 0x55a05ce57c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55a05ce43cd0 tx=0x7fe4280178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 shutdown_connections 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe308036e60 0x7fe308039320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/972995167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05ce577e0 0x55a05ce57c00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 >> 172.21.15.63:0/972995167 conn(0x55a05ce4b880 msgr2=0x55a05ce4bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 shutdown_connections 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/972995167 wait complete. 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a05ce46ac0 msgr2=0x55a05ce48eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a05ce46ac0 0x55a05ce48eb0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05cd444a0 msgr2=0x55a05ce43390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05cd444a0 0x55a05ce43390 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe318036eb0 msgr2=0x7fe318039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe318036eb0 0x7fe318039370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe424054000 tx=0x7fe42404e000 comp rx=0 tx=0).stop 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 msgr2=0x55a05ce329a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:49.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ce329a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe42000b7c0 tx=0x7fe420000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 shutdown_connections 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a05ce46ac0 0x55a05ce48eb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a05cd444a0 0x55a05ce43390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe318036eb0 0x7fe318039370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 --2- 172.21.15.63:0/2337359354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a05cd40010 0x55a05ce329a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 >> 172.21.15.63:0/2337359354 conn(0x55a05cd3b630 msgr2=0x55a05ce2b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 shutdown_connections 2024-04-23T19:10:49.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:49.971+0000 7fe43290c800 1 -- 172.21.15.63:0/2337359354 wait complete. 2024-04-23T19:10:49.984 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-23T19:10:26.328391Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:10:26.328370Z', 'tags': [], 'group_ids': []} 2024-04-23T19:10:49.994 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:49.994 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-23T19:10:49.994 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-23T19:10:49.995 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-23T19:10:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:10:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65f63f800 1 Processor -- start 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65f63f800 1 -- start start 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65f63f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5559bfb9f730 0x5559bfba1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65f63f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5559bfa4d7f0 con 0x5559bfb9bc20 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65f63f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5559bfa3ce20 con 0x5559bfb9f730 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65c93d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5559bfb9f730 0x5559bfba1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65d13e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5559bfb9bc20 0x5559bfb9e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:49720/0 (socket says 172.21.15.63:49720) 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.040+0000 7fe65d13e640 1 -- 172.21.15.63:0/3409022672 learned_addr learned my addr 172.21.15.63:0/3409022672 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.041+0000 7fe65c93d640 1 -- 172.21.15.63:0/3409022672 >> v1:172.21.15.16:6789/0 conn(0x5559bfb9bc20 legacy=0x5559bfb9e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.041+0000 7fe65c93d640 1 -- 172.21.15.63:0/3409022672 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bfa5bb50 con 0x5559bfb9f730 2024-04-23T19:10:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.042+0000 7fe65c93d640 1 --2- 172.21.15.63:0/3409022672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9f730 0x5559bfba1b20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe65400a470 tx=0x7fe65401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ce7d5cabd844f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe64ffff640 1 -- 172.21.15.63:0/3409022672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe654057070 con 0x5559bfb9f730 2024-04-23T19:10:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe64ffff640 1 -- 172.21.15.63:0/3409022672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe654052570 con 0x5559bfb9f730 2024-04-23T19:10:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe64ffff640 1 -- 172.21.15.63:0/3409022672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe654052870 con 0x5559bfb9f730 2024-04-23T19:10:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe65f63f800 1 -- 172.21.15.63:0/3409022672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9f730 msgr2=0x5559bfba1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe65f63f800 1 --2- 172.21.15.63:0/3409022672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9f730 0x5559bfba1b20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe65400a470 tx=0x7fe65401ce90 comp rx=0 tx=0).stop 2024-04-23T19:10:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe65f63f800 1 -- 172.21.15.63:0/3409022672 shutdown_connections 2024-04-23T19:10:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe65f63f800 1 --2- 172.21.15.63:0/3409022672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9f730 0x5559bfba1b20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe65f63f800 1 -- 172.21.15.63:0/3409022672 >> 172.21.15.63:0/3409022672 conn(0x5559bfb95630 msgr2=0x5559bfb97a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe65f63f800 1 -- 172.21.15.63:0/3409022672 shutdown_connections 2024-04-23T19:10:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.043+0000 7fe65f63f800 1 -- 172.21.15.63:0/3409022672 wait complete. 2024-04-23T19:10:50.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.056+0000 7fe65f63f800 1 Processor -- start 2024-04-23T19:10:50.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.056+0000 7fe65f63f800 1 -- start start 2024-04-23T19:10:50.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.056+0000 7fe65f63f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc8c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.056+0000 7fe65f63f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559bfae66f0 con 0x5559bfb9bc20 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.056+0000 7fe65d13e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc8c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.056+0000 7fe65d13e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc8c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35932/0 (socket says 172.21.15.63:35932) 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.056+0000 7fe65d13e640 1 -- 172.21.15.63:0/2389404654 learned_addr learned my addr 172.21.15.63:0/2389404654 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.057+0000 7fe65d13e640 1 -- 172.21.15.63:0/2389404654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bfc8c980 con 0x5559bfb9bc20 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.057+0000 7fe65d13e640 1 --2- 172.21.15.63:0/2389404654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc8c3b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe650001de0 tx=0x7fe65000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c8fe46903cdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.057+0000 7fe54b606640 1 -- 172.21.15.63:0/2389404654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe650012020 con 0x5559bfb9bc20 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.057+0000 7fe54b606640 1 -- 172.21.15.63:0/2389404654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65000e5a0 con 0x5559bfb9bc20 2024-04-23T19:10:50.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.057+0000 7fe54b606640 1 -- 172.21.15.63:0/2389404654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65000e8c0 con 0x5559bfb9bc20 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 -- 172.21.15.63:0/2389404654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 msgr2=0x5559bfc8c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2389404654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc8c3b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe650001de0 tx=0x7fe65000b6c0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 -- 172.21.15.63:0/2389404654 shutdown_connections 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2389404654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc8c3b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 -- 172.21.15.63:0/2389404654 >> 172.21.15.63:0/2389404654 conn(0x5559bfb95630 msgr2=0x5559bfc87cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 -- 172.21.15.63:0/2389404654 shutdown_connections 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 -- 172.21.15.63:0/2389404654 wait complete. 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 Processor -- start 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 -- start start 2024-04-23T19:10:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc93ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65f63f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559bfba4370 con 0x5559bfb9bc20 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65d13e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc93ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65d13e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc93ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35938/0 (socket says 172.21.15.63:35938) 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.058+0000 7fe65d13e640 1 -- 172.21.15.63:0/1286825250 learned_addr learned my addr 172.21.15.63:0/1286825250 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.059+0000 7fe65d13e640 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bfc94430 con 0x5559bfb9bc20 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.059+0000 7fe65d13e640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc93ef0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe65000c010 tx=0x7fe650000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.059+0000 7fe549e03640 1 -- 172.21.15.63:0/1286825250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe650012070 con 0x5559bfb9bc20 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.059+0000 7fe549e03640 1 -- 172.21.15.63:0/1286825250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65001a7a0 con 0x5559bfb9bc20 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.059+0000 7fe549e03640 1 -- 172.21.15.63:0/1286825250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65001aa80 con 0x5559bfb9bc20 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.059+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5559bfc946c0 con 0x5559bfb9bc20 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.059+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bfc97370 con 0x5559bfb9bc20 2024-04-23T19:10:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.060+0000 7fe549e03640 1 -- 172.21.15.63:0/1286825250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe650015020 con 0x5559bfb9bc20 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.060+0000 7fe549e03640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe544036eb0 0x7fe544039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.060+0000 7fe549e03640 1 -- 172.21.15.63:0/1286825250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe650020070 con 0x5559bfb9bc20 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.060+0000 7fe65c93d640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe544036eb0 0x7fe544039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.060+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfb9f730 0x5559bfc9b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.060+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfc9b670 con 0x5559bfb9f730 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.061+0000 7fe65d93f640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfb9f730 0x5559bfc9b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.061+0000 7fe65c93d640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe544036eb0 0x7fe544039370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe654054000 tx=0x7fe65404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.061+0000 7fe65d93f640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfb9f730 0x5559bfc9b130 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.061+0000 7fe65d93f640 1 -- 172.21.15.63:0/1286825250 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe65800a4a0 con 0x5559bfb9f730 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.061+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfc98700 con 0x5559bfb9f730 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.061+0000 7fe65d93f640 1 -- 172.21.15.63:0/1286825250 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe65800a4a0 con 0x5559bfb9f730 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.061+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfc9d4b0 con 0x5559bfb9f730 2024-04-23T19:10:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.062+0000 7fe65d93f640 1 -- 172.21.15.63:0/1286825250 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe65800a4a0 con 0x5559bfb9f730 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.062+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfc9c7f0 con 0x5559bfb9f730 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.062+0000 7fe65d93f640 1 -- 172.21.15.63:0/1286825250 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe65800a4a0 con 0x5559bfb9f730 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.062+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559bfc9e860 0x5559bfca0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.062+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfca1210 con 0x5559bfc9e860 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.062+0000 7fe65d13e640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559bfc9e860 0x5559bfca0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.063+0000 7fe65d13e640 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559bfc9e860 0x5559bfca0c50 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.063+0000 7fe65d13e640 1 -- 172.21.15.63:0/1286825250 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe65002b070 con 0x5559bfc9e860 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.063+0000 7fe65f63f800 1 Processor -- start 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.063+0000 7fe65f63f800 1 -- start start 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.063+0000 7fe65f63f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcaf660 0x5559bfcafa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.063+0000 7fe65f63f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559bfa8ce30 con 0x5559bfcaf660 2024-04-23T19:10:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe65d93f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcaf660 0x5559bfcafa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe65d93f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcaf660 0x5559bfcafa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35940/0 (socket says 172.21.15.63:35940) 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe65d93f640 1 -- 172.21.15.63:0/315790993 learned_addr learned my addr 172.21.15.63:0/315790993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe65d93f640 1 -- 172.21.15.63:0/315790993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bfcaffc0 con 0x5559bfcaf660 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe65d93f640 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcaf660 0x5559bfcafa80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x5559bfc952f0 tx=0x7fe6580178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe542ffd640 1 -- 172.21.15.63:0/315790993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe658013440 con 0x5559bfcaf660 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe542ffd640 1 -- 172.21.15.63:0/315790993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580135e0 con 0x5559bfcaf660 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5559bfcb0a30 con 0x5559bfcaf660 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe542ffd640 1 -- 172.21.15.63:0/315790993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6580138c0 con 0x5559bfcaf660 2024-04-23T19:10:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.064+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bfcb0f30 con 0x5559bfcaf660 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.065+0000 7fe542ffd640 1 -- 172.21.15.63:0/315790993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe658011040 con 0x5559bfcaf660 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.065+0000 7fe542ffd640 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe538036e60 0x7fe538039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.065+0000 7fe542ffd640 1 -- 172.21.15.63:0/315790993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe65801d070 con 0x5559bfcaf660 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.065+0000 7fe65d13e640 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe538036e60 0x7fe538039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.065+0000 7fe65f63f800 1 Processor -- start 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65f63f800 1 -- start start 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65d13e640 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe538036e60 0x7fe538039320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe65004b0b0 tx=0x7fe65004af80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65f63f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcbcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65f63f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559bfc8cd00 con 0x5559bfcbc9d0 2024-04-23T19:10:50.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65d93f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcbcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65d93f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcbcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35954/0 (socket says 172.21.15.63:35954) 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65d93f640 1 -- 172.21.15.63:0/454421686 learned_addr learned my addr 172.21.15.63:0/454421686 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65d93f640 1 -- 172.21.15.63:0/454421686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bfcbd3c0 con 0x5559bfcbc9d0 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe65d93f640 1 --2- 172.21.15.63:0/454421686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcbcdf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe65804cca0 tx=0x7fe658012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2b94671d8daf2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe540ff9640 1 -- 172.21.15.63:0/454421686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe658042030 con 0x5559bfcbc9d0 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe540ff9640 1 -- 172.21.15.63:0/454421686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe658001310 con 0x5559bfcbc9d0 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.066+0000 7fe540ff9640 1 -- 172.21.15.63:0/454421686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe658027040 con 0x5559bfcbc9d0 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 -- 172.21.15.63:0/454421686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 msgr2=0x5559bfcbcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 --2- 172.21.15.63:0/454421686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcbcdf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe65804cca0 tx=0x7fe658012a00 comp rx=0 tx=0).stop 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 -- 172.21.15.63:0/454421686 shutdown_connections 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 --2- 172.21.15.63:0/454421686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcbcdf0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 -- 172.21.15.63:0/454421686 >> 172.21.15.63:0/454421686 conn(0x5559bfcb8ca0 msgr2=0x5559bfcb6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 -- 172.21.15.63:0/454421686 shutdown_connections 2024-04-23T19:10:50.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 -- 172.21.15.63:0/454421686 wait complete. 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 Processor -- start 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 -- start start 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.067+0000 7fe65f63f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcc3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65f63f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559bfcbdb50 con 0x5559bfcbc9d0 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65d93f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcc3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65d93f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcc3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:35960/0 (socket says 172.21.15.63:35960) 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 learned_addr learned my addr 172.21.15.63:0/2024492623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bfcc4430 con 0x5559bfcbc9d0 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65d93f640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcc3ef0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe658001df0 tx=0x7fe65804cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe52effd640 1 -- 172.21.15.63:0/2024492623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe658027030 con 0x5559bfcbc9d0 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe52effd640 1 -- 172.21.15.63:0/2024492623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65802a350 con 0x5559bfcbc9d0 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5559bfcc46c0 con 0x5559bfcbc9d0 2024-04-23T19:10:50.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe52effd640 1 -- 172.21.15.63:0/2024492623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe65802a650 con 0x5559bfcbc9d0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.068+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bfcc7370 con 0x5559bfcbc9d0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.069+0000 7fe52effd640 1 -- 172.21.15.63:0/2024492623 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe658045070 con 0x5559bfcbc9d0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.069+0000 7fe52effd640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe528036eb0 0x7fe528039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.069+0000 7fe52effd640 1 -- 172.21.15.63:0/2024492623 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe658043070 con 0x5559bfcbc9d0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.069+0000 7fe65d13e640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe528036eb0 0x7fe528039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.069+0000 7fe65d13e640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe528036eb0 0x7fe528039370 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5559bfca15f0 tx=0x7fe650030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfccfbd0 0x5559bfcd2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd27e0 con 0x5559bfccfbd0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65c93d640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfccfbd0 0x5559bfcd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65c93d640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfccfbd0 0x5559bfcd2090 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65f63f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65f63f800 20 realm 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.070+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd2bb0 con 0x5559bfccfbd0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd3480 con 0x5559bfccfbd0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd3820 con 0x5559bfccfbd0 2024-04-23T19:10:50.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd4070 con 0x5559bfccfbd0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 20 searching for the correct realm 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.071+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559bfcd5de0 0x5559bfcd81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.072+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.072+0000 7fe65d93f640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559bfcd5de0 0x5559bfcd81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.072+0000 7fe65d93f640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559bfcd5de0 0x5559bfcd81d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.072+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.072+0000 7fe65d93f640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6580123a0 0x7fe65802c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.072+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe6580123a0 2024-04-23T19:10:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.072+0000 7fe65d13e640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6580123a0 0x7fe65802c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.073+0000 7fe65d13e640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6580123a0 0x7fe65802c790 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.073+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.073+0000 7fe65d13e640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe650017050 0x7fe650019000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.073+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.073+0000 7fe65c93d640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe650017050 0x7fe650019000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.074+0000 7fe65c93d640 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe650017050 0x7fe650019000 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.074+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.074+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.074+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.074+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.074+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.074+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe6580123a0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.075+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.076+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.076+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.076+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.076+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.076+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.076+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe6580123a0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.076+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.077+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.077+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.077+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.077+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.077+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.077+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.077+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.078+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe6580123a0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.079+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe6580123a0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.080+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.080+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.080+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.080+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.080+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.080+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe6580123a0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfcd5de0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe650017050 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x5559bfccfbd0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.081+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8760 con 0x7fe6580123a0 2024-04-23T19:10:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd4a20 con 0x5559bfccfbd0 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd52c0 con 0x5559bfccfbd0 2024-04-23T19:10:50.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd8be0 con 0x7fe650017050 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.082+0000 7fe65f63f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 zone default found 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 4 Realm: () 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 zonegroup default 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcd9090 con 0x7fe6580123a0 2024-04-23T19:10:50.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdae10 con 0x5559bfcd5de0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdb290 con 0x5559bfcd5de0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 started sync module instance, tier type = 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.083+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcda2b0 con 0x7fe6580123a0 2024-04-23T19:10:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.085+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.085+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93843958376096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdbba0 con 0x7fe6580123a0 2024-04-23T19:10:50.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.085+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdc3f0 con 0x5559bfccfbd0 2024-04-23T19:10:50.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.086+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93843958376096] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.086+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.086+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93843958380272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdc470 con 0x5559bfccfbd0 2024-04-23T19:10:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.086+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdcf70 con 0x5559bfcd5de0 2024-04-23T19:10:50.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.087+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.087+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93843958385248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdd800 con 0x5559bfcd5de0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.087+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcde2f0 con 0x7fe6580123a0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.087+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93843958380272] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.088+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93843958385248] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.088+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.088+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93843958390208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdd310 con 0x7fe6580123a0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.088+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcded30 con 0x5559bfccfbd0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.089+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.089+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93843958392960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdf150 con 0x5559bfccfbd0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.089+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdfc40 con 0x7fe650017050 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.089+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93843958390208] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.090+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.090+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93843958392960] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.090+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93843958396688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdfc00 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.090+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdeca0 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.091+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93843958396688] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.091+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.092+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93843958394016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce01a0 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.092+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0b60 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93843958400688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0f80 con 0x7fe650017050 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=0 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=1 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=2 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=3 2024-04-23T19:10:50.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=4 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=5 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93843958394016] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=6 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93843958400688] v32'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 20 add_watcher() i=7 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.093+0000 7fe65f63f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.094+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0f80 con 0x5559bfccfbd0 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.094+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.094+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93843958401744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0ef0 con 0x5559bfccfbd0 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.095+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93843958401744] v32'191 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.095+0000 7fe5137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.095+0000 7fe65f63f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:10:50.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.095+0000 7fe65f63f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.095+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcec770 con 0x5559bfccfbd0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.096+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'32 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.096+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcece40 con 0x7fe650017050 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.097+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.098+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfced260 con 0x5559bfcd5de0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.099+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.099+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfced240 con 0x5559bfccfbd0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.100+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'192 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.100+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfced810 con 0x7fe6580123a0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.101+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'65 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.101+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcedc90 con 0x5559bfcd5de0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.101+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'43 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.102+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcedc90 con 0x5559bfccfbd0 2024-04-23T19:10:50.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.103+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.103+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcedc90 con 0x7fe650017050 2024-04-23T19:10:50.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.104+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'43 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.104+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcedc90 con 0x7fe6580123a0 2024-04-23T19:10:50.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.105+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.105+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.105+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.106+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe6580123a0 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.106+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'66 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.106+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe650017050 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.108+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.108+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe6580123a0 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.109+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.109+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe650017050 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.110+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'49 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.110+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfccfbd0 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.111+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'193 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.111+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.112+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'60 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.112+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe6580123a0 2024-04-23T19:10:50.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.113+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'67 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.113+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.114+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.114+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfccfbd0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.115+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'33 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.115+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfccfbd0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.116+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'194 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.116+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.117+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'61 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.117+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe6580123a0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.118+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'35 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.118+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.119+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.119+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.120+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.120+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.121+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'21 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.121+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.122+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'44 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.122+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe650017050 2024-04-23T19:10:50.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.123+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.123+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.124+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.124+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe650017050 2024-04-23T19:10:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.125+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'44 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.125+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x5559bfcd5de0 2024-04-23T19:10:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.126+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'62 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.126+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe650017050 2024-04-23T19:10:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.128+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'50 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.128+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcee360 con 0x7fe6580123a0 2024-04-23T19:10:50.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe5127e4640 20 reqs_thread_entry: start 2024-04-23T19:10:50.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe65f63f800 20 init_complete bucket index max shards: 11 2024-04-23T19:10:50.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe65f63f800 20 Filter name: none 2024-04-23T19:10:50.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe483fff640 20 reqs_thread_entry: start 2024-04-23T19:10:50.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe65f63f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:10:50.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe65f63f800 20 rados->read ofs=0 len=0 2024-04-23T19:10:50.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.129+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfd191c0 con 0x5559bfccfbd0 2024-04-23T19:10:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.130+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.130+0000 7fe65f63f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:10:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.130+0000 7fe65f63f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:10:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.130+0000 7fe65f63f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:10:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.130+0000 7fe65f63f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:10:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.130+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559bfd18ce0 con 0x7fe6580123a0 2024-04-23T19:10:50.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.130+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv41 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.133 INFO:teuthology.orchestra.run.smithi063.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-23T19:10:50.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.131+0000 7fe5137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:10:50.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.131+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93843958401744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfceb680 con 0x5559bfccfbd0 2024-04-23T19:10:50.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.132+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 93843958401744] v32'195 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.132+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93843958376096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0ad0 con 0x7fe6580123a0 2024-04-23T19:10:50.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.133+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 93843958376096] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.133+0000 7fe65f63f800 20 remove_watcher() i=0 2024-04-23T19:10:50.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.133+0000 7fe65f63f800 2 removed watcher, disabling cache 2024-04-23T19:10:50.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.133+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93843958380272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0ad0 con 0x5559bfccfbd0 2024-04-23T19:10:50.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.134+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 93843958380272] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.134+0000 7fe65f63f800 20 remove_watcher() i=1 2024-04-23T19:10:50.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.134+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93843958385248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdb6f0 con 0x5559bfcd5de0 2024-04-23T19:10:50.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.135+0000 7fe65d93f640 1 -- 172.21.15.63:0/2024492623 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 93843958385248] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe658029070 con 0x5559bfcd5de0 2024-04-23T19:10:50.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.135+0000 7fe65f63f800 20 remove_watcher() i=2 2024-04-23T19:10:50.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.135+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93843958390208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0ad0 con 0x7fe6580123a0 2024-04-23T19:10:50.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.136+0000 7fe65d13e640 1 -- 172.21.15.63:0/2024492623 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 93843958390208] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe650055040 con 0x7fe6580123a0 2024-04-23T19:10:50.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.136+0000 7fe65f63f800 20 remove_watcher() i=3 2024-04-23T19:10:50.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.136+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93843958392960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdb6f0 con 0x5559bfccfbd0 2024-04-23T19:10:50.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.137+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 93843958392960] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x5559bfccfbd0 2024-04-23T19:10:50.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.137+0000 7fe65f63f800 20 remove_watcher() i=4 2024-04-23T19:10:50.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.137+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93843958396688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdddc0 con 0x7fe650017050 2024-04-23T19:10:50.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.138+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 93843958396688] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.138+0000 7fe65f63f800 20 remove_watcher() i=5 2024-04-23T19:10:50.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.138+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93843958394016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfce0ad0 con 0x7fe650017050 2024-04-23T19:10:50.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.139+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 93843958394016] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.139+0000 7fe65f63f800 20 remove_watcher() i=6 2024-04-23T19:10:50.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.139+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93843958400688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559bfcdb6f0 con 0x7fe650017050 2024-04-23T19:10:50.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 93843958400688] v32'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe65405a040 con 0x7fe650017050 2024-04-23T19:10:50.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 20 remove_watcher() i=7 2024-04-23T19:10:50.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe650017050 msgr2=0x7fe650019000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe650017050 0x7fe650019000 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfccfbd0 msgr2=0x5559bfcd2090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfccfbd0 0x5559bfcd2090 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6580123a0 msgr2=0x7fe65802c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6580123a0 0x7fe65802c790 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559bfcd5de0 msgr2=0x5559bfcd81d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559bfcd5de0 0x5559bfcd81d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.140+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe528036eb0 msgr2=0x7fe528039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe528036eb0 0x7fe528039370 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5559bfca15f0 tx=0x7fe650030000 comp rx=0 tx=0).stop 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 msgr2=0x5559bfcc3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/2024492623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcbc9d0 0x5559bfcc3ef0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe658001df0 tx=0x7fe65804cc00 comp rx=0 tx=0).stop 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 reap_dead start 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65c93d640 1 -- 172.21.15.63:0/2024492623 reap_dead start 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 shutdown_connections 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 >> 172.21.15.63:0/2024492623 conn(0x5559bfcb8ca0 msgr2=0x5559bfcb6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 shutdown_connections 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/2024492623 wait complete. 2024-04-23T19:10:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe538036e60 msgr2=0x7fe538039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe538036e60 0x7fe538039320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe65004b0b0 tx=0x7fe65004af80 comp rx=0 tx=0).stop 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcaf660 msgr2=0x5559bfcafa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcaf660 0x5559bfcafa80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x5559bfc952f0 tx=0x7fe6580178e0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 shutdown_connections 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe538036e60 0x7fe538039320 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/315790993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfcaf660 0x5559bfcafa80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 >> 172.21.15.63:0/315790993 conn(0x5559bfca3700 msgr2=0x5559bfca3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 shutdown_connections 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/315790993 wait complete. 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559bfc9e860 msgr2=0x5559bfca0c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559bfc9e860 0x5559bfca0c50 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfb9f730 msgr2=0x5559bfc9b130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfb9f730 0x5559bfc9b130 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe544036eb0 msgr2=0x7fe544039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe544036eb0 0x7fe544039370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe654054000 tx=0x7fe65404e000 comp rx=0 tx=0).stop 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 msgr2=0x5559bfc93ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.141+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc93ef0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe65000c010 tx=0x7fe650000ed0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 shutdown_connections 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559bfc9e860 0x5559bfca0c50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559bfb9f730 0x5559bfc9b130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe544036eb0 0x7fe544039370 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 --2- 172.21.15.63:0/1286825250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559bfb9bc20 0x5559bfc93ef0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 >> 172.21.15.63:0/1286825250 conn(0x5559bfb95630 msgr2=0x5559bfc864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 shutdown_connections 2024-04-23T19:10:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:10:50.142+0000 7fe65f63f800 1 -- 172.21.15.63:0/1286825250 wait complete. 2024-04-23T19:10:50.151 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-23T19:10:50.165 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:10:50.170 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-23T19:11:25.171 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-04-23T19:11:25.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-04-23T19:11:25.172 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-04-23T19:11:25.224 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:25.224 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.223+0000 7f4c19d2b800 1 Processor -- start 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.223+0000 7f4c19d2b800 1 -- start start 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.223+0000 7f4c19d2b800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56080c61a010 0x56080c61a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.223+0000 7f4c19d2b800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56080c4cd7f0 con 0x56080c61b270 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.223+0000 7f4c19d2b800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x56080c4bce20 con 0x56080c61a010 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.224+0000 7f4c15aef640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x56080c61a010 0x56080c61a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.224+0000 7f4c152ee640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x56080c61b270 0x56080c61b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:58950/0 (socket says 172.21.15.63:58950) 2024-04-23T19:11:25.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.224+0000 7f4c152ee640 1 -- 172.21.15.63:0/1040244910 learned_addr learned my addr 172.21.15.63:0/1040244910 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.224+0000 7f4c15aef640 1 -- 172.21.15.63:0/1040244910 >> v1:172.21.15.16:6789/0 conn(0x56080c61b270 legacy=0x56080c61b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.224+0000 7f4c15aef640 1 -- 172.21.15.63:0/1040244910 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c4dbb50 con 0x56080c61a010 2024-04-23T19:11:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.226+0000 7f4c15aef640 1 --2- 172.21.15.63:0/1040244910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c61a430 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800a470 tx=0x7f4c0801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cef3e782a5c090 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.226+0000 7f4c14aed640 1 -- 172.21.15.63:0/1040244910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c08057070 con 0x56080c61a010 2024-04-23T19:11:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.226+0000 7f4c14aed640 1 -- 172.21.15.63:0/1040244910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08052570 con 0x56080c61a010 2024-04-23T19:11:25.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.226+0000 7f4c14aed640 1 -- 172.21.15.63:0/1040244910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c08052870 con 0x56080c61a010 2024-04-23T19:11:25.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.227+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1040244910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 msgr2=0x56080c61a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.227+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/1040244910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c61a430 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800a470 tx=0x7f4c0801ce90 comp rx=0 tx=0).stop 2024-04-23T19:11:25.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.227+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1040244910 shutdown_connections 2024-04-23T19:11:25.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.227+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/1040244910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c61a430 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.227+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1040244910 >> 172.21.15.63:0/1040244910 conn(0x56080c615630 msgr2=0x56080c617a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.227+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1040244910 shutdown_connections 2024-04-23T19:11:25.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.227+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1040244910 wait complete. 2024-04-23T19:11:25.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.242+0000 7f4c19d2b800 1 Processor -- start 2024-04-23T19:11:25.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.242+0000 7f4c19d2b800 1 -- start start 2024-04-23T19:11:25.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.242+0000 7f4c19d2b800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c58e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.242+0000 7f4c19d2b800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56080c5666f0 con 0x56080c61a010 2024-04-23T19:11:25.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4c15aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c58e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4c15aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c58e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37778/0 (socket says 172.21.15.63:37778) 2024-04-23T19:11:25.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4c15aef640 1 -- 172.21.15.63:0/1726187809 learned_addr learned my addr 172.21.15.63:0/1726187809 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4c15aef640 1 -- 172.21.15.63:0/1726187809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c70f590 con 0x56080c61a010 2024-04-23T19:11:25.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4c15aef640 1 --2- 172.21.15.63:0/1726187809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c58e0e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4c08055040 tx=0x7f4c0804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ae8dcad706a53b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4b03e07640 1 -- 172.21.15.63:0/1726187809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c08057040 con 0x56080c61a010 2024-04-23T19:11:25.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4b03e07640 1 -- 172.21.15.63:0/1726187809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08052990 con 0x56080c61a010 2024-04-23T19:11:25.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.243+0000 7f4b03e07640 1 -- 172.21.15.63:0/1726187809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c08052c70 con 0x56080c61a010 2024-04-23T19:11:25.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1726187809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 msgr2=0x56080c58e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/1726187809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c58e0e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4c08055040 tx=0x7f4c0804c000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1726187809 shutdown_connections 2024-04-23T19:11:25.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/1726187809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c58e0e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1726187809 >> 172.21.15.63:0/1726187809 conn(0x56080c615630 msgr2=0x56080c707080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1726187809 shutdown_connections 2024-04-23T19:11:25.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 -- 172.21.15.63:0/1726187809 wait complete. 2024-04-23T19:11:25.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.244+0000 7f4c19d2b800 1 Processor -- start 2024-04-23T19:11:25.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.245+0000 7f4c19d2b800 1 -- start start 2024-04-23T19:11:25.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.245+0000 7f4c19d2b800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c714260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.245+0000 7f4c19d2b800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56080c6236e0 con 0x56080c61a010 2024-04-23T19:11:25.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.245+0000 7f4c15aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c714260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.245+0000 7f4c15aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c714260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37780/0 (socket says 172.21.15.63:37780) 2024-04-23T19:11:25.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.245+0000 7f4c15aef640 1 -- 172.21.15.63:0/2169192490 learned_addr learned my addr 172.21.15.63:0/2169192490 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.245+0000 7f4c15aef640 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c7147a0 con 0x56080c61a010 2024-04-23T19:11:25.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.246+0000 7f4c15aef640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c714260 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800a470 tx=0x7f4c08003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.246+0000 7f4b02604640 1 -- 172.21.15.63:0/2169192490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c08073070 con 0x56080c61a010 2024-04-23T19:11:25.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.246+0000 7f4b02604640 1 -- 172.21.15.63:0/2169192490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0805d570 con 0x56080c61a010 2024-04-23T19:11:25.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.246+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56080c714a30 con 0x56080c61a010 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.246+0000 7f4b02604640 1 -- 172.21.15.63:0/2169192490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c08057070 con 0x56080c61a010 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.246+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080c7176e0 con 0x56080c61a010 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.247+0000 7f4b02604640 1 -- 172.21.15.63:0/2169192490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4c08076070 con 0x56080c61a010 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.247+0000 7f4b02604640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4afc036e60 0x7f4afc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.247+0000 7f4b02604640 1 -- 172.21.15.63:0/2169192490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4c0808e4d0 con 0x56080c61a010 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.247+0000 7f4c152ee640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4afc036e60 0x7f4afc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.247+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c61b270 0x56080c71b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.247+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c71ba70 con 0x56080c61b270 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.247+0000 7f4c162f0640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c61b270 0x56080c71b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.248+0000 7f4c152ee640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4afc036e60 0x7f4afc039320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c000f60 tx=0x7f4c0c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.248+0000 7f4c162f0640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c61b270 0x56080c71b4a0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.248+0000 7f4c162f0640 1 -- 172.21.15.63:0/2169192490 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c1000a4a0 con 0x56080c61b270 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.248+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c718a50 con 0x56080c61b270 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.248+0000 7f4c162f0640 1 -- 172.21.15.63:0/2169192490 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4c1000a4a0 con 0x56080c61b270 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.248+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c71d870 con 0x56080c61b270 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.249+0000 7f4c162f0640 1 -- 172.21.15.63:0/2169192490 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4c1000a4a0 con 0x56080c61b270 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.249+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c71cbb0 con 0x56080c61b270 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.249+0000 7f4c162f0640 1 -- 172.21.15.63:0/2169192490 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4c1000a4a0 con 0x56080c61b270 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.249+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56080c71ec20 0x56080c721010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.249+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c721550 con 0x56080c71ec20 2024-04-23T19:11:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.249+0000 7f4c15aef640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56080c71ec20 0x56080c721010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.250+0000 7f4c15aef640 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56080c71ec20 0x56080c721010 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.250+0000 7f4c15aef640 1 -- 172.21.15.63:0/2169192490 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4c08075030 con 0x56080c71ec20 2024-04-23T19:11:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.250+0000 7f4c19d2b800 1 Processor -- start 2024-04-23T19:11:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.250+0000 7f4c19d2b800 1 -- start start 2024-04-23T19:11:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.251+0000 7f4c19d2b800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c72f990 0x56080c72fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.251+0000 7f4c19d2b800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56080c50ce30 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.251+0000 7f4c162f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c72f990 0x56080c72fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.251+0000 7f4c162f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c72f990 0x56080c72fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37796/0 (socket says 172.21.15.63:37796) 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.251+0000 7f4c162f0640 1 -- 172.21.15.63:0/2239139176 learned_addr learned my addr 172.21.15.63:0/2239139176 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.251+0000 7f4c162f0640 1 -- 172.21.15.63:0/2239139176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c7302f0 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.251+0000 7f4c162f0640 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c72f990 0x56080c72fdb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x56080c71be50 tx=0x7f4c100178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.252+0000 7f4afb7fe640 1 -- 172.21.15.63:0/2239139176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c10013440 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.252+0000 7f4afb7fe640 1 -- 172.21.15.63:0/2239139176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c100135e0 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.252+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56080c730d60 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.252+0000 7f4afb7fe640 1 -- 172.21.15.63:0/2239139176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c100138c0 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.252+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080c731260 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4afb7fe640 1 -- 172.21.15.63:0/2239139176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4c10011040 con 0x56080c72f990 2024-04-23T19:11:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4afb7fe640 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4af0036e60 0x7f4af0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4afb7fe640 1 -- 172.21.15.63:0/2239139176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4c1001d070 con 0x56080c72f990 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4c15aef640 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4af0036e60 0x7f4af0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4c19d2b800 1 Processor -- start 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4c19d2b800 1 -- start start 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4c15aef640 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4af0036e60 0x7f4af0039320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56080c71cf50 tx=0x7f4c08006f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4c19d2b800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c73d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.253+0000 7f4c19d2b800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56080c70f820 con 0x56080c73cd40 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4c162f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c73d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4c162f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c73d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37812/0 (socket says 172.21.15.63:37812) 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4c162f0640 1 -- 172.21.15.63:0/4171544331 learned_addr learned my addr 172.21.15.63:0/4171544331 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4c162f0640 1 -- 172.21.15.63:0/4171544331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c73d730 con 0x56080c73cd40 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4c162f0640 1 --2- 172.21.15.63:0/4171544331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c73d160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4c1004cd50 tx=0x7f4c1002bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e44c1968f077e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4af97fa640 1 -- 172.21.15.63:0/4171544331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c1002a480 con 0x56080c73cd40 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4af97fa640 1 -- 172.21.15.63:0/4171544331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c10012ca0 con 0x56080c73cd40 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.254+0000 7f4af97fa640 1 -- 172.21.15.63:0/4171544331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c10021bf0 con 0x56080c73cd40 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4171544331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 msgr2=0x56080c73d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4171544331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c73d160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4c1004cd50 tx=0x7f4c1002bae0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4171544331 shutdown_connections 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4171544331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c73d160 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4171544331 >> 172.21.15.63:0/4171544331 conn(0x56080c739010 msgr2=0x56080c737070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4171544331 shutdown_connections 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4171544331 wait complete. 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 Processor -- start 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 -- start start 2024-04-23T19:11:25.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c744200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.255+0000 7f4c19d2b800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56080c73dec0 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4c162f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c744200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4c162f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c744200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37820/0 (socket says 172.21.15.63:37820) 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 learned_addr learned my addr 172.21.15.63:0/4266796790 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c744740 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4c162f0640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c744200 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4c10015740 tx=0x7f4c1004cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4ae77fe640 1 -- 172.21.15.63:0/4266796790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c10013d00 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4ae77fe640 1 -- 172.21.15.63:0/4266796790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1002b3a0 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56080c7449d0 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4ae77fe640 1 -- 172.21.15.63:0/4266796790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c1002b6a0 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.256+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080c747680 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.257+0000 7f4ae77fe640 1 -- 172.21.15.63:0/4266796790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4c1004ebb0 con 0x56080c73cd40 2024-04-23T19:11:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4ae77fe640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4ae0036e60 0x7f4ae0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4ae77fe640 1 -- 172.21.15.63:0/4266796790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4c1000e040 con 0x56080c73cd40 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4c15aef640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4ae0036e60 0x7f4ae0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c74ff30 0x56080c7523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c752b10 con 0x56080c74ff30 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4c152ee640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c74ff30 0x56080c7523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4c15aef640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4ae0036e60 0x7f4ae0039320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006e60 tx=0x7f4c08089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.258+0000 7f4c152ee640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c74ff30 0x56080c7523c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c0c009050 con 0x56080c74ff30 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 20 realm 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c74f440 con 0x56080c74ff30 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c0c009050 con 0x56080c74ff30 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c7535f0 con 0x56080c74ff30 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c0c009050 con 0x56080c74ff30 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.259+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c753ec0 con 0x56080c74ff30 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4c0c009050 con 0x56080c74ff30 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c19d2b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c754260 con 0x56080c74ff30 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4c0c009050 con 0x56080c74ff30 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c19d2b800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c19d2b800 20 searching for the correct realm 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56080c756140 0x56080c758530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c162f0640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56080c756140 0x56080c758530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.260+0000 7f4c162f0640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56080c756140 0x56080c758530 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c162f0640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4c10059040 0x7f4c10046680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c10059040 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c15aef640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4c10059040 0x7f4c10046680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c15aef640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4c10059040 0x7f4c10046680 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c15aef640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4c0809c040 0x7f4c0805da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.261+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.262+0000 7f4c152ee640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4c0809c040 0x7f4c0805da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.262+0000 7f4c152ee640 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4c0809c040 0x7f4c0805da50 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.263+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c009050 con 0x7f4c0809c040 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.263+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.263+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c009050 con 0x56080c74ff30 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.263+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.263+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.263+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c10059040 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.264+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.265+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.265+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.265+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.265+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.265+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.265+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c10059040 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.266+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.266+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.266+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.266+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.266+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.266+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.267+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c10059040 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.268+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.269+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.269+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c10059040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.269+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.269+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.269+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.269+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.270+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.270+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c10059040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.270+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.270+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c756140 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.271+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.271+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c0809c040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.271+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.271+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x56080c74ff30 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.271+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.271+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56080c758be0 con 0x7f4c10059040 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c754c10 con 0x56080c74ff30 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c7554b0 con 0x56080c74ff30 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.272+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c754850 con 0x7f4c0809c040 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 zone default found 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 4 Realm: () 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:25.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 zonegroup default 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c759040 con 0x7f4c10059040 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c75b270 con 0x56080c756140 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.273+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c75b610 con 0x56080c756140 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.274+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.274+0000 7f4c19d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.274+0000 7f4c19d2b800 20 started sync module instance, tier type = 2024-04-23T19:11:25.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.274+0000 7f4c19d2b800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.274+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75a700 con 0x7f4c10059040 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.275+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.275+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94592568765168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75a360 con 0x7f4c10059040 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.275+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75c4d0 con 0x56080c74ff30 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.276+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94592568765168] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.277+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.277+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94592568770592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75c8f0 con 0x56080c74ff30 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.277+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75d2b0 con 0x56080c756140 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.278+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94592568770592] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.278+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.278+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94592568771648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75c860 con 0x56080c756140 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.278+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75e170 con 0x7f4c10059040 2024-04-23T19:11:25.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.279+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94592568771648] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.279+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.279+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94592568777920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75e560 con 0x7f4c10059040 2024-04-23T19:11:25.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.279+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75ef80 con 0x56080c74ff30 2024-04-23T19:11:25.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.280+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94592568777920] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.280+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.281+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94592568781424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75f370 con 0x56080c74ff30 2024-04-23T19:11:25.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.281+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75fdb0 con 0x7f4c0809c040 2024-04-23T19:11:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.282+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94592568781424] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.282+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.282+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94592568785024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75fd70 con 0x7f4c0809c040 2024-04-23T19:11:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.282+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c7607c0 con 0x7f4c0809c040 2024-04-23T19:11:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.283+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(56 notify.6 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.283+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94592568787728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c760be0 con 0x7f4c0809c040 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.283+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c7616c0 con 0x7f4c0809c040 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.283+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94592568785024] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94592568787728] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94592568776352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c760730 con 0x7f4c0809c040 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 20 add_watcher() i=0 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 20 add_watcher() i=1 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 20 add_watcher() i=2 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 20 add_watcher() i=3 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 20 add_watcher() i=4 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 20 add_watcher() i=5 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.284+0000 7f4c19d2b800 20 add_watcher() i=6 2024-04-23T19:11:25.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.286+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94592568776352] v32'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.286+0000 7f4c19d2b800 20 add_watcher() i=7 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.286+0000 7f4c19d2b800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.286+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c7647c0 con 0x56080c74ff30 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.286+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.286+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94592568791504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c761c40 con 0x56080c74ff30 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.287+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94592568791504] v32'196 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.287+0000 7f4ac77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c19d2b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c19d2b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c19d2b800 5 note: GC not initialized 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4ac67fc640 20 reqs_thread_entry: start 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c19d2b800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:25.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c19d2b800 20 Filter name: none 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4a37fff640 20 reqs_thread_entry: start 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c7980d0 con 0x56080c74ff30 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.288+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c7980d0 con 0x7f4c10059040 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c798090 con 0x56080c74ff30 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c798090 con 0x56080c756140 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c798620 con 0x56080c756140 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.289+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c798620 con 0x56080c756140 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c7985d0 con 0x56080c74ff30 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c7985d0 con 0x56080c74ff30 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c798580 con 0x56080c756140 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.290+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c798580 con 0x56080c756140 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.291+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.291+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c798580 con 0x7f4c0809c040 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.291+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(72 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.291+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c74ff30 2024-04-23T19:11:25.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c10059040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c756140 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c0809c040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.292+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c0809c040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.293+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.293+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c756140 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.293+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.293+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c0809c040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.293+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(79 usage.17 [call out=185b] v0'0 uv5 ondisk = 0) ==== 152+0+185 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.293+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c74ff30 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c756140 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c0809c040 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c10059040 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.294+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c0809c040 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c74ff30 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c74ff30 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c756140 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.295+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c74ff30 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c74ff30 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c756140 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c756140 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.296+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x56080c756140 2024-04-23T19:11:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.297+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.297+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56080c799bf0 con 0x7f4c0809c040 2024-04-23T19:11:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.297+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.298+0000 7f4ac77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.298+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94592568791504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c74d420 con 0x56080c74ff30 2024-04-23T19:11:25.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.299+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94592568791504] v32'197 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.299+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94592568765168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c7611d0 con 0x7f4c10059040 2024-04-23T19:11:25.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.300+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94592568765168] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.300+0000 7f4c19d2b800 20 remove_watcher() i=0 2024-04-23T19:11:25.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.300+0000 7f4c19d2b800 2 removed watcher, disabling cache 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.300+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94592568770592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75aaf0 con 0x56080c74ff30 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.302+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94592568770592] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.302+0000 7f4c19d2b800 20 remove_watcher() i=1 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.302+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94592568771648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75aaf0 con 0x56080c756140 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.303+0000 7f4c162f0640 1 -- 172.21.15.63:0/4266796790 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94592568771648] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c1003b020 con 0x56080c756140 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.303+0000 7f4c19d2b800 20 remove_watcher() i=2 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.303+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94592568777920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75aaf0 con 0x7f4c10059040 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.304+0000 7f4c15aef640 1 -- 172.21.15.63:0/4266796790 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94592568777920] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c08075030 con 0x7f4c10059040 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.304+0000 7f4c19d2b800 20 remove_watcher() i=3 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.304+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94592568781424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75aaf0 con 0x56080c74ff30 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.305+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94592568781424] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x56080c74ff30 2024-04-23T19:11:25.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.305+0000 7f4c19d2b800 20 remove_watcher() i=4 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.305+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94592568785024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75ea70 con 0x7f4c0809c040 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.306+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94592568785024] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.306+0000 7f4c19d2b800 20 remove_watcher() i=5 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.306+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94592568787728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c75aaf0 con 0x7f4c0809c040 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.308+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94592568787728] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.308+0000 7f4c19d2b800 20 remove_watcher() i=6 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.308+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94592568776352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56080c760310 con 0x7f4c0809c040 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94592568776352] v32'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c0c02a040 con 0x7f4c0809c040 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 20 remove_watcher() i=7 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4c0809c040 msgr2=0x7f4c0805da50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4c0809c040 0x7f4c0805da50 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c74ff30 msgr2=0x56080c7523c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c74ff30 0x56080c7523c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4c10059040 msgr2=0x7f4c10046680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4c10059040 0x7f4c10046680 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56080c756140 msgr2=0x56080c758530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56080c756140 0x56080c758530 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4ae0036e60 msgr2=0x7f4ae0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4ae0036e60 0x7f4ae0039320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006e60 tx=0x7f4c08089000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 msgr2=0x56080c744200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c152ee640 1 -- 172.21.15.63:0/4266796790 reap_dead start 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c744200 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4c10015740 tx=0x7f4c1004cce0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 shutdown_connections 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/4266796790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c73cd40 0x56080c744200 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 >> 172.21.15.63:0/4266796790 conn(0x56080c739010 msgr2=0x56080c736670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 shutdown_connections 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/4266796790 wait complete. 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.309+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4af0036e60 msgr2=0x7f4af0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4af0036e60 0x7f4af0039320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56080c71cf50 tx=0x7f4c08006f50 comp rx=0 tx=0).stop 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c72f990 msgr2=0x56080c72fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c72f990 0x56080c72fdb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x56080c71be50 tx=0x7f4c100178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 shutdown_connections 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4af0036e60 0x7f4af0039320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2239139176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c72f990 0x56080c72fdb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 >> 172.21.15.63:0/2239139176 conn(0x56080c723a30 msgr2=0x56080c723e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 shutdown_connections 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2239139176 wait complete. 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56080c71ec20 msgr2=0x56080c721010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56080c71ec20 0x56080c721010 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c61b270 msgr2=0x56080c71b4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c61b270 0x56080c71b4a0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4afc036e60 msgr2=0x7f4afc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4afc036e60 0x7f4afc039320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c000f60 tx=0x7f4c0c0012b0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 msgr2=0x56080c714260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c714260 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800a470 tx=0x7f4c08003020 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 shutdown_connections 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x56080c71ec20 0x56080c721010 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x56080c61b270 0x56080c71b4a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4afc036e60 0x7f4afc039320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 --2- 172.21.15.63:0/2169192490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56080c61a010 0x56080c714260 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 >> 172.21.15.63:0/2169192490 conn(0x56080c615630 msgr2=0x56080c705880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 shutdown_connections 2024-04-23T19:11:25.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.310+0000 7f4c19d2b800 1 -- 172.21.15.63:0/2169192490 wait complete. 2024-04-23T19:11:25.320 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"anonymous","categories":[],"s3select":{}}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"s3select":{}},{"bucket":"myfoo2","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"anonymous","categories":[],"total":{"bytes_sent":0,"bytes_received":0,"ops":0,"successful_ops":0,"bytes_processed":0,"bytes_returned":0}},{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-04-23T19:11:25.320 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'anonymous', 'categories': [], 's3select': {}}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:11:25.320 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-23T19:11:25.321 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-23T19:11:25.321 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-23T19:11:25.363 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:25.363 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:25.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.362+0000 7f9dbd00c800 1 Processor -- start 2024-04-23T19:11:25.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.362+0000 7f9dbd00c800 1 -- start start 2024-04-23T19:11:25.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.362+0000 7f9dbd00c800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5598c8845a10 0x5598c8845e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.362+0000 7f9dbd00c800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5598c86f97f0 con 0x5598c8846370 2024-04-23T19:11:25.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.362+0000 7f9dbd00c800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5598c86e8e20 con 0x5598c8845a10 2024-04-23T19:11:25.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.363+0000 7f9dbab0b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5598c8845a10 0x5598c8845e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.363+0000 7f9dba30a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5598c8846370 0x5598c8846820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:58962/0 (socket says 172.21.15.63:58962) 2024-04-23T19:11:25.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.363+0000 7f9dba30a640 1 -- 172.21.15.63:0/3703034465 learned_addr learned my addr 172.21.15.63:0/3703034465 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:25.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.363+0000 7f9dbab0b640 1 -- 172.21.15.63:0/3703034465 >> v1:172.21.15.16:6789/0 conn(0x5598c8846370 legacy=0x5598c8846820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.363+0000 7f9dbab0b640 1 -- 172.21.15.63:0/3703034465 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c8707b50 con 0x5598c8845a10 2024-04-23T19:11:25.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.365+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/3703034465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c8845e30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00a470 tx=0x7f9dac01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650057695ed5af32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.365+0000 7f9db9b09640 1 -- 172.21.15.63:0/3703034465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dac057070 con 0x5598c8845a10 2024-04-23T19:11:25.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.365+0000 7f9db9b09640 1 -- 172.21.15.63:0/3703034465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac04adc0 con 0x5598c8845a10 2024-04-23T19:11:25.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.365+0000 7f9db9b09640 1 -- 172.21.15.63:0/3703034465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dac052590 con 0x5598c8845a10 2024-04-23T19:11:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.366+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3703034465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 msgr2=0x5598c8845e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.366+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/3703034465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c8845e30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00a470 tx=0x7f9dac01ce90 comp rx=0 tx=0).stop 2024-04-23T19:11:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.366+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3703034465 shutdown_connections 2024-04-23T19:11:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.366+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/3703034465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c8845e30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.366+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3703034465 >> 172.21.15.63:0/3703034465 conn(0x5598c88415f0 msgr2=0x5598c8843a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.366+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3703034465 shutdown_connections 2024-04-23T19:11:25.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.366+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3703034465 wait complete. 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.378+0000 7f9dbd00c800 1 Processor -- start 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.378+0000 7f9dbd00c800 1 -- start start 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.379+0000 7f9dbd00c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c893a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.379+0000 7f9dbd00c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598c87926e0 con 0x5598c8845a10 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.379+0000 7f9dbab0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c893a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.379+0000 7f9dbab0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c893a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37842/0 (socket says 172.21.15.63:37842) 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.379+0000 7f9dbab0b640 1 -- 172.21.15.63:0/6906758 learned_addr learned my addr 172.21.15.63:0/6906758 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.379+0000 7f9dbab0b640 1 -- 172.21.15.63:0/6906758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c893ac20 con 0x5598c8845a10 2024-04-23T19:11:25.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.379+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/6906758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c893a650 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9dac055040 tx=0x7f9dac04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175852c171e14a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9ca8e09640 1 -- 172.21.15.63:0/6906758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dac057040 con 0x5598c8845a10 2024-04-23T19:11:25.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9ca8e09640 1 -- 172.21.15.63:0/6906758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac063070 con 0x5598c8845a10 2024-04-23T19:11:25.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9ca8e09640 1 -- 172.21.15.63:0/6906758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dac05d790 con 0x5598c8845a10 2024-04-23T19:11:25.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9dbd00c800 1 -- 172.21.15.63:0/6906758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 msgr2=0x5598c893a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/6906758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c893a650 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9dac055040 tx=0x7f9dac04c000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9dbd00c800 1 -- 172.21.15.63:0/6906758 shutdown_connections 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/6906758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c893a650 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9dbd00c800 1 -- 172.21.15.63:0/6906758 >> 172.21.15.63:0/6906758 conn(0x5598c88415f0 msgr2=0x5598c8935f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9dbd00c800 1 -- 172.21.15.63:0/6906758 shutdown_connections 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.380+0000 7f9dbd00c800 1 -- 172.21.15.63:0/6906758 wait complete. 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbd00c800 1 Processor -- start 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbd00c800 1 -- start start 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbd00c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c89421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbd00c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598c88471d0 con 0x5598c8845a10 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbab0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c89421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbab0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c89421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37856/0 (socket says 172.21.15.63:37856) 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbab0b640 1 -- 172.21.15.63:0/4124012728 learned_addr learned my addr 172.21.15.63:0/4124012728 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.381+0000 7f9dbab0b640 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c8942710 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.382+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c89421d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9dac060040 tx=0x7f9dac002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.382+0000 7f9ca7606640 1 -- 172.21.15.63:0/4124012728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dac057050 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.382+0000 7f9ca7606640 1 -- 172.21.15.63:0/4124012728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac05dcd0 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.382+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598c89429a0 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.382+0000 7f9ca7606640 1 -- 172.21.15.63:0/4124012728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dac052440 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.382+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c8945600 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9ca7606640 1 -- 172.21.15.63:0/4124012728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9dac052650 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9ca7606640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ca0036e60 0x7f9ca0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9ca7606640 1 -- 172.21.15.63:0/4124012728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9dac08e430 con 0x5598c8845a10 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9dba30a640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ca0036e60 0x7f9ca0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c8846370 0x5598c89493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8949940 con 0x5598c8846370 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c8846370 0x5598c89493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9dba30a640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ca0036e60 0x7f9ca0039320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9db0000f60 tx=0x7f9db00012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.383+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c8846370 0x5598c89493c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.384+0000 7f9dbb30c640 1 -- 172.21.15.63:0/4124012728 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8846370 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.384+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8946970 con 0x5598c8846370 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.384+0000 7f9dbb30c640 1 -- 172.21.15.63:0/4124012728 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8846370 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.384+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c894b790 con 0x5598c8846370 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.384+0000 7f9dbb30c640 1 -- 172.21.15.63:0/4124012728 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8846370 2024-04-23T19:11:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.384+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c894aad0 con 0x5598c8846370 2024-04-23T19:11:25.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.385+0000 7f9dbb30c640 1 -- 172.21.15.63:0/4124012728 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8846370 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.385+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5598c894cb40 0x5598c894ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.385+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c894f470 con 0x5598c894cb40 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.385+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5598c894cb40 0x5598c894ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.385+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5598c894cb40 0x5598c894ef30 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbab0b640 1 -- 172.21.15.63:0/4124012728 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9dac078070 con 0x5598c894cb40 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbd00c800 1 Processor -- start 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbd00c800 1 -- start start 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbd00c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c895d840 0x5598c895dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbd00c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598c8738e30 con 0x5598c895d840 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbb30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c895d840 0x5598c895dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbb30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c895d840 0x5598c895dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37860/0 (socket says 172.21.15.63:37860) 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.386+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1739551871 learned_addr learned my addr 172.21.15.63:0/1739551871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.387+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1739551871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c895e1a0 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.387+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c895d840 0x5598c895dc60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x5598c8949d20 tx=0x7f9db40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.387+0000 7f9ca4e01640 1 -- 172.21.15.63:0/1739551871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db4013440 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.387+0000 7f9ca4e01640 1 -- 172.21.15.63:0/1739551871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db40135e0 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.387+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598c895ec10 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.387+0000 7f9ca4e01640 1 -- 172.21.15.63:0/1739551871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db40138c0 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.387+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c895f110 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.388+0000 7f9ca4e01640 1 -- 172.21.15.63:0/1739551871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9db4011040 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.388+0000 7f9ca4e01640 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c94036e60 0x7f9c94039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.388+0000 7f9ca4e01640 1 -- 172.21.15.63:0/1739551871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9db401d070 con 0x5598c895d840 2024-04-23T19:11:25.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.388+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c94036e60 0x7f9c94039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.388+0000 7f9dbd00c800 1 Processor -- start 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.388+0000 7f9dbd00c800 1 -- start start 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.388+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c94036e60 0x7f9c94039320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5598c894ae70 tx=0x7f9dac07d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9dbd00c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c896b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9dbd00c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598c893aee0 con 0x5598c896abf0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9dbb30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c896b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9dbb30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c896b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37864/0 (socket says 172.21.15.63:37864) 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9dbb30c640 1 -- 172.21.15.63:0/3592075761 learned_addr learned my addr 172.21.15.63:0/3592075761 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9dbb30c640 1 -- 172.21.15.63:0/3592075761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c896b5e0 con 0x5598c896abf0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/3592075761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c896b010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9db404cd50 tx=0x7f9db404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b51408f99c768 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9c9e7fc640 1 -- 172.21.15.63:0/3592075761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db4015560 con 0x5598c896abf0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9c9e7fc640 1 -- 172.21.15.63:0/3592075761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db402a350 con 0x5598c896abf0 2024-04-23T19:11:25.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.389+0000 7f9c9e7fc640 1 -- 172.21.15.63:0/3592075761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db4012ca0 con 0x5598c896abf0 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3592075761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 msgr2=0x5598c896b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/3592075761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c896b010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9db404cd50 tx=0x7f9db404d000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3592075761 shutdown_connections 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/3592075761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c896b010 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3592075761 >> 172.21.15.63:0/3592075761 conn(0x5598c8966ec0 msgr2=0x5598c8964f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3592075761 shutdown_connections 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 -- 172.21.15.63:0/3592075761 wait complete. 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 Processor -- start 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 -- start start 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c89720a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.390+0000 7f9dbd00c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598c896bd70 con 0x5598c896abf0 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9dbb30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c89720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9dbb30c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c89720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37880/0 (socket says 172.21.15.63:37880) 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 learned_addr learned my addr 172.21.15.63:0/1357459797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c89725e0 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c89720a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9db404bc20 tx=0x7f9db404cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9c9cff9640 1 -- 172.21.15.63:0/1357459797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db402a8e0 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9c9cff9640 1 -- 172.21.15.63:0/1357459797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db4013af0 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598c8972870 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9c9cff9640 1 -- 172.21.15.63:0/1357459797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db4013e50 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.391+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c8975520 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.392+0000 7f9c9cff9640 1 -- 172.21.15.63:0/1357459797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9db402b3a0 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9c9cff9640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c84037050 0x7f9c84039510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9c9cff9640 1 -- 172.21.15.63:0/1357459797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9db400e040 con 0x5598c896abf0 2024-04-23T19:11:25.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c84037050 0x7f9c84039510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c897dc10 0x5598c89800a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89807f0 con 0x5598c897dc10 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dba30a640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c897dc10 0x5598c89800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c84037050 0x7f9c84039510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00a470 tx=0x7f9dac08e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dba30a640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c897dc10 0x5598c89800a0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9db0009050 con 0x5598c897dc10 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbd00c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbd00c800 20 realm 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.393+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c897d120 con 0x5598c897dc10 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9db0009050 con 0x5598c897dc10 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89812d0 con 0x5598c897dc10 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9db0009050 con 0x5598c897dc10 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8981ba0 con 0x5598c897dc10 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9db0009050 con 0x5598c897dc10 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8981f40 con 0x5598c897dc10 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9db0009050 con 0x5598c897dc10 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:25.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.394+0000 7f9dbd00c800 20 searching for the correct realm 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5598c8983e20 0x5598c8986210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5598c8983e20 0x5598c8986210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5598c8983e20 0x5598c8986210 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbb30c640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9db402c7b0 0x7f9db4027060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9db402c7b0 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.395+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9db402c7b0 0x7f9db4027060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.396+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9db402c7b0 0x7f9db4027060 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.396+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.396+0000 7f9dbab0b640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9dac09b040 0x7f9dac05b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.396+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.396+0000 7f9dba30a640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9dac09b040 0x7f9dac05b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.396+0000 7f9dba30a640 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9dac09b040 0x7f9dac05b370 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.397+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db0009050 con 0x7f9dac09b040 2024-04-23T19:11:25.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.397+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.397+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db0009050 con 0x5598c897dc10 2024-04-23T19:11:25.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.397+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.397+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.397+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9db402c7b0 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.398+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.399+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.399+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.399+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.399+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.399+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.399+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9db402c7b0 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.400+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.401+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.401+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.401+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.401+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.401+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.401+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.401+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.402+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.402+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.402+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.402+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.402+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9db402c7b0 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.402+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.402+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9db402c7b0 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.403+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9db402c7b0 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c8983e20 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9dac09b040 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x5598c897dc10 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.404+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5598c89868c0 con 0x7f9db402c7b0 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89828c0 con 0x5598c897dc10 2024-04-23T19:11:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8983160 con 0x5598c897dc10 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.405+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8982500 con 0x7f9dac09b040 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 zone default found 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 4 Realm: () 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 zonegroup default 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8986d20 con 0x7f9db402c7b0 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8988f50 con 0x5598c8983e20 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89892f0 con 0x5598c8983e20 2024-04-23T19:11:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 started sync module instance, tier type = 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.406+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c8988390 con 0x7f9db402c7b0 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.408+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.408+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94114688829360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c8987ff0 con 0x7f9db402c7b0 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.408+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898a030 con 0x5598c897dc10 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.409+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94114688829360] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.409+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.409+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94114688833168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898a450 con 0x5598c897dc10 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.409+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898ae80 con 0x5598c8983e20 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.410+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94114688833168] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.410+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.410+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94114688834224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898b2a0 con 0x5598c8983e20 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.410+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898bd20 con 0x7f9db402c7b0 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.411+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94114688834224] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.411+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.411+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94114688841744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898bce0 con 0x7f9db402c7b0 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.411+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898c7a0 con 0x5598c897dc10 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.412+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94114688841744] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.412+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.412+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94114688844464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898c7a0 con 0x5598c897dc10 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.412+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898d210 con 0x7f9dac09b040 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.413+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.413+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94114688847104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898d1d0 con 0x7f9dac09b040 2024-04-23T19:11:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.413+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898e180 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.414+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94114688844464] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.414+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94114688847104] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.415+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.415+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94114688851120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898d480 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.415+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898e630 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94114688851120] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94114688838032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898e630 con 0x7f9dac09b040 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 20 add_watcher() i=0 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 20 add_watcher() i=1 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 20 add_watcher() i=2 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 20 add_watcher() i=3 2024-04-23T19:11:25.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 20 add_watcher() i=4 2024-04-23T19:11:25.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 20 add_watcher() i=5 2024-04-23T19:11:25.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.416+0000 7f9dbd00c800 20 add_watcher() i=6 2024-04-23T19:11:25.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.417+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94114688838032] v32'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.417+0000 7f9dbd00c800 20 add_watcher() i=7 2024-04-23T19:11:25.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.417+0000 7f9dbd00c800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:25.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.417+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c8991a30 con 0x5598c897dc10 2024-04-23T19:11:25.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.417+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.417+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94114688852816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c898ef80 con 0x5598c897dc10 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.418+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94114688852816] v32'198 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.418+0000 7f9c717e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.418+0000 7f9dbd00c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.418+0000 7f9dbd00c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.418+0000 7f9dbd00c800 5 note: GC not initialized 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.418+0000 7f9bebfff640 20 reqs_thread_entry: start 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9dbd00c800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9dbd00c800 20 Filter name: none 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9be9ffb640 20 reqs_thread_entry: start 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c5530 con 0x5598c897dc10 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c5530 con 0x7f9db402c7b0 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.419+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c897dc10 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c8983e20 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c8983e20 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c5530 con 0x5598c8983e20 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.420+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c897dc10 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c897dc10 2024-04-23T19:11:25.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c8983e20 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c8983e20 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.421+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x7f9dac09b040 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.422+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(72 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.422+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c70b0 con 0x5598c897dc10 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.422+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.422+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c70b0 con 0x7f9db402c7b0 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c70b0 con 0x5598c8983e20 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x7f9dac09b040 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x7f9dac09b040 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.423+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c8983e20 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.424+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.424+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x7f9dac09b040 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.424+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(79 usage.17 [call out=185b] v0'0 uv5 ondisk = 0) ==== 152+0+185 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.424+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c897dc10 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.424+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.424+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c8983e20 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.425+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.425+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x7f9dac09b040 2024-04-23T19:11:25.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.425+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.425+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x7f9db402c7b0 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.425+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.425+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x7f9dac09b040 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c897dc10 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c897dc10 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c8983e20 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.426+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c897dc10 2024-04-23T19:11:25.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c897dc10 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c8983e20 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c8983e20 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.427+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x5598c8983e20 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.428+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.428+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c7030 con 0x7f9dac09b040 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.428+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.429+0000 7f9c717e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.429+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94114688852816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c899acc0 con 0x5598c897dc10 2024-04-23T19:11:25.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.430+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94114688852816] v32'199 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.430+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94114688829360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x7f9db402c7b0 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.431+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94114688829360] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.431+0000 7f9dbd00c800 20 remove_watcher() i=0 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.431+0000 7f9dbd00c800 2 removed watcher, disabling cache 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.431+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94114688833168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c897dc10 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.432+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94114688833168] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.432+0000 7f9dbd00c800 20 remove_watcher() i=1 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.432+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94114688834224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c8983e20 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.433+0000 7f9dbb30c640 1 -- 172.21.15.63:0/1357459797 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94114688834224] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db400a4a0 con 0x5598c8983e20 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.433+0000 7f9dbd00c800 20 remove_watcher() i=2 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.433+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94114688841744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x7f9db402c7b0 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.434+0000 7f9dbab0b640 1 -- 172.21.15.63:0/1357459797 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94114688841744] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dac078070 con 0x7f9db402c7b0 2024-04-23T19:11:25.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.434+0000 7f9dbd00c800 20 remove_watcher() i=3 2024-04-23T19:11:25.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.434+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94114688844464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x5598c897dc10 2024-04-23T19:11:25.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.435+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94114688844464] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x5598c897dc10 2024-04-23T19:11:25.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.435+0000 7f9dbd00c800 20 remove_watcher() i=4 2024-04-23T19:11:25.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.435+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94114688847104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x7f9dac09b040 2024-04-23T19:11:25.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.436+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94114688847104] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.436+0000 7f9dbd00c800 20 remove_watcher() i=5 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.436+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94114688851120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x7f9dac09b040 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.437+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94114688851120] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.437+0000 7f9dbd00c800 20 remove_watcher() i=6 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.437+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94114688838032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5598c89c54f0 con 0x7f9dac09b040 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.438+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94114688838032] v32'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9db002a040 con 0x7f9dac09b040 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.438+0000 7f9dbd00c800 20 remove_watcher() i=7 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9dac09b040 msgr2=0x7f9dac05b370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9dac09b040 0x7f9dac05b370 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c897dc10 msgr2=0x5598c89800a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c897dc10 0x5598c89800a0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9db402c7b0 msgr2=0x7f9db4027060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9db402c7b0 0x7f9db4027060 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5598c8983e20 msgr2=0x5598c8986210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5598c8983e20 0x5598c8986210 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c84037050 msgr2=0x7f9c84039510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c84037050 0x7f9c84039510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00a470 tx=0x7f9dac08e040 comp rx=0 tx=0).stop 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 msgr2=0x5598c89720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1357459797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c896abf0 0x5598c89720a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9db404bc20 tx=0x7f9db404cce0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 reap_dead start 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dba30a640 1 -- 172.21.15.63:0/1357459797 reap_dead start 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 shutdown_connections 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 >> 172.21.15.63:0/1357459797 conn(0x5598c8966ec0 msgr2=0x5598c8964520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 shutdown_connections 2024-04-23T19:11:25.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1357459797 wait complete. 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c94036e60 msgr2=0x7f9c94039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c94036e60 0x7f9c94039320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5598c894ae70 tx=0x7f9dac07d040 comp rx=0 tx=0).stop 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c895d840 msgr2=0x5598c895dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c895d840 0x5598c895dc60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x5598c8949d20 tx=0x7f9db40178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 shutdown_connections 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9c94036e60 0x7f9c94039320 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/1739551871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c895d840 0x5598c895dc60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 >> 172.21.15.63:0/1739551871 conn(0x5598c89518e0 msgr2=0x5598c8951ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 shutdown_connections 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.439+0000 7f9dbd00c800 1 -- 172.21.15.63:0/1739551871 wait complete. 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5598c894cb40 msgr2=0x5598c894ef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5598c894cb40 0x5598c894ef30 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c8846370 msgr2=0x5598c89493c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c8846370 0x5598c89493c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ca0036e60 msgr2=0x7f9ca0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ca0036e60 0x7f9ca0039320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9db0000f60 tx=0x7f9db00012b0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 msgr2=0x5598c89421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c89421d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9dac060040 tx=0x7f9dac002fb0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 shutdown_connections 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5598c894cb40 0x5598c894ef30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5598c8846370 0x5598c89493c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ca0036e60 0x7f9ca0039320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 --2- 172.21.15.63:0/4124012728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598c8845a10 0x5598c89421d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 >> 172.21.15.63:0/4124012728 conn(0x5598c88415f0 msgr2=0x5598c89347d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 shutdown_connections 2024-04-23T19:11:25.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.440+0000 7f9dbd00c800 1 -- 172.21.15.63:0/4124012728 wait complete. 2024-04-23T19:11:25.453 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo2","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo3","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo4","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]}],"summary":[{"user":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}],"total":{"bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1,"bytes_processed":0,"bytes_returned":0}},{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34,"bytes_processed":0,"bytes_returned":0}}]} 2024-04-23T19:11:25.454 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo2', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo3', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo4', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:11:25.454 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-23T19:11:25.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-23T19:11:25.454 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-23T19:11:25.538 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:25.538 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae4b20f800 1 Processor -- start 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae4b20f800 1 -- start start 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae4b20f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5605c96dec00 0x5605c96dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae4b20f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5605c961b7f0 con 0x5605c96dd6a0 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae4b20f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5605c960ae20 con 0x5605c96dec00 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae46814640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5605c96dec00 0x5605c96dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae46814640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5605c96dec00 0x5605c96dd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37894/0 (socket says 172.21.15.63:37894) 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae46814640 1 -- 172.21.15.63:0/3087392549 learned_addr learned my addr 172.21.15.63:0/3087392549 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae46013640 1 --1- 172.21.15.63:0/3087392549 >> v1:172.21.15.16:6789/0 conn(0x5605c96dd6a0 0x5605c9771010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:58976/0 (socket says 172.21.15.63:58976) 2024-04-23T19:11:25.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae46814640 1 -- 172.21.15.63:0/3087392549 >> v1:172.21.15.16:6789/0 conn(0x5605c96dd6a0 legacy=0x5605c9771010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.538+0000 7fae46814640 1 -- 172.21.15.63:0/3087392549 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c9629b50 con 0x5605c96dec00 2024-04-23T19:11:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.540+0000 7fae46814640 1 --2- 172.21.15.63:0/3087392549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dec00 0x5605c96dd160 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fae3800a520 tx=0x7fae3801cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40f2b51af8fef88 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae45812640 1 -- 172.21.15.63:0/3087392549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae38057070 con 0x5605c96dec00 2024-04-23T19:11:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae45812640 1 -- 172.21.15.63:0/3087392549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3804ad80 con 0x5605c96dec00 2024-04-23T19:11:25.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae45812640 1 -- 172.21.15.63:0/3087392549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae38052590 con 0x5605c96dec00 2024-04-23T19:11:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae4b20f800 1 -- 172.21.15.63:0/3087392549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dec00 msgr2=0x5605c96dd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3087392549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dec00 0x5605c96dd160 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fae3800a520 tx=0x7fae3801cd50 comp rx=0 tx=0).stop 2024-04-23T19:11:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae4b20f800 1 -- 172.21.15.63:0/3087392549 shutdown_connections 2024-04-23T19:11:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3087392549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dec00 0x5605c96dd160 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae4b20f800 1 -- 172.21.15.63:0/3087392549 >> 172.21.15.63:0/3087392549 conn(0x5605c97647e0 msgr2=0x5605c9766c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae4b20f800 1 -- 172.21.15.63:0/3087392549 shutdown_connections 2024-04-23T19:11:25.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.541+0000 7fae4b20f800 1 -- 172.21.15.63:0/3087392549 wait complete. 2024-04-23T19:11:25.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.554+0000 7fae4b20f800 1 Processor -- start 2024-04-23T19:11:25.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.554+0000 7fae4b20f800 1 -- start start 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.554+0000 7fae4b20f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c985ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.554+0000 7fae4b20f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5605c96b46f0 con 0x5605c96dd6a0 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.554+0000 7fae46814640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c985ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.554+0000 7fae46814640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c985ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37896/0 (socket says 172.21.15.63:37896) 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.554+0000 7fae46814640 1 -- 172.21.15.63:0/849566173 learned_addr learned my addr 172.21.15.63:0/849566173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.555+0000 7fae46814640 1 -- 172.21.15.63:0/849566173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c985cfe0 con 0x5605c96dd6a0 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.555+0000 7fae46814640 1 --2- 172.21.15.63:0/849566173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c985ca10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fae38055040 tx=0x7fae3804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18adc4668dbb9418 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.555+0000 7fad34e09640 1 -- 172.21.15.63:0/849566173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae38057040 con 0x5605c96dd6a0 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.555+0000 7fad34e09640 1 -- 172.21.15.63:0/849566173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae38063070 con 0x5605c96dd6a0 2024-04-23T19:11:25.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.555+0000 7fad34e09640 1 -- 172.21.15.63:0/849566173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3805d750 con 0x5605c96dd6a0 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 -- 172.21.15.63:0/849566173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 msgr2=0x5605c985ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 --2- 172.21.15.63:0/849566173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c985ca10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fae38055040 tx=0x7fae3804c000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 -- 172.21.15.63:0/849566173 shutdown_connections 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 --2- 172.21.15.63:0/849566173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c985ca10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 -- 172.21.15.63:0/849566173 >> 172.21.15.63:0/849566173 conn(0x5605c97647e0 msgr2=0x5605c9858330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 -- 172.21.15.63:0/849566173 shutdown_connections 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 -- 172.21.15.63:0/849566173 wait complete. 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 Processor -- start 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 -- start start 2024-04-23T19:11:25.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c9864470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.556+0000 7fae4b20f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5605c97749b0 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fae46814640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c9864470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fae46814640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c9864470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37902/0 (socket says 172.21.15.63:37902) 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fae46814640 1 -- 172.21.15.63:0/2405312723 learned_addr learned my addr 172.21.15.63:0/2405312723 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fae46814640 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c98649b0 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fae46814640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c9864470 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fae3804ad50 tx=0x7fae38002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fad33606640 1 -- 172.21.15.63:0/2405312723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae38057050 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fad33606640 1 -- 172.21.15.63:0/2405312723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3805da20 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fad33606640 1 -- 172.21.15.63:0/2405312723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3805dd40 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5605c9864c40 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.557+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c98678f0 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.558+0000 7fad33606640 1 -- 172.21.15.63:0/2405312723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fae38052440 con 0x5605c96dd6a0 2024-04-23T19:11:25.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.558+0000 7fad33606640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad2c036eb0 0x7fad2c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.558+0000 7fad33606640 1 -- 172.21.15.63:0/2405312723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fae3808e0b0 con 0x5605c96dd6a0 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.558+0000 7fae46013640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad2c036eb0 0x7fad2c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.558+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c96dec00 0x5605c986b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.559+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c986bbf0 con 0x5605c96dec00 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.559+0000 7fae47015640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c96dec00 0x5605c986b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.559+0000 7fae46013640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad2c036eb0 0x7fad2c039370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae3c001e40 tx=0x7fae3c0013a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.559+0000 7fae47015640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c96dec00 0x5605c986b6b0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.559+0000 7fae47015640 1 -- 172.21.15.63:0/2405312723 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae4000a4a0 con 0x5605c96dec00 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.559+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c9868c40 con 0x5605c96dec00 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.560+0000 7fae47015640 1 -- 172.21.15.63:0/2405312723 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fae4000a4a0 con 0x5605c96dec00 2024-04-23T19:11:25.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.560+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c986d9b0 con 0x5605c96dec00 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.560+0000 7fae47015640 1 -- 172.21.15.63:0/2405312723 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fae4000a4a0 con 0x5605c96dec00 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.560+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c986ccf0 con 0x5605c96dec00 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.560+0000 7fae47015640 1 -- 172.21.15.63:0/2405312723 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fae4000a4a0 con 0x5605c96dec00 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.560+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5605c986ed60 0x5605c9871150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.560+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c9871690 con 0x5605c986ed60 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.561+0000 7fae46814640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5605c986ed60 0x5605c9871150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.561+0000 7fae46814640 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5605c986ed60 0x5605c9871150 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.561+0000 7fae46814640 1 -- 172.21.15.63:0/2405312723 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fae38078070 con 0x5605c986ed60 2024-04-23T19:11:25.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae4b20f800 1 Processor -- start 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae4b20f800 1 -- start start 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae4b20f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c987fb00 0x5605c987ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae4b20f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5605c965ae30 con 0x5605c987fb00 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae47015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c987fb00 0x5605c987ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae47015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c987fb00 0x5605c987ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37914/0 (socket says 172.21.15.63:37914) 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae47015640 1 -- 172.21.15.63:0/1617034690 learned_addr learned my addr 172.21.15.63:0/1617034690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.562+0000 7fae47015640 1 -- 172.21.15.63:0/1617034690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c9880460 con 0x5605c987fb00 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.563+0000 7fae47015640 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c987fb00 0x5605c987ff20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5605c9865690 tx=0x7fae400178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.563+0000 7fad30e01640 1 -- 172.21.15.63:0/1617034690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae40013440 con 0x5605c987fb00 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.563+0000 7fad30e01640 1 -- 172.21.15.63:0/1617034690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae400135e0 con 0x5605c987fb00 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.563+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5605c9880ed0 con 0x5605c987fb00 2024-04-23T19:11:25.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.563+0000 7fad30e01640 1 -- 172.21.15.63:0/1617034690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae400138c0 con 0x5605c987fb00 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.563+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c98813d0 con 0x5605c987fb00 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fad30e01640 1 -- 172.21.15.63:0/1617034690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fae40011040 con 0x5605c987fb00 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fad30e01640 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad20036eb0 0x7fad20039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fad30e01640 1 -- 172.21.15.63:0/1617034690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fae4001d070 con 0x5605c987fb00 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fae46814640 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad20036eb0 0x7fad20039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fae4b20f800 1 Processor -- start 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fae4b20f800 1 -- start start 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fae46814640 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad20036eb0 0x7fad20039370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5605c986d090 tx=0x7fae3807d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fae4b20f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c988d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.564+0000 7fae4b20f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5605c985d2a0 con 0x5605c988ce20 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae47015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c988d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae47015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c988d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37916/0 (socket says 172.21.15.63:37916) 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae47015640 1 -- 172.21.15.63:0/2491014605 learned_addr learned my addr 172.21.15.63:0/2491014605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae47015640 1 -- 172.21.15.63:0/2491014605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c988d810 con 0x5605c988ce20 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae47015640 1 --2- 172.21.15.63:0/2491014605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c988d240 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fae4004cc80 tx=0x7fae400129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f158ff4ece630389 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fad2a7fc640 1 -- 172.21.15.63:0/2491014605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae40012dc0 con 0x5605c988ce20 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fad2a7fc640 1 -- 172.21.15.63:0/2491014605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae400123a0 con 0x5605c988ce20 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fad2a7fc640 1 -- 172.21.15.63:0/2491014605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae40013a60 con 0x5605c988ce20 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae4b20f800 1 -- 172.21.15.63:0/2491014605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 msgr2=0x5605c988d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2491014605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c988d240 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fae4004cc80 tx=0x7fae400129d0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae4b20f800 1 -- 172.21.15.63:0/2491014605 shutdown_connections 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2491014605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c988d240 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.565+0000 7fae4b20f800 1 -- 172.21.15.63:0/2491014605 >> 172.21.15.63:0/2491014605 conn(0x5605c9889140 msgr2=0x5605c98871a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae4b20f800 1 -- 172.21.15.63:0/2491014605 shutdown_connections 2024-04-23T19:11:25.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae4b20f800 1 -- 172.21.15.63:0/2491014605 wait complete. 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae4b20f800 1 Processor -- start 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae4b20f800 1 -- start start 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae4b20f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c9894470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae4b20f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5605c988df90 con 0x5605c988ce20 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae47015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c9894470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae47015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c9894470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37922/0 (socket says 172.21.15.63:37922) 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 learned_addr learned my addr 172.21.15.63:0/3433346014 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.566+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c98949b0 con 0x5605c988ce20 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.567+0000 7fae47015640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c9894470 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fae4004bb70 tx=0x7fae4004cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.567+0000 7fad28ff9640 1 -- 172.21.15.63:0/3433346014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae40012550 con 0x5605c988ce20 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.567+0000 7fad28ff9640 1 -- 172.21.15.63:0/3433346014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae40021a50 con 0x5605c988ce20 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.567+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5605c9894c40 con 0x5605c988ce20 2024-04-23T19:11:25.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.567+0000 7fad28ff9640 1 -- 172.21.15.63:0/3433346014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae40021d30 con 0x5605c988ce20 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.567+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c98978f0 con 0x5605c988ce20 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.567+0000 7fad28ff9640 1 -- 172.21.15.63:0/3433346014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fae4004ea30 con 0x5605c988ce20 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fad28ff9640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad10036e60 0x7fad10039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fad28ff9640 1 -- 172.21.15.63:0/3433346014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fae40033020 con 0x5605c988ce20 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fae46814640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad10036e60 0x7fad10039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c98a0070 0x5605c98a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a2c80 con 0x5605c98a0070 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fae46013640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c98a0070 0x5605c98a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fae46814640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad10036e60 0x7fad10039320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae3808c690 tx=0x7fae38086000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.568+0000 7fae46013640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c98a0070 0x5605c98a2530 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 20 realm 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c989f500 con 0x5605c98a0070 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a3750 con 0x5605c98a0070 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.569+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a3ca0 con 0x5605c98a0070 2024-04-23T19:11:25.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae4b20f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a4590 con 0x5605c98a0070 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae4b20f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae4b20f800 20 searching for the correct realm 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5605c98a6300 0x5605c98a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae47015640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5605c98a6300 0x5605c98a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.570+0000 7fae47015640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5605c98a6300 0x5605c98a86f0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40029070 con 0x5605c98a6300 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae47015640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fae40041040 0x7fae400089b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae40041040 2024-04-23T19:11:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae46814640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fae40041040 0x7fae400089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae46814640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fae40041040 0x7fae400089b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae46814640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fae3808c740 0x7fae38070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.571+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.572+0000 7fae46013640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fae3808c740 0x7fae38070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.572+0000 7fae46013640 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fae3808c740 0x7fae38070350 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.572+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.572+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.573+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae40041040 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.574+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.574+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.574+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40029070 con 0x5605c98a6300 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.574+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.574+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.574+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae40041040 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40029070 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40029070 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.575+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40029070 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.576+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.577+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.577+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.577+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.577+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.577+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.577+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae40041040 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae40041040 2024-04-23T19:11:25.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.578+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.579+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.579+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae40041040 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.579+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.579+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a6300 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.579+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.579+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae3808c740 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x5605c98a0070 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a8d70 con 0x7fae40041040 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a4ee0 con 0x5605c98a0070 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.580+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a5780 con 0x5605c98a0070 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a4b20 con 0x7fae3808c740 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 zone default found 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 4 Realm: () 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 zonegroup default 2024-04-23T19:11:25.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98a91d0 con 0x7fae40041040 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.581+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ab400 con 0x5605c98a6300 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae4b20f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ab7a0 con 0x5605c98a6300 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae4b20f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae4b20f800 20 started sync module instance, tier type = 2024-04-23T19:11:25.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae4b20f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.582+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aa840 con 0x7fae40041040 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.583+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.583+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94582856133680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ab050 con 0x7fae40041040 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.583+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ac4e0 con 0x5605c98a0070 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.584+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94582856133680] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.584+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.584+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94582856137536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ac8d0 con 0x5605c98a0070 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.584+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ad320 con 0x5605c98a6300 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.585+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94582856137536] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.585+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.585+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94582856142352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ad710 con 0x5605c98a6300 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.585+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ae130 con 0x7fae40041040 2024-04-23T19:11:25.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.586+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94582856142352] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.587+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.587+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94582856145952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98ae520 con 0x7fae40041040 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.587+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aef40 con 0x5605c98a0070 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.588+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.588+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94582856149552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aef40 con 0x5605c98a0070 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.588+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aff60 con 0x7fae3808c740 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.588+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94582856145952] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.589+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.589+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94582856153648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aff20 con 0x7fae3808c740 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.589+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94582856149552] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.589+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98af2e0 con 0x7fae3808c740 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.590+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94582856153648] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.590+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.590+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94582856138592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98af2e0 con 0x7fae3808c740 2024-04-23T19:11:25.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.590+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98b0e20 con 0x7fae3808c740 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94582856138592] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94582856156352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98b0d90 con 0x7fae3808c740 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 20 add_watcher() i=0 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 20 add_watcher() i=1 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 20 add_watcher() i=2 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 20 add_watcher() i=3 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 20 add_watcher() i=4 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 20 add_watcher() i=5 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.591+0000 7fae4b20f800 20 add_watcher() i=6 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.592+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94582856156352] v32'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.592+0000 7fae4b20f800 20 add_watcher() i=7 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.592+0000 7fae4b20f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.592+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98b4190 con 0x5605c98a0070 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.592+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.593+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94582856157872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98b16b0 con 0x5605c98a0070 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.593+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94582856157872] v32'200 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7facfd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fae4b20f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fae4b20f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fae4b20f800 5 note: GC not initialized 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fac77fff640 20 reqs_thread_entry: start 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fae4b20f800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fae4b20f800 20 Filter name: none 2024-04-23T19:11:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fac75ffb640 20 reqs_thread_entry: start 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fae4b20f800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.594+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98e7240 con 0x5605c98a0070 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.595+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.595+0000 7fae4b20f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.595+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5605c98e8630 con 0x7fae3808c740 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.595+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.596+0000 7facfd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.596+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94582856157872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98b3940 con 0x5605c98a0070 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.597+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94582856157872] v32'201 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.597+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94582856133680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98b0a80 con 0x7fae40041040 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.598+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94582856133680] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.598+0000 7fae4b20f800 20 remove_watcher() i=0 2024-04-23T19:11:25.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.598+0000 7fae4b20f800 2 removed watcher, disabling cache 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.598+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94582856137536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aac30 con 0x5605c98a0070 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.599+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94582856137536] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.599+0000 7fae4b20f800 20 remove_watcher() i=1 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.599+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94582856142352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aac30 con 0x5605c98a6300 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.600+0000 7fae47015640 1 -- 172.21.15.63:0/3433346014 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94582856142352] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae40067040 con 0x5605c98a6300 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.600+0000 7fae4b20f800 20 remove_watcher() i=2 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.600+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94582856145952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aac30 con 0x7fae40041040 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.601+0000 7fae46814640 1 -- 172.21.15.63:0/3433346014 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94582856145952] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae38078070 con 0x7fae40041040 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.601+0000 7fae4b20f800 20 remove_watcher() i=3 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.601+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94582856149552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98adc20 con 0x5605c98a0070 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.602+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94582856149552] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x5605c98a0070 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.602+0000 7fae4b20f800 20 remove_watcher() i=4 2024-04-23T19:11:25.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.602+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94582856153648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98aac30 con 0x7fae3808c740 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.604+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94582856153648] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.604+0000 7fae4b20f800 20 remove_watcher() i=5 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.604+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94582856138592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98adc20 con 0x7fae3808c740 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.605+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94582856138592] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.605+0000 7fae4b20f800 20 remove_watcher() i=6 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.605+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94582856156352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5605c98afa30 con 0x7fae3808c740 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94582856156352] v32'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae3c00f670 con 0x7fae3808c740 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 20 remove_watcher() i=7 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fae3808c740 msgr2=0x7fae38070350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fae3808c740 0x7fae38070350 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c98a0070 msgr2=0x5605c98a2530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c98a0070 0x5605c98a2530 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fae40041040 msgr2=0x7fae400089b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fae40041040 0x7fae400089b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5605c98a6300 msgr2=0x5605c98a86f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5605c98a6300 0x5605c98a86f0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad10036e60 msgr2=0x7fad10039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad10036e60 0x7fad10039320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae3808c690 tx=0x7fae38086000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae46013640 1 -- 172.21.15.63:0/3433346014 reap_dead start 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 msgr2=0x5605c9894470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c9894470 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fae4004bb70 tx=0x7fae4004cbe0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 shutdown_connections 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/3433346014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c988ce20 0x5605c9894470 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 >> 172.21.15.63:0/3433346014 conn(0x5605c9889140 msgr2=0x5605c98867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 shutdown_connections 2024-04-23T19:11:25.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/3433346014 wait complete. 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad20036eb0 msgr2=0x7fad20039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad20036eb0 0x7fad20039370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5605c986d090 tx=0x7fae3807d040 comp rx=0 tx=0).stop 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c987fb00 msgr2=0x5605c987ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.606+0000 7fae4b20f800 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c987fb00 0x5605c987ff20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5605c9865690 tx=0x7fae400178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 shutdown_connections 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad20036eb0 0x7fad20039370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/1617034690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c987fb00 0x5605c987ff20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 >> 172.21.15.63:0/1617034690 conn(0x5605c9873ba0 msgr2=0x5605c9873fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 shutdown_connections 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/1617034690 wait complete. 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5605c986ed60 msgr2=0x5605c9871150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5605c986ed60 0x5605c9871150 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c96dec00 msgr2=0x5605c986b6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c96dec00 0x5605c986b6b0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad2c036eb0 msgr2=0x7fad2c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad2c036eb0 0x7fad2c039370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae3c001e40 tx=0x7fae3c0013a0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 msgr2=0x5605c9864470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c9864470 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fae3804ad50 tx=0x7fae38002fb0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 shutdown_connections 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5605c986ed60 0x5605c9871150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5605c96dec00 0x5605c986b6b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fad2c036eb0 0x7fad2c039370 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 --2- 172.21.15.63:0/2405312723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605c96dd6a0 0x5605c9864470 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 >> 172.21.15.63:0/2405312723 conn(0x5605c97647e0 msgr2=0x5605c9856b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 shutdown_connections 2024-04-23T19:11:25.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.607+0000 7fae4b20f800 1 -- 172.21.15.63:0/2405312723 wait complete. 2024-04-23T19:11:25.618 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo2","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo3","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo4","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34,"bytes_processed":0,"bytes_returned":0}}]} 2024-04-23T19:11:25.619 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo2', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo3', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo4', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:11:25.619 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-23T19:11:25.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-23T19:11:25.619 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-04-23T19:11:25.665 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:25.666 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:25.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f9644f0c800 1 Processor -- start 2024-04-23T19:11:25.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f9644f0c800 1 -- start start 2024-04-23T19:11:25.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f9644f0c800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55b7b4704c20 0x55b7b4707010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f9644f0c800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b7b45b67f0 con 0x55b7b4707550 2024-04-23T19:11:25.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f9644f0c800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55b7b45a5e20 con 0x55b7b4704c20 2024-04-23T19:11:25.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f9642a0b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55b7b4704c20 0x55b7b4707010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f964220a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55b7b4707550 0x55b7b4709940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:58984/0 (socket says 172.21.15.63:58984) 2024-04-23T19:11:25.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.665+0000 7f964220a640 1 -- 172.21.15.63:0/2500710882 learned_addr learned my addr 172.21.15.63:0/2500710882 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:25.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.666+0000 7f9641a09640 1 -- 172.21.15.63:0/2500710882 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861407325 0 0) 0x55b7b45b67f0 con 0x55b7b4707550 2024-04-23T19:11:25.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.666+0000 7f9641a09640 1 -- 172.21.15.63:0/2500710882 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f962c001e10 con 0x55b7b4707550 2024-04-23T19:11:25.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.666+0000 7f9642a0b640 1 -- 172.21.15.63:0/2500710882 >> v1:172.21.15.16:6789/0 conn(0x55b7b4707550 legacy=0x55b7b4709940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.666+0000 7f9642a0b640 1 -- 172.21.15.63:0/2500710882 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b45c4b50 con 0x55b7b4704c20 2024-04-23T19:11:25.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9642a0b640 1 --2- 172.21.15.63:0/2500710882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b4707010 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f963400a470 tx=0x7f963401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ee529e412cbd81 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9641a09640 1 -- 172.21.15.63:0/2500710882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9634057070 con 0x55b7b4704c20 2024-04-23T19:11:25.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9641a09640 1 -- 172.21.15.63:0/2500710882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9634052570 con 0x55b7b4704c20 2024-04-23T19:11:25.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9641a09640 1 -- 172.21.15.63:0/2500710882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9634052870 con 0x55b7b4704c20 2024-04-23T19:11:25.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9644f0c800 1 -- 172.21.15.63:0/2500710882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 msgr2=0x55b7b4707010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9644f0c800 1 --2- 172.21.15.63:0/2500710882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b4707010 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f963400a470 tx=0x7f963401cdb0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9644f0c800 1 -- 172.21.15.63:0/2500710882 shutdown_connections 2024-04-23T19:11:25.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9644f0c800 1 --2- 172.21.15.63:0/2500710882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b4707010 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9644f0c800 1 -- 172.21.15.63:0/2500710882 >> 172.21.15.63:0/2500710882 conn(0x55b7b46fe630 msgr2=0x55b7b4700a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9644f0c800 1 -- 172.21.15.63:0/2500710882 shutdown_connections 2024-04-23T19:11:25.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.668+0000 7f9644f0c800 1 -- 172.21.15.63:0/2500710882 wait complete. 2024-04-23T19:11:25.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.681+0000 7f9644f0c800 1 Processor -- start 2024-04-23T19:11:25.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.681+0000 7f9644f0c800 1 -- start start 2024-04-23T19:11:25.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.681+0000 7f9644f0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.681+0000 7f9644f0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7b464f6f0 con 0x55b7b4704c20 2024-04-23T19:11:25.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.681+0000 7f9642a0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.681+0000 7f9642a0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37942/0 (socket says 172.21.15.63:37942) 2024-04-23T19:11:25.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.681+0000 7f9642a0b640 1 -- 172.21.15.63:0/282955042 learned_addr learned my addr 172.21.15.63:0/282955042 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.682+0000 7f9642a0b640 1 -- 172.21.15.63:0/282955042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b47f57c0 con 0x55b7b4704c20 2024-04-23T19:11:25.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.682+0000 7f9642a0b640 1 --2- 172.21.15.63:0/282955042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47f51f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f963404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ee0ce03f685781 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.682+0000 7f9530e09640 1 -- 172.21.15.63:0/282955042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9634057040 con 0x55b7b4704c20 2024-04-23T19:11:25.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.682+0000 7f9530e09640 1 -- 172.21.15.63:0/282955042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96340528d0 con 0x55b7b4704c20 2024-04-23T19:11:25.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.682+0000 7f9530e09640 1 -- 172.21.15.63:0/282955042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9634052bf0 con 0x55b7b4704c20 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 -- 172.21.15.63:0/282955042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 msgr2=0x55b7b47f51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 --2- 172.21.15.63:0/282955042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47f51f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f963404c000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 -- 172.21.15.63:0/282955042 shutdown_connections 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 --2- 172.21.15.63:0/282955042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47f51f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 -- 172.21.15.63:0/282955042 >> 172.21.15.63:0/282955042 conn(0x55b7b46fe630 msgr2=0x55b7b47f0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 -- 172.21.15.63:0/282955042 shutdown_connections 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 -- 172.21.15.63:0/282955042 wait complete. 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.683+0000 7f9644f0c800 1 Processor -- start 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9644f0c800 1 -- start start 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9644f0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9644f0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7b470d370 con 0x55b7b4704c20 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9642a0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9642a0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37946/0 (socket says 172.21.15.63:37946) 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9642a0b640 1 -- 172.21.15.63:0/803906480 learned_addr learned my addr 172.21.15.63:0/803906480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9642a0b640 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b47fd1d0 con 0x55b7b4704c20 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.684+0000 7f9642a0b640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47fcc90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f963401baa0 tx=0x7f9634003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.685+0000 7f952f606640 1 -- 172.21.15.63:0/803906480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9634073070 con 0x55b7b4704c20 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.685+0000 7f952f606640 1 -- 172.21.15.63:0/803906480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963405d650 con 0x55b7b4704c20 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.685+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7b47fd460 con 0x55b7b4704c20 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.685+0000 7f952f606640 1 -- 172.21.15.63:0/803906480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9634057050 con 0x55b7b4704c20 2024-04-23T19:11:25.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.685+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b4800110 con 0x55b7b4704c20 2024-04-23T19:11:25.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.685+0000 7f952f606640 1 -- 172.21.15.63:0/803906480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9634076070 con 0x55b7b4704c20 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f952f606640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9528036eb0 0x7f9528039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f952f606640 1 -- 172.21.15.63:0/803906480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f963408e540 con 0x55b7b4704c20 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f964220a640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9528036eb0 0x7f9528039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b4707550 0x55b7b4803ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4804410 con 0x55b7b4707550 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f964320c640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b4707550 0x55b7b4803ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f964220a640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9528036eb0 0x7f9528039370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9638000f60 tx=0x7f96380010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.686+0000 7f964320c640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b4707550 0x55b7b4803ed0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f964320c640 1 -- 172.21.15.63:0/803906480 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f963c00a4a0 con 0x55b7b4707550 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4801480 con 0x55b7b4707550 2024-04-23T19:11:25.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f964320c640 1 -- 172.21.15.63:0/803906480 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f963c00a4a0 con 0x55b7b4707550 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4806260 con 0x55b7b4707550 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f964320c640 1 -- 172.21.15.63:0/803906480 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f963c00a4a0 con 0x55b7b4707550 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b48055a0 con 0x55b7b4707550 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f964320c640 1 -- 172.21.15.63:0/803906480 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f963c00a4a0 con 0x55b7b4707550 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7b4807610 0x55b7b4809a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.687+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4809f80 con 0x55b7b4807610 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.688+0000 7f9642a0b640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7b4807610 0x55b7b4809a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.688+0000 7f9642a0b640 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7b4807610 0x55b7b4809a00 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f9642a0b640 1 -- 172.21.15.63:0/803906480 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9634075030 con 0x55b7b4807610 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f9644f0c800 1 Processor -- start 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f9644f0c800 1 -- start start 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f9644f0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b48183a0 0x55b7b48187c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f9644f0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7b45f5e30 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f964320c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b48183a0 0x55b7b48187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f964320c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b48183a0 0x55b7b48187c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37952/0 (socket says 172.21.15.63:37952) 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f964320c640 1 -- 172.21.15.63:0/4197415770 learned_addr learned my addr 172.21.15.63:0/4197415770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.689+0000 7f964320c640 1 -- 172.21.15.63:0/4197415770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b4818d00 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.690+0000 7f964320c640 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b48183a0 0x55b7b48187c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55b7b47fe090 tx=0x7f963c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.690+0000 7f952ce01640 1 -- 172.21.15.63:0/4197415770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f963c013440 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.690+0000 7f952ce01640 1 -- 172.21.15.63:0/4197415770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c0135e0 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.690+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7b4819770 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.690+0000 7f952ce01640 1 -- 172.21.15.63:0/4197415770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f963c0138c0 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.690+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b4819c70 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f952ce01640 1 -- 172.21.15.63:0/4197415770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f963c011040 con 0x55b7b48183a0 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f952ce01640 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f951c036eb0 0x7f951c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f952ce01640 1 -- 172.21.15.63:0/4197415770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f963c01d070 con 0x55b7b48183a0 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f9642a0b640 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f951c036eb0 0x7f951c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f9644f0c800 1 Processor -- start 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f9644f0c800 1 -- start start 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f9642a0b640 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f951c036eb0 0x7f951c039370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9634076470 tx=0x7f963408c810 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f9644f0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4825aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.691+0000 7f9644f0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7b47f5a80 con 0x55b7b4825680 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f964320c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4825aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f964320c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4825aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37954/0 (socket says 172.21.15.63:37954) 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f964320c640 1 -- 172.21.15.63:0/547484204 learned_addr learned my addr 172.21.15.63:0/547484204 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f964320c640 1 -- 172.21.15.63:0/547484204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b4826070 con 0x55b7b4825680 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f964320c640 1 --2- 172.21.15.63:0/547484204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4825aa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f963c04cca0 tx=0x7f963c0127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babcec49a788f821 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f95267fc640 1 -- 172.21.15.63:0/547484204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f963c042030 con 0x55b7b4825680 2024-04-23T19:11:25.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f95267fc640 1 -- 172.21.15.63:0/547484204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c012cc0 con 0x55b7b4825680 2024-04-23T19:11:25.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.692+0000 7f95267fc640 1 -- 172.21.15.63:0/547484204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f963c027040 con 0x55b7b4825680 2024-04-23T19:11:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 -- 172.21.15.63:0/547484204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 msgr2=0x55b7b4825aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 --2- 172.21.15.63:0/547484204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4825aa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f963c04cca0 tx=0x7f963c0127a0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 -- 172.21.15.63:0/547484204 shutdown_connections 2024-04-23T19:11:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 --2- 172.21.15.63:0/547484204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4825aa0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 -- 172.21.15.63:0/547484204 >> 172.21.15.63:0/547484204 conn(0x55b7b48219e0 msgr2=0x55b7b481fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 -- 172.21.15.63:0/547484204 shutdown_connections 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 -- 172.21.15.63:0/547484204 wait complete. 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 Processor -- start 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 -- start start 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4704540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f9644f0c800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7b48267e0 con 0x55b7b4825680 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f964320c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4704540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f964320c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4704540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37970/0 (socket says 172.21.15.63:37970) 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.693+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 learned_addr learned my addr 172.21.15.63:0/1726701578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.694+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b4702c30 con 0x55b7b4825680 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.694+0000 7f964320c640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4704540 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f963c04bb90 tx=0x7f963c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.694+0000 7f9524ff9640 1 -- 172.21.15.63:0/1726701578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f963c027030 con 0x55b7b4825680 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.694+0000 7f9524ff9640 1 -- 172.21.15.63:0/1726701578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c021c60 con 0x55b7b4825680 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.694+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7b4702ec0 con 0x55b7b4825680 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.694+0000 7f9524ff9640 1 -- 172.21.15.63:0/1726701578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f963c02a350 con 0x55b7b4825680 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.694+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b4703360 con 0x55b7b4825680 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f9524ff9640 1 -- 172.21.15.63:0/1726701578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f963c045070 con 0x55b7b4825680 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f9524ff9640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f950c036e60 0x7f950c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f9524ff9640 1 -- 172.21.15.63:0/1726701578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f963c043070 con 0x55b7b4825680 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f9642a0b640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f950c036e60 0x7f950c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b483ab20 0x55b7b483cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b483d730 con 0x55b7b483ab20 2024-04-23T19:11:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.695+0000 7f964220a640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b483ab20 0x55b7b483cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9642a0b640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f950c036e60 0x7f950c039320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9634044720 tx=0x7f9634091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f964220a640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b483ab20 0x55b7b483cfe0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9638009050 con 0x55b7b483ab20 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 20 realm 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4839f70 con 0x55b7b483ab20 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9638009050 con 0x55b7b483ab20 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.696+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b483e210 con 0x55b7b483ab20 2024-04-23T19:11:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9638009050 con 0x55b7b483ab20 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b483eae0 con 0x55b7b483ab20 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9638009050 con 0x55b7b483ab20 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b483ee80 con 0x55b7b483ab20 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9638009050 con 0x55b7b483ab20 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 20 searching for the correct realm 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7b4840d60 0x55b7b4843150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.697+0000 7f964320c640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7b4840d60 0x55b7b4843150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.698+0000 7f964320c640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7b4840d60 0x55b7b4843150 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.698+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.698+0000 7f964320c640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f963c0123a0 0x7f963c046820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.698+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f963c0123a0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.698+0000 7f9642a0b640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f963c0123a0 0x7f963c046820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.698+0000 7f9642a0b640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f963c0123a0 0x7f963c046820 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.699+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.699+0000 7f9642a0b640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9634078060 0x7f9634077a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.699+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.699+0000 7f964220a640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9634078060 0x7f9634077a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.699+0000 7f964220a640 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9634078060 0x7f9634077a70 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9638009050 con 0x7f9634078060 2024-04-23T19:11:25.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9638009050 con 0x55b7b483ab20 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.700+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f963c0123a0 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.701+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f963c0123a0 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.702+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.703+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.704+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.704+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.704+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.704+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.704+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.704+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f963c0123a0 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f963c0123a0 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.705+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f963c0123a0 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b4840d60 2024-04-23T19:11:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.706+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f9634078060 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x55b7b483ab20 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843820 con 0x7f963c0123a0 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:25.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.707+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b483f800 con 0x55b7b483ab20 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b483f440 con 0x55b7b483ab20 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4840160 con 0x7f9634078060 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 zone default found 2024-04-23T19:11:25.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 4 Realm: () 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 zonegroup default 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4843ca0 con 0x7f963c0123a0 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.708+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4845ed0 con 0x55b7b4840d60 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4846270 con 0x55b7b4840d60 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 20 started sync module instance, tier type = 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.709+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4845310 con 0x7f963c0123a0 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.710+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.710+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94247495948080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4844f70 con 0x7f963c0123a0 2024-04-23T19:11:25.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.710+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4846fd0 con 0x55b7b483ab20 2024-04-23T19:11:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.711+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94247495948080] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.712+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.712+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94247495951888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b48473c0 con 0x55b7b483ab20 2024-04-23T19:11:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.712+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4847e00 con 0x55b7b4840d60 2024-04-23T19:11:25.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.713+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94247495951888] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.713+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.713+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94247495956688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b48481f0 con 0x55b7b4840d60 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.713+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4848c50 con 0x7f963c0123a0 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.714+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94247495956688] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.714+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.714+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94247495952944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4849040 con 0x7f963c0123a0 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.714+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4849a70 con 0x55b7b483ab20 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.715+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94247495952944] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.716+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.716+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94247495964032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4849a70 con 0x55b7b483ab20 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.716+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484a4f0 con 0x7f9634078060 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.717+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.717+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94247495966720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484a420 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.717+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484b420 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.717+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94247495964032] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.718+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94247495966720] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.718+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.718+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94247495960368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484af30 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.718+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484bcc0 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94247495960368] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94247495968912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484af30 con 0x7f9634078060 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 20 add_watcher() i=0 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 20 add_watcher() i=1 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 20 add_watcher() i=2 2024-04-23T19:11:25.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 20 add_watcher() i=3 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 20 add_watcher() i=4 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 20 add_watcher() i=5 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.719+0000 7f9644f0c800 20 add_watcher() i=6 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.720+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94247495968912] v32'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.720+0000 7f9644f0c800 20 add_watcher() i=7 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.720+0000 7f9644f0c800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.720+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484ecf0 con 0x55b7b483ab20 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.720+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.720+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94247495972880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484c230 con 0x55b7b483ab20 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.721+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94247495972880] v32'202 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f94f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9644f0c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:25.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9644f0c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9644f0c800 5 note: GC not initialized 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9473fff640 20 reqs_thread_entry: start 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9644f0c800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9644f0c800 20 Filter name: none 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9471ffb640 20 reqs_thread_entry: start 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9644f0c800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4881d20 con 0x55b7b483ab20 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.722+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.723+0000 7f9644f0c800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.723+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4883110 con 0x7f9634078060 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.723+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.724+0000 7f94f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.724+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94247495972880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4837ee0 con 0x55b7b483ab20 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.725+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94247495972880] v32'203 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.725+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94247495948080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484b810 con 0x7f963c0123a0 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.726+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94247495948080] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.726+0000 7f9644f0c800 20 remove_watcher() i=0 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.726+0000 7f9644f0c800 2 removed watcher, disabling cache 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.726+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94247495951888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4845730 con 0x55b7b483ab20 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.727+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94247495951888] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.727+0000 7f9644f0c800 20 remove_watcher() i=1 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.727+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94247495956688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4846610 con 0x55b7b4840d60 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.728+0000 7f964320c640 1 -- 172.21.15.63:0/1726701578 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94247495956688] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963c029070 con 0x55b7b4840d60 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.728+0000 7f9644f0c800 20 remove_watcher() i=2 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.728+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94247495952944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4846610 con 0x7f963c0123a0 2024-04-23T19:11:25.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.729+0000 7f9642a0b640 1 -- 172.21.15.63:0/1726701578 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94247495952944] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9634075030 con 0x7f963c0123a0 2024-04-23T19:11:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.729+0000 7f9644f0c800 20 remove_watcher() i=3 2024-04-23T19:11:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.729+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94247495964032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4846ac0 con 0x55b7b483ab20 2024-04-23T19:11:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.730+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94247495964032] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x55b7b483ab20 2024-04-23T19:11:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.730+0000 7f9644f0c800 20 remove_watcher() i=4 2024-04-23T19:11:25.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.730+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94247495966720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4849580 con 0x7f9634078060 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.731+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94247495966720] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.732+0000 7f9644f0c800 20 remove_watcher() i=5 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.732+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94247495960368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b484a000 con 0x7f9634078060 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.732+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94247495960368] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.733+0000 7f9644f0c800 20 remove_watcher() i=6 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.733+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94247495968912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55b7b4849580 con 0x7f9634078060 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.733+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94247495968912] v32'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f963802a040 con 0x7f9634078060 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.733+0000 7f9644f0c800 20 remove_watcher() i=7 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9634078060 msgr2=0x7f9634077a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9634078060 0x7f9634077a70 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b483ab20 msgr2=0x55b7b483cfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b483ab20 0x55b7b483cfe0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f963c0123a0 msgr2=0x7f963c046820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f963c0123a0 0x7f963c046820 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7b4840d60 msgr2=0x55b7b4843150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55b7b4840d60 0x55b7b4843150 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f950c036e60 msgr2=0x7f950c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f950c036e60 0x7f950c039320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9634044720 tx=0x7f9634091000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 msgr2=0x55b7b4704540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f964220a640 1 -- 172.21.15.63:0/1726701578 reap_dead start 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4704540 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f963c04bb90 tx=0x7f963c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 shutdown_connections 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/1726701578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4825680 0x55b7b4704540 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 >> 172.21.15.63:0/1726701578 conn(0x55b7b48219e0 msgr2=0x55b7b481f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 shutdown_connections 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/1726701578 wait complete. 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f951c036eb0 msgr2=0x7f951c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f951c036eb0 0x7f951c039370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9634076470 tx=0x7f963408c810 comp rx=0 tx=0).stop 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b48183a0 msgr2=0x55b7b48187c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b48183a0 0x55b7b48187c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55b7b47fe090 tx=0x7f963c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 shutdown_connections 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f951c036eb0 0x7f951c039370 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 --2- 172.21.15.63:0/4197415770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b48183a0 0x55b7b48187c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 >> 172.21.15.63:0/4197415770 conn(0x55b7b480c440 msgr2=0x55b7b480c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 shutdown_connections 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.734+0000 7f9644f0c800 1 -- 172.21.15.63:0/4197415770 wait complete. 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7b4807610 msgr2=0x55b7b4809a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7b4807610 0x55b7b4809a00 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b4707550 msgr2=0x55b7b4803ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b4707550 0x55b7b4803ed0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9528036eb0 msgr2=0x7f9528039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9528036eb0 0x7f9528039370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9638000f60 tx=0x7f96380010a0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 msgr2=0x55b7b47fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47fcc90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f963401baa0 tx=0x7f9634003020 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 shutdown_connections 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55b7b4807610 0x55b7b4809a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55b7b4707550 0x55b7b4803ed0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9528036eb0 0x7f9528039370 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 --2- 172.21.15.63:0/803906480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7b4704c20 0x55b7b47fcc90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 >> 172.21.15.63:0/803906480 conn(0x55b7b46fe630 msgr2=0x55b7b47ef310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 shutdown_connections 2024-04-23T19:11:25.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.735+0000 7f9644f0c800 1 -- 172.21.15.63:0/803906480 wait complete. 2024-04-23T19:11:25.747 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5}],"s3select":{}},{"bucket":"myfoo2","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo3","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo4","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8}],"total":{"bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8,"bytes_processed":0,"bytes_returned":0}}]} 2024-04-23T19:11:25.747 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:11:25.747 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-23T19:11:25.747 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-23T19:11:25.747 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-04-23T19:11:25.793 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:25.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f2370333800 1 Processor -- start 2024-04-23T19:11:25.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f2370333800 1 -- start start 2024-04-23T19:11:25.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f2370333800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5620a3fb2b90 0x5620a3fb4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f2370333800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5620a3e647f0 con 0x5620a3fb54c0 2024-04-23T19:11:25.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f2370333800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5620a3e53e20 con 0x5620a3fb2b90 2024-04-23T19:11:25.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f236bff5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5620a3fb2b90 0x5620a3fb4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f236b7f4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5620a3fb54c0 0x5620a3fb78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:58996/0 (socket says 172.21.15.63:58996) 2024-04-23T19:11:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f236bff5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5620a3fb2b90 0x5620a3fb4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37976/0 (socket says 172.21.15.63:37976) 2024-04-23T19:11:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.793+0000 7f236b7f4640 1 -- 172.21.15.63:0/1991256505 learned_addr learned my addr 172.21.15.63:0/1991256505 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.794+0000 7f236bff5640 1 -- 172.21.15.63:0/1991256505 >> v1:172.21.15.16:6789/0 conn(0x5620a3fb54c0 legacy=0x5620a3fb78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.794+0000 7f236bff5640 1 -- 172.21.15.63:0/1991256505 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a3e72b50 con 0x5620a3fb2b90 2024-04-23T19:11:25.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.795+0000 7f236bff5640 1 --2- 172.21.15.63:0/1991256505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a3fb4f80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f235c00a470 tx=0x7f235c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d0754e584b7c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f236aff3640 1 -- 172.21.15.63:0/1991256505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c057070 con 0x5620a3fb2b90 2024-04-23T19:11:25.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f236aff3640 1 -- 172.21.15.63:0/1991256505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c052570 con 0x5620a3fb2b90 2024-04-23T19:11:25.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f236aff3640 1 -- 172.21.15.63:0/1991256505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c052870 con 0x5620a3fb2b90 2024-04-23T19:11:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f2370333800 1 -- 172.21.15.63:0/1991256505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 msgr2=0x5620a3fb4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f2370333800 1 --2- 172.21.15.63:0/1991256505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a3fb4f80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f235c00a470 tx=0x7f235c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:11:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f2370333800 1 -- 172.21.15.63:0/1991256505 shutdown_connections 2024-04-23T19:11:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f2370333800 1 --2- 172.21.15.63:0/1991256505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a3fb4f80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f2370333800 1 -- 172.21.15.63:0/1991256505 >> 172.21.15.63:0/1991256505 conn(0x5620a3fac600 msgr2=0x5620a3faea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f2370333800 1 -- 172.21.15.63:0/1991256505 shutdown_connections 2024-04-23T19:11:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.796+0000 7f2370333800 1 -- 172.21.15.63:0/1991256505 wait complete. 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.809+0000 7f2370333800 1 Processor -- start 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.809+0000 7f2370333800 1 -- start start 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.809+0000 7f2370333800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.809+0000 7f2370333800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620a3efd6f0 con 0x5620a3fb2b90 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f236bff5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f236bff5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37984/0 (socket says 172.21.15.63:37984) 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f236bff5640 1 -- 172.21.15.63:0/3220900909 learned_addr learned my addr 172.21.15.63:0/3220900909 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f236bff5640 1 -- 172.21.15.63:0/3220900909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a40a3910 con 0x5620a3fb2b90 2024-04-23T19:11:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f236bff5640 1 --2- 172.21.15.63:0/3220900909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40a3340 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f235c01ced0 tx=0x7f235c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02544ab1cecb003 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f2259e0b640 1 -- 172.21.15.63:0/3220900909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c057040 con 0x5620a3fb2b90 2024-04-23T19:11:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f2259e0b640 1 -- 172.21.15.63:0/3220900909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c0529a0 con 0x5620a3fb2b90 2024-04-23T19:11:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.810+0000 7f2259e0b640 1 -- 172.21.15.63:0/3220900909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c052cc0 con 0x5620a3fb2b90 2024-04-23T19:11:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 -- 172.21.15.63:0/3220900909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 msgr2=0x5620a40a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 --2- 172.21.15.63:0/3220900909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40a3340 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f235c01ced0 tx=0x7f235c04c000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 -- 172.21.15.63:0/3220900909 shutdown_connections 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 --2- 172.21.15.63:0/3220900909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40a3340 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 -- 172.21.15.63:0/3220900909 >> 172.21.15.63:0/3220900909 conn(0x5620a3fac600 msgr2=0x5620a409ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 -- 172.21.15.63:0/3220900909 shutdown_connections 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 -- 172.21.15.63:0/3220900909 wait complete. 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 Processor -- start 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.811+0000 7f2370333800 1 -- start start 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f2370333800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f2370333800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620a3fb9270 con 0x5620a3fb2b90 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f236bff5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f236bff5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40aacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:37998/0 (socket says 172.21.15.63:37998) 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f236bff5640 1 -- 172.21.15.63:0/1699866137 learned_addr learned my addr 172.21.15.63:0/1699866137 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f236bff5640 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a40ab230 con 0x5620a3fb2b90 2024-04-23T19:11:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f236bff5640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40aacf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f235c052740 tx=0x7f235c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f2253fff640 1 -- 172.21.15.63:0/1699866137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c073070 con 0x5620a3fb2b90 2024-04-23T19:11:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f2253fff640 1 -- 172.21.15.63:0/1699866137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c05d860 con 0x5620a3fb2b90 2024-04-23T19:11:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5620a40ab4c0 con 0x5620a3fb2b90 2024-04-23T19:11:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.812+0000 7f2253fff640 1 -- 172.21.15.63:0/1699866137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c057070 con 0x5620a3fb2b90 2024-04-23T19:11:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.813+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a40ae170 con 0x5620a3fb2b90 2024-04-23T19:11:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.813+0000 7f2253fff640 1 -- 172.21.15.63:0/1699866137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f235c076070 con 0x5620a3fb2b90 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.813+0000 7f2253fff640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2254036e60 0x7f2254039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.813+0000 7f2253fff640 1 -- 172.21.15.63:0/1699866137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f235c08e790 con 0x5620a3fb2b90 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.813+0000 7f236b7f4640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2254036e60 0x7f2254039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.814+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a3fb54c0 0x5620a40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.814+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40b2510 con 0x5620a3fb54c0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.814+0000 7f236c7f6640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a3fb54c0 0x5620a40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.814+0000 7f236b7f4640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2254036e60 0x7f2254039320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2360000f60 tx=0x7f23600012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.814+0000 7f236c7f6640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a3fb54c0 0x5620a40b1f30 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.814+0000 7f236c7f6640 1 -- 172.21.15.63:0/1699866137 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f236400a4a0 con 0x5620a3fb54c0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.814+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40af500 con 0x5620a3fb54c0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.815+0000 7f236c7f6640 1 -- 172.21.15.63:0/1699866137 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f236400a4a0 con 0x5620a3fb54c0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.815+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40b4350 con 0x5620a3fb54c0 2024-04-23T19:11:25.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.815+0000 7f236c7f6640 1 -- 172.21.15.63:0/1699866137 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f236400a4a0 con 0x5620a3fb54c0 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.815+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40b3690 con 0x5620a3fb54c0 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.815+0000 7f236c7f6640 1 -- 172.21.15.63:0/1699866137 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f236400a4a0 con 0x5620a3fb54c0 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.815+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620a40b5700 0x5620a40b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.815+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40b8070 con 0x5620a40b5700 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.816+0000 7f236bff5640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620a40b5700 0x5620a40b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.816+0000 7f236bff5640 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620a40b5700 0x5620a40b7af0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f236bff5640 1 -- 172.21.15.63:0/1699866137 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f235c075030 con 0x5620a40b5700 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f2370333800 1 Processor -- start 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f2370333800 1 -- start start 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f2370333800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40c6490 0x5620a40c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f2370333800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620a3ea3e30 con 0x5620a40c6490 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f236c7f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40c6490 0x5620a40c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f236c7f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40c6490 0x5620a40c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38004/0 (socket says 172.21.15.63:38004) 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f236c7f6640 1 -- 172.21.15.63:0/1867227430 learned_addr learned my addr 172.21.15.63:0/1867227430 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.817+0000 7f236c7f6640 1 -- 172.21.15.63:0/1867227430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a40c6df0 con 0x5620a40c6490 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.818+0000 7f236c7f6640 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40c6490 0x5620a40c68b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x5620a40b2910 tx=0x7f23640178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.818+0000 7f22517fa640 1 -- 172.21.15.63:0/1867227430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2364013440 con 0x5620a40c6490 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.818+0000 7f22517fa640 1 -- 172.21.15.63:0/1867227430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23640135e0 con 0x5620a40c6490 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.818+0000 7f22517fa640 1 -- 172.21.15.63:0/1867227430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23640138c0 con 0x5620a40c6490 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.818+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5620a40c7860 con 0x5620a40c6490 2024-04-23T19:11:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.818+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a40c7d60 con 0x5620a40c6490 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f22517fa640 1 -- 172.21.15.63:0/1867227430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f2364011040 con 0x5620a40c6490 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f22517fa640 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2244036e60 0x7f2244039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f22517fa640 1 -- 172.21.15.63:0/1867227430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f236401d070 con 0x5620a40c6490 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f236bff5640 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2244036e60 0x7f2244039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f2370333800 1 Processor -- start 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f2370333800 1 -- start start 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f236bff5640 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2244036e60 0x7f2244039320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f235c077d90 tx=0x7f235c08ca80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f2370333800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.819+0000 7f2370333800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620a40a3bd0 con 0x5620a40d3790 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f236c7f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f236c7f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38010/0 (socket says 172.21.15.63:38010) 2024-04-23T19:11:25.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f236c7f6640 1 -- 172.21.15.63:0/3805242087 learned_addr learned my addr 172.21.15.63:0/3805242087 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f236c7f6640 1 -- 172.21.15.63:0/3805242087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a40d4180 con 0x5620a40d3790 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f236c7f6640 1 --2- 172.21.15.63:0/3805242087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40d3bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2364001470 tx=0x7f2364012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa512c89e54ab37 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f224affd640 1 -- 172.21.15.63:0/3805242087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2364015430 con 0x5620a40d3790 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f224affd640 1 -- 172.21.15.63:0/3805242087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236402ac50 con 0x5620a40d3790 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.820+0000 7f224affd640 1 -- 172.21.15.63:0/3805242087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23640155d0 con 0x5620a40d3790 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 -- 172.21.15.63:0/3805242087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 msgr2=0x5620a40d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 --2- 172.21.15.63:0/3805242087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40d3bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2364001470 tx=0x7f2364012a30 comp rx=0 tx=0).stop 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 -- 172.21.15.63:0/3805242087 shutdown_connections 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 --2- 172.21.15.63:0/3805242087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40d3bb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 -- 172.21.15.63:0/3805242087 >> 172.21.15.63:0/3805242087 conn(0x5620a40cfaf0 msgr2=0x5620a40cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 -- 172.21.15.63:0/3805242087 shutdown_connections 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 -- 172.21.15.63:0/3805242087 wait complete. 2024-04-23T19:11:25.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 Processor -- start 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 -- start start 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40dad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f2370333800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620a40d48f0 con 0x5620a40d3790 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f236c7f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f236c7f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40dad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38020/0 (socket says 172.21.15.63:38020) 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.821+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 learned_addr learned my addr 172.21.15.63:0/1811737095 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.822+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a40db2a0 con 0x5620a40d3790 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.822+0000 7f236c7f6640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40dad60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2364012a70 tx=0x7f236404cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.822+0000 7f22497fa640 1 -- 172.21.15.63:0/1811737095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f236402ae00 con 0x5620a40d3790 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.822+0000 7f22497fa640 1 -- 172.21.15.63:0/1811737095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23640123a0 con 0x5620a40d3790 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.822+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5620a40db530 con 0x5620a40d3790 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.822+0000 7f22497fa640 1 -- 172.21.15.63:0/1811737095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2364012680 con 0x5620a40d3790 2024-04-23T19:11:25.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.822+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a40de1e0 con 0x5620a40d3790 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f22497fa640 1 -- 172.21.15.63:0/1811737095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f236404ea80 con 0x5620a40d3790 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f22497fa640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2234036eb0 0x7f2234039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f22497fa640 1 -- 172.21.15.63:0/1811737095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f236400e040 con 0x5620a40d3790 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f236bff5640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2234036eb0 0x7f2234039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f236bff5640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2234036eb0 0x7f2234039370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5620a40b8450 tx=0x7f235c088040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a40e6800 0x5620a40e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40e9410 con 0x5620a40e6800 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.823+0000 7f236b7f4640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a40e6800 0x5620a40e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f236b7f4640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a40e6800 0x5620a40e8cc0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2360009050 con 0x5620a40e6800 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 20 realm 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40e5c50 con 0x5620a40e6800 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2360009050 con 0x5620a40e6800 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40e9ef0 con 0x5620a40e6800 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2360009050 con 0x5620a40e6800 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.824+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ea7c0 con 0x5620a40e6800 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2360009050 con 0x5620a40e6800 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f2370333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40eab60 con 0x5620a40e6800 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2360009050 con 0x5620a40e6800 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f2370333800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f2370333800 20 searching for the correct realm 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5620a40eca40 0x5620a40eee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.825+0000 7f236c7f6640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5620a40eca40 0x5620a40eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.826+0000 7f236c7f6640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5620a40eca40 0x5620a40eee30 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.826+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.826+0000 7f236c7f6640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f236405a040 0x7f236402b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.826+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f236405a040 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.826+0000 7f236bff5640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f236405a040 0x7f236402b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.826+0000 7f236bff5640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f236405a040 0x7f236402b800 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.827+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.827+0000 7f236bff5640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f235c09d040 0x7f235c08e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.827+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.827+0000 7f236b7f4640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f235c09d040 0x7f235c08e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.827+0000 7f236b7f4640 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f235c09d040 0x7f235c08e0b0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2360009050 con 0x7f235c09d040 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2360009050 con 0x5620a40e6800 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.828+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f236405a040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.829+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.830+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.830+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.830+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.830+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f236405a040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.830+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.830+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.831+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.832+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.832+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.832+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.832+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.832+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.832+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.833+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.833+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.833+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.833+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f236405a040 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.833+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.833+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.834+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.834+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f236405a040 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.834+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.834+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.834+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.834+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f236405a040 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40eca40 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f235c09d040 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.835+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x5620a40e6800 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef4e0 con 0x7f236405a040 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40eb510 con 0x5620a40e6800 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.836+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ebdb0 con 0x5620a40e6800 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40eb150 con 0x7f235c09d040 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 zone default found 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 4 Realm: () 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 zonegroup default 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40ef940 con 0x7f236405a040 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.837+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f1b70 con 0x5620a40eca40 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f2370333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f1f10 con 0x5620a40eca40 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f2370333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f2370333800 20 started sync module instance, tier type = 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f2370333800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.838+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f0fb0 con 0x7f236405a040 2024-04-23T19:11:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.839+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.839+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94698191393744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f0c10 con 0x7f236405a040 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.839+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f2c70 con 0x5620a40e6800 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.840+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94698191393744] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.840+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.840+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94698191397552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f3090 con 0x5620a40e6800 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.840+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f3ab0 con 0x5620a40eca40 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.841+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94698191397552] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.842+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.842+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94698191402400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f3ed0 con 0x5620a40eca40 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.842+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f4930 con 0x7f236405a040 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.843+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94698191402400] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.843+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.843+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94698191398608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f4d20 con 0x7f236405a040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.843+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f5780 con 0x5620a40e6800 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.844+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.844+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94698191409760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f6020 con 0x5620a40e6800 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.844+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f6ae0 con 0x7f235c09d040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.844+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94698191398608] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.845+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.845+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94698191409760] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.845+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94698191414704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f6aa0 con 0x7f235c09d040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.845+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f7040 con 0x7f235c09d040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.846+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94698191414704] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.846+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.846+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94698191406096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f7040 con 0x7f235c09d040 2024-04-23T19:11:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.846+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f7a00 con 0x7f235c09d040 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94698191406096] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94698191411968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f7a00 con 0x7f235c09d040 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 20 add_watcher() i=0 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 20 add_watcher() i=1 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 20 add_watcher() i=2 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 20 add_watcher() i=3 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 20 add_watcher() i=4 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 20 add_watcher() i=5 2024-04-23T19:11:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.847+0000 7f2370333800 20 add_watcher() i=6 2024-04-23T19:11:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.848+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94698191411968] v32'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.848+0000 7f2370333800 20 add_watcher() i=7 2024-04-23T19:11:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.848+0000 7f2370333800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.848+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a40fad40 con 0x5620a40e6800 2024-04-23T19:11:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.848+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.849+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94698191431520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f7b70 con 0x5620a40e6800 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.849+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94698191431520] v32'204 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2221fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2370333800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2370333800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2370333800 5 note: GC not initialized 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2220fe1640 20 reqs_thread_entry: start 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2370333800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2370333800 20 Filter name: none 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f219a7fc640 20 reqs_thread_entry: start 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2370333800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.850+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a412ffe0 con 0x5620a40e6800 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.851+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.851+0000 7f2370333800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.851+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5620a41313d0 con 0x7f235c09d040 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.851+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.852+0000 7f2221fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.852+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94698191431520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a412ec30 con 0x5620a40e6800 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.853+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94698191431520] v32'205 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.853+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94698191393744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40fa760 con 0x7f236405a040 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.854+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94698191393744] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.854+0000 7f2370333800 20 remove_watcher() i=0 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.854+0000 7f2370333800 2 removed watcher, disabling cache 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.854+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94698191397552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f13d0 con 0x5620a40e6800 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.855+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94698191397552] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.855+0000 7f2370333800 20 remove_watcher() i=1 2024-04-23T19:11:25.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.855+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94698191402400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f13d0 con 0x5620a40eca40 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.856+0000 7f236c7f6640 1 -- 172.21.15.63:0/1811737095 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94698191402400] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2364029070 con 0x5620a40eca40 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.856+0000 7f2370333800 20 remove_watcher() i=2 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.856+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94698191398608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f35a0 con 0x7f236405a040 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.857+0000 7f236bff5640 1 -- 172.21.15.63:0/1811737095 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94698191398608] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f235c075030 con 0x7f236405a040 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.857+0000 7f2370333800 20 remove_watcher() i=3 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.857+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94698191409760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f2760 con 0x5620a40e6800 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.858+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94698191409760] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x5620a40e6800 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.858+0000 7f2370333800 20 remove_watcher() i=4 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.858+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94698191414704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f5260 con 0x7f235c09d040 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.859+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94698191414704] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.859+0000 7f2370333800 20 remove_watcher() i=5 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.859+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94698191406096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f5260 con 0x7f235c09d040 2024-04-23T19:11:25.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.860+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94698191406096] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.860+0000 7f2370333800 20 remove_watcher() i=6 2024-04-23T19:11:25.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.860+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94698191411968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5620a40f4410 con 0x7f235c09d040 2024-04-23T19:11:25.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.861+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94698191411968] v32'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f236002a040 con 0x7f235c09d040 2024-04-23T19:11:25.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.861+0000 7f2370333800 20 remove_watcher() i=7 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f235c09d040 msgr2=0x7f235c08e0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f235c09d040 0x7f235c08e0b0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a40e6800 msgr2=0x5620a40e8cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a40e6800 0x5620a40e8cc0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f236405a040 msgr2=0x7f236402b800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f236405a040 0x7f236402b800 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5620a40eca40 msgr2=0x5620a40eee30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5620a40eca40 0x5620a40eee30 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2234036eb0 msgr2=0x7f2234039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2234036eb0 0x7f2234039370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5620a40b8450 tx=0x7f235c088040 comp rx=0 tx=0).stop 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 msgr2=0x5620a40dad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f236b7f4640 1 -- 172.21.15.63:0/1811737095 reap_dead start 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40dad60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2364012a70 tx=0x7f236404cbb0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 shutdown_connections 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1811737095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40d3790 0x5620a40dad60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 >> 172.21.15.63:0/1811737095 conn(0x5620a40cfaf0 msgr2=0x5620a40cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 shutdown_connections 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1811737095 wait complete. 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2244036e60 msgr2=0x7f2244039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2244036e60 0x7f2244039320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f235c077d90 tx=0x7f235c08ca80 comp rx=0 tx=0).stop 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40c6490 msgr2=0x5620a40c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40c6490 0x5620a40c68b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x5620a40b2910 tx=0x7f23640178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 shutdown_connections 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2244036e60 0x7f2244039320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 --2- 172.21.15.63:0/1867227430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a40c6490 0x5620a40c68b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 >> 172.21.15.63:0/1867227430 conn(0x5620a40ba530 msgr2=0x5620a40ba930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 shutdown_connections 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.862+0000 7f2370333800 1 -- 172.21.15.63:0/1867227430 wait complete. 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620a40b5700 msgr2=0x5620a40b7af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620a40b5700 0x5620a40b7af0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a3fb54c0 msgr2=0x5620a40b1f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a3fb54c0 0x5620a40b1f30 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2254036e60 msgr2=0x7f2254039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2254036e60 0x7f2254039320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2360000f60 tx=0x7f23600012b0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 msgr2=0x5620a40aacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40aacf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f235c052740 tx=0x7f235c003020 comp rx=0 tx=0).stop 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 shutdown_connections 2024-04-23T19:11:25.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5620a40b5700 0x5620a40b7af0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5620a3fb54c0 0x5620a40b1f30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2254036e60 0x7f2254039320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 --2- 172.21.15.63:0/1699866137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a3fb2b90 0x5620a40aacf0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 >> 172.21.15.63:0/1699866137 conn(0x5620a3fac600 msgr2=0x5620a409d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 shutdown_connections 2024-04-23T19:11:25.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.863+0000 7f2370333800 1 -- 172.21.15.63:0/1699866137 wait complete. 2024-04-23T19:11:25.873 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{}},{"bucket":"myfoo2","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11,"bytes_processed":0,"bytes_returned":0}}]} 2024-04-23T19:11:25.873 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:11:25.873 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-23T19:11:25.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-23T19:11:25.874 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-04-23T19:11:25.919 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:25.920 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:25.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80ec613800 1 Processor -- start 2024-04-23T19:11:25.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80ec613800 1 -- start start 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80ec613800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5602cc6b0a70 0x5602cc6b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80ec613800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5602cc5647f0 con 0x5602cc6b13d0 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80ec613800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5602cc553e20 con 0x5602cc6b0a70 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80e7d47640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5602cc6b0a70 0x5602cc6b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80e7d47640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5602cc6b0a70 0x5602cc6b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38026/0 (socket says 172.21.15.63:38026) 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80e7d47640 1 -- 172.21.15.63:0/4254396443 learned_addr learned my addr 172.21.15.63:0/4254396443 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.919+0000 7f80e7546640 1 --1- 172.21.15.63:0/4254396443 >> v1:172.21.15.16:6789/0 conn(0x5602cc6b13d0 0x5602cc6b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:59004/0 (socket says 172.21.15.63:59004) 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.920+0000 7f80e7d47640 1 -- 172.21.15.63:0/4254396443 >> v1:172.21.15.16:6789/0 conn(0x5602cc6b13d0 legacy=0x5602cc6b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.920+0000 7f80e7d47640 1 -- 172.21.15.63:0/4254396443 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cc572b50 con 0x5602cc6b0a70 2024-04-23T19:11:25.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80e7d47640 1 --2- 172.21.15.63:0/4254396443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc6b0e90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f80d800a520 tx=0x7f80d801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3bd57de58c77f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80e6d45640 1 -- 172.21.15.63:0/4254396443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8057070 con 0x5602cc6b0a70 2024-04-23T19:11:25.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80e6d45640 1 -- 172.21.15.63:0/4254396443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d8052570 con 0x5602cc6b0a70 2024-04-23T19:11:25.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80e6d45640 1 -- 172.21.15.63:0/4254396443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8052870 con 0x5602cc6b0a70 2024-04-23T19:11:25.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80ec613800 1 -- 172.21.15.63:0/4254396443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 msgr2=0x5602cc6b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80ec613800 1 --2- 172.21.15.63:0/4254396443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc6b0e90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f80d800a520 tx=0x7f80d801d010 comp rx=0 tx=0).stop 2024-04-23T19:11:25.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80ec613800 1 -- 172.21.15.63:0/4254396443 shutdown_connections 2024-04-23T19:11:25.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80ec613800 1 --2- 172.21.15.63:0/4254396443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc6b0e90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80ec613800 1 -- 172.21.15.63:0/4254396443 >> 172.21.15.63:0/4254396443 conn(0x5602cc6ac630 msgr2=0x5602cc6aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80ec613800 1 -- 172.21.15.63:0/4254396443 shutdown_connections 2024-04-23T19:11:25.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.922+0000 7f80ec613800 1 -- 172.21.15.63:0/4254396443 wait complete. 2024-04-23T19:11:25.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.935+0000 7f80ec613800 1 Processor -- start 2024-04-23T19:11:25.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.935+0000 7f80ec613800 1 -- start start 2024-04-23T19:11:25.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.935+0000 7f80ec613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.935+0000 7f80ec613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5602cc5fd6f0 con 0x5602cc6b0a70 2024-04-23T19:11:25.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f80e7d47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f80e7d47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38030/0 (socket says 172.21.15.63:38030) 2024-04-23T19:11:25.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f80e7d47640 1 -- 172.21.15.63:0/3751138052 learned_addr learned my addr 172.21.15.63:0/3751138052 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f80e7d47640 1 -- 172.21.15.63:0/3751138052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cc7a5c60 con 0x5602cc6b0a70 2024-04-23T19:11:25.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f80e7d47640 1 --2- 172.21.15.63:0/3751138052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7a5690 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f80d801d050 tx=0x7f80d804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe4cae56bf6cb44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f7fd5e0b640 1 -- 172.21.15.63:0/3751138052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8057040 con 0x5602cc6b0a70 2024-04-23T19:11:25.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f7fd5e0b640 1 -- 172.21.15.63:0/3751138052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d8052850 con 0x5602cc6b0a70 2024-04-23T19:11:25.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.936+0000 7f7fd5e0b640 1 -- 172.21.15.63:0/3751138052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8052b70 con 0x5602cc6b0a70 2024-04-23T19:11:25.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 -- 172.21.15.63:0/3751138052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 msgr2=0x5602cc7a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 --2- 172.21.15.63:0/3751138052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7a5690 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f80d801d050 tx=0x7f80d804c000 comp rx=0 tx=0).stop 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 -- 172.21.15.63:0/3751138052 shutdown_connections 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 --2- 172.21.15.63:0/3751138052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7a5690 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 -- 172.21.15.63:0/3751138052 >> 172.21.15.63:0/3751138052 conn(0x5602cc6ac630 msgr2=0x5602cc7a0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 -- 172.21.15.63:0/3751138052 shutdown_connections 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 -- 172.21.15.63:0/3751138052 wait complete. 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 Processor -- start 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.937+0000 7f80ec613800 1 -- start start 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f80ec613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f80ec613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5602cc6b21d0 con 0x5602cc6b0a70 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f80e7d47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f80e7d47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38042/0 (socket says 172.21.15.63:38042) 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f80e7d47640 1 -- 172.21.15.63:0/145708406 learned_addr learned my addr 172.21.15.63:0/145708406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f80e7d47640 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cc7ad590 con 0x5602cc6b0a70 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f80e7d47640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7ad050 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005930 tx=0x7f80d8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.938+0000 7f7fcffff640 1 -- 172.21.15.63:0/145708406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8073070 con 0x5602cc6b0a70 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.939+0000 7f7fcffff640 1 -- 172.21.15.63:0/145708406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d805d650 con 0x5602cc6b0a70 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.939+0000 7f7fcffff640 1 -- 172.21.15.63:0/145708406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8057070 con 0x5602cc6b0a70 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.939+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602cc7ad820 con 0x5602cc6b0a70 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.939+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602cc7b04d0 con 0x5602cc6b0a70 2024-04-23T19:11:25.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.939+0000 7f7fcffff640 1 -- 172.21.15.63:0/145708406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f80d8076070 con 0x5602cc6b0a70 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f7fcffff640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fd0036eb0 0x7f7fd0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f7fcffff640 1 -- 172.21.15.63:0/145708406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f80d808e610 con 0x5602cc6b0a70 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80e7546640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fd0036eb0 0x7f7fd0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc6b13d0 0x5602cc7b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7b4810 con 0x5602cc6b13d0 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80e8548640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc6b13d0 0x5602cc7b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80e7546640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fd0036eb0 0x7f7fd0039370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80dc000f60 tx=0x7f80dc001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80e8548640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc6b13d0 0x5602cc7b4290 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80e8548640 1 -- 172.21.15.63:0/145708406 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f80e000a4a0 con 0x5602cc6b13d0 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.940+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7b1860 con 0x5602cc6b13d0 2024-04-23T19:11:25.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.941+0000 7f80e8548640 1 -- 172.21.15.63:0/145708406 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f80e000a4a0 con 0x5602cc6b13d0 2024-04-23T19:11:25.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.941+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7b6680 con 0x5602cc6b13d0 2024-04-23T19:11:25.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.941+0000 7f80e8548640 1 -- 172.21.15.63:0/145708406 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f80e000a4a0 con 0x5602cc6b13d0 2024-04-23T19:11:25.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.941+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7b59c0 con 0x5602cc6b13d0 2024-04-23T19:11:25.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.941+0000 7f80e8548640 1 -- 172.21.15.63:0/145708406 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f80e000a4a0 con 0x5602cc6b13d0 2024-04-23T19:11:25.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.941+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5602cc7b7a30 0x5602cc7b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.941+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7ba3a0 con 0x5602cc7b7a30 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.942+0000 7f80e7d47640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5602cc7b7a30 0x5602cc7b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.942+0000 7f80e7d47640 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5602cc7b7a30 0x5602cc7b9e20 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80e7d47640 1 -- 172.21.15.63:0/145708406 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f80d8075030 con 0x5602cc7b7a30 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80ec613800 1 Processor -- start 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80ec613800 1 -- start start 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80ec613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7c87c0 0x5602cc7c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80ec613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5602cc5a3e30 con 0x5602cc7c87c0 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80e8548640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7c87c0 0x5602cc7c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80e8548640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7c87c0 0x5602cc7c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38056/0 (socket says 172.21.15.63:38056) 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.943+0000 7f80e8548640 1 -- 172.21.15.63:0/562011115 learned_addr learned my addr 172.21.15.63:0/562011115 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.944+0000 7f80e8548640 1 -- 172.21.15.63:0/562011115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cc7c9120 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.944+0000 7f80e8548640 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7c87c0 0x5602cc7c8be0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x5602cc7ae450 tx=0x7f80e00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.944+0000 7f7fcd7fa640 1 -- 172.21.15.63:0/562011115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e0013440 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.944+0000 7f7fcd7fa640 1 -- 172.21.15.63:0/562011115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e00135e0 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.944+0000 7f7fcd7fa640 1 -- 172.21.15.63:0/562011115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e00138c0 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.944+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602cc7c9b90 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.944+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602cc7ca090 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f7fcd7fa640 1 -- 172.21.15.63:0/562011115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f80e0011040 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f7fcd7fa640 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fc0036e60 0x7f7fc0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f7fcd7fa640 1 -- 172.21.15.63:0/562011115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f80e001d070 con 0x5602cc7c87c0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f80e7d47640 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fc0036e60 0x7f7fc0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f80ec613800 1 Processor -- start 2024-04-23T19:11:25.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f80ec613800 1 -- start start 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f80e7d47640 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fc0036e60 0x7f7fc0039320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f80d8052a70 tx=0x7f80d806ff80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f80ec613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7d5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.945+0000 7f80ec613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5602cc7a5f20 con 0x5602cc7d5b70 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f80e8548640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7d5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f80e8548640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7d5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38068/0 (socket says 172.21.15.63:38068) 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f80e8548640 1 -- 172.21.15.63:0/103702766 learned_addr learned my addr 172.21.15.63:0/103702766 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f80e8548640 1 -- 172.21.15.63:0/103702766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cc7d6560 con 0x5602cc7d5b70 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f80e8548640 1 --2- 172.21.15.63:0/103702766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7d5f90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f80e004ccb0 tx=0x7f80e0012ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b651a55a7f0c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f7fc6ffd640 1 -- 172.21.15.63:0/103702766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e0012df0 con 0x5602cc7d5b70 2024-04-23T19:11:25.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f7fc6ffd640 1 -- 172.21.15.63:0/103702766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e0001310 con 0x5602cc7d5b70 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.946+0000 7f7fc6ffd640 1 -- 172.21.15.63:0/103702766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e0001630 con 0x5602cc7d5b70 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 -- 172.21.15.63:0/103702766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 msgr2=0x5602cc7d5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 --2- 172.21.15.63:0/103702766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7d5f90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f80e004ccb0 tx=0x7f80e0012ac0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 -- 172.21.15.63:0/103702766 shutdown_connections 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 --2- 172.21.15.63:0/103702766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7d5f90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 -- 172.21.15.63:0/103702766 >> 172.21.15.63:0/103702766 conn(0x5602cc7d1e40 msgr2=0x5602cc7cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 -- 172.21.15.63:0/103702766 shutdown_connections 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 -- 172.21.15.63:0/103702766 wait complete. 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 Processor -- start 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 -- start start 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7dd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80ec613800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5602cc7d6cb0 con 0x5602cc7d5b70 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80e8548640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80e8548640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38082/0 (socket says 172.21.15.63:38082) 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.947+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 learned_addr learned my addr 172.21.15.63:0/821013511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.948+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cc7dd5c0 con 0x5602cc7d5b70 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.948+0000 7f80e8548640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7dd080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f80e0012c10 tx=0x7f80e004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.948+0000 7f7fc57fa640 1 -- 172.21.15.63:0/821013511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e0021c60 con 0x5602cc7d5b70 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.948+0000 7f7fc57fa640 1 -- 172.21.15.63:0/821013511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e0021e00 con 0x5602cc7d5b70 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.948+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602cc7dd8b0 con 0x5602cc7d5b70 2024-04-23T19:11:25.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.948+0000 7f7fc57fa640 1 -- 172.21.15.63:0/821013511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e002a350 con 0x5602cc7d5b70 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.948+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602cc7e0500 con 0x5602cc7d5b70 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f7fc57fa640 1 -- 172.21.15.63:0/821013511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f80e0040040 con 0x5602cc7d5b70 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f7fc57fa640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fb0036e60 0x7f7fb0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f7fc57fa640 1 -- 172.21.15.63:0/821013511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f80e000e040 con 0x5602cc7d5b70 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f80e7d47640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fb0036e60 0x7f7fb0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc7e8df0 0x5602cc7eb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7eb9e0 con 0x5602cc7e8df0 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.949+0000 7f80e7546640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc7e8df0 0x5602cc7eb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80e7d47640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fb0036e60 0x7f7fb0039320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x5602cc7ba780 tx=0x7f80d8091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80e7546640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc7e8df0 0x5602cc7eb290 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 20 realm 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7e8300 con 0x5602cc7e8df0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7ec4e0 con 0x5602cc7e8df0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.950+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7ecdb0 con 0x5602cc7e8df0 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80ec613800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7ed150 con 0x5602cc7e8df0 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80ec613800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80ec613800 20 searching for the correct realm 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5602cc7ef030 0x5602cc7f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80e8548640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5602cc7ef030 0x5602cc7f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.951+0000 7f80e8548640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5602cc7ef030 0x5602cc7f1420 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e8548640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f80e005b040 0x7f80e002c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80e005b040 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e7d47640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f80e005b040 0x7f80e002c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e7d47640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f80e005b040 0x7f80e002c450 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e7d47640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f80d8078060 0x7f80d80773b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:25.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.952+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.953+0000 7f80e7546640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f80d8078060 0x7f80d80773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.953+0000 7f80e7546640 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f80d8078060 0x7f80d80773b0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.953+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.953+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.954+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80e005b040 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.955+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.955+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.955+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.955+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.955+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.955+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.955+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80e005b040 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.956+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.957+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.957+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.957+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.957+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.957+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.957+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.958+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.958+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.958+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.958+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.958+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.958+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80e005b040 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80e005b040 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.959+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80e005b040 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7ef030 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.960+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80d8078060 2024-04-23T19:11:25.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x5602cc7e8df0 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1ad0 con 0x7f80e005b040 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:25.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7edb00 con 0x5602cc7e8df0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.961+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7ee3a0 con 0x5602cc7e8df0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7ed740 con 0x7f80d8078060 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 20 zone default found 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 4 Realm: () 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 20 zonegroup default 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.962+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f1f30 con 0x7f80e005b040 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f4160 con 0x5602cc7ef030 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f4500 con 0x5602cc7ef030 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 20 started sync module instance, tier type = 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.963+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f35f0 con 0x7f80e005b040 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.964+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.964+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94570020813328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f3250 con 0x7f80e005b040 2024-04-23T19:11:25.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.964+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f5370 con 0x5602cc7e8df0 2024-04-23T19:11:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.965+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94570020813328] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.965+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.966+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94570020818624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f5790 con 0x5602cc7e8df0 2024-04-23T19:11:25.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.966+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f6150 con 0x5602cc7ef030 2024-04-23T19:11:25.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.966+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94570020818624] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.967+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.967+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94570020822176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f64f0 con 0x5602cc7ef030 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.967+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f6fa0 con 0x7f80e005b040 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.968+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94570020822176] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.968+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.968+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94570020819680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f7390 con 0x7f80e005b040 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.968+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f7dd0 con 0x5602cc7e8df0 2024-04-23T19:11:25.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.969+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.969+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94570020829344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f7d90 con 0x5602cc7e8df0 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.969+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f8da0 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.969+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94570020819680] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.970+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94570020829344] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.970+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.970+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94570020825728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f8da0 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.970+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f8240 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.971+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94570020825728] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.971+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.971+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94570020831744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f9190 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.971+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f9bf0 con 0x7f80d8078060 2024-04-23T19:11:25.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94570020831744] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94570020833376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f9bf0 con 0x7f80d8078060 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 20 add_watcher() i=0 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 20 add_watcher() i=1 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 20 add_watcher() i=2 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 20 add_watcher() i=3 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 20 add_watcher() i=4 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 20 add_watcher() i=5 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.972+0000 7f80ec613800 20 add_watcher() i=6 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.973+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94570020833376] v32'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.973+0000 7f80ec613800 20 add_watcher() i=7 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.973+0000 7f80ec613800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.973+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7fd020 con 0x5602cc7e8df0 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.974+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.974+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94570020837072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7fa4e0 con 0x5602cc7e8df0 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94570020837072] v32'206 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f7f9dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f80ec613800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f80ec613800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f80ec613800 5 note: GC not initialized 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f7f9cfe1640 20 reqs_thread_entry: start 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f80ec613800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f80ec613800 20 Filter name: none 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.975+0000 7f7f167fc640 20 reqs_thread_entry: start 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.976+0000 7f80ec613800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.976+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc830060 con 0x5602cc7e8df0 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.976+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.976+0000 7f80ec613800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.976+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5602cc831450 con 0x7f80d8078060 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.976+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.977+0000 7f7f9dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.977+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94570020837072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7fc7d0 con 0x5602cc7e8df0 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.978+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94570020837072] v32'207 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.978+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94570020813328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f96d0 con 0x7f80e005b040 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.979+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94570020813328] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.979+0000 7f80ec613800 20 remove_watcher() i=0 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.979+0000 7f80ec613800 2 removed watcher, disabling cache 2024-04-23T19:11:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.979+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94570020818624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f96d0 con 0x5602cc7e8df0 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.980+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94570020818624] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.980+0000 7f80ec613800 20 remove_watcher() i=1 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.980+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94570020822176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f96d0 con 0x5602cc7ef030 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.981+0000 7f80e8548640 1 -- 172.21.15.63:0/821013511 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94570020822176] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80e0029070 con 0x5602cc7ef030 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.981+0000 7f80ec613800 20 remove_watcher() i=2 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.981+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94570020819680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f5ca0 con 0x7f80e005b040 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.982+0000 7f80e7d47640 1 -- 172.21.15.63:0/821013511 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94570020819680] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80d8075030 con 0x7f80e005b040 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.982+0000 7f80ec613800 20 remove_watcher() i=3 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.982+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94570020829344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f5ca0 con 0x5602cc7e8df0 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.983+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94570020829344] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x5602cc7e8df0 2024-04-23T19:11:26.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.983+0000 7f80ec613800 20 remove_watcher() i=4 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.983+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94570020825728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f5ca0 con 0x7f80d8078060 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.984+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94570020825728] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.984+0000 7f80ec613800 20 remove_watcher() i=5 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.984+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94570020831744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f5ca0 con 0x7f80d8078060 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.985+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94570020831744] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.985+0000 7f80ec613800 20 remove_watcher() i=6 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.985+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94570020833376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5602cc7f8200 con 0x7f80d8078060 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94570020833376] v32'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f80dc00f610 con 0x7f80d8078060 2024-04-23T19:11:26.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 20 remove_watcher() i=7 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f80d8078060 msgr2=0x7f80d80773b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f80d8078060 0x7f80d80773b0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc7e8df0 msgr2=0x5602cc7eb290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc7e8df0 0x5602cc7eb290 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f80e005b040 msgr2=0x7f80e002c450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f80e005b040 0x7f80e002c450 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5602cc7ef030 msgr2=0x5602cc7f1420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5602cc7ef030 0x5602cc7f1420 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fb0036e60 msgr2=0x7f7fb0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fb0036e60 0x7f7fb0039320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x5602cc7ba780 tx=0x7f80d8091000 comp rx=0 tx=0).stop 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 msgr2=0x5602cc7dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 --2- 172.21.15.63:0/821013511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7d5b70 0x5602cc7dd080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f80e0012c10 tx=0x7f80e004cc40 comp rx=0 tx=0).stop 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 reap_dead start 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80e7546640 1 -- 172.21.15.63:0/821013511 reap_dead start 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.986+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 shutdown_connections 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 >> 172.21.15.63:0/821013511 conn(0x5602cc7d1e40 msgr2=0x5602cc7cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 shutdown_connections 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/821013511 wait complete. 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fc0036e60 msgr2=0x7f7fc0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fc0036e60 0x7f7fc0039320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f80d8052a70 tx=0x7f80d806ff80 comp rx=0 tx=0).stop 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7c87c0 msgr2=0x5602cc7c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7c87c0 0x5602cc7c8be0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x5602cc7ae450 tx=0x7f80e00178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 shutdown_connections 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fc0036e60 0x7f7fc0039320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/562011115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc7c87c0 0x5602cc7c8be0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 >> 172.21.15.63:0/562011115 conn(0x5602cc7bc860 msgr2=0x5602cc7bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 shutdown_connections 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/562011115 wait complete. 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5602cc7b7a30 msgr2=0x5602cc7b9e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5602cc7b7a30 0x5602cc7b9e20 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc6b13d0 msgr2=0x5602cc7b4290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc6b13d0 0x5602cc7b4290 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fd0036eb0 msgr2=0x7f7fd0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fd0036eb0 0x7f7fd0039370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80dc000f60 tx=0x7f80dc001370 comp rx=0 tx=0).stop 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 msgr2=0x5602cc7ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7ad050 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005930 tx=0x7f80d8003020 comp rx=0 tx=0).stop 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 shutdown_connections 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5602cc7b7a30 0x5602cc7b9e20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5602cc6b13d0 0x5602cc7b4290 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7fd0036eb0 0x7f7fd0039370 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 --2- 172.21.15.63:0/145708406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602cc6b0a70 0x5602cc7ad050 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.987+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 >> 172.21.15.63:0/145708406 conn(0x5602cc6ac630 msgr2=0x5602cc79f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.988+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 shutdown_connections 2024-04-23T19:11:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:25.988+0000 7f80ec613800 1 -- 172.21.15.63:0/145708406 wait complete. 2024-04-23T19:11:26.008 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"s3select":{}},{"bucket":"myfoo2","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-04-23T19:11:26.008 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:11:26.009 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-23T19:11:26.009 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-23T19:11:26.009 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-04-23T19:11:26.057 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:26.057 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:26.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.056+0000 7f3228610800 1 Processor -- start 2024-04-23T19:11:26.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.056+0000 7f3228610800 1 -- start start 2024-04-23T19:11:26.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.056+0000 7f3228610800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e98a357580 0x55e98a360080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.056+0000 7f3228610800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e98a2097f0 con 0x55e98a355a40 2024-04-23T19:11:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.056+0000 7f3228610800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55e98a1f8e20 con 0x55e98a357580 2024-04-23T19:11:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.057+0000 7f32233ed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e98a357580 0x55e98a360080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.057+0000 7f3223bee640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55e98a355a40 0x55e98a355e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:59016/0 (socket says 172.21.15.63:59016) 2024-04-23T19:11:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.057+0000 7f3223bee640 1 -- 172.21.15.63:0/3796687334 learned_addr learned my addr 172.21.15.63:0/3796687334 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.057+0000 7f32233ed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e98a357580 0x55e98a360080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38098/0 (socket says 172.21.15.63:38098) 2024-04-23T19:11:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.057+0000 7f32233ed640 1 -- 172.21.15.63:0/3796687334 >> v1:172.21.15.16:6789/0 conn(0x55e98a355a40 legacy=0x55e98a355e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.057+0000 7f32233ed640 1 -- 172.21.15.63:0/3796687334 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98a217b50 con 0x55e98a357580 2024-04-23T19:11:26.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f32233ed640 1 --2- 172.21.15.63:0/3796687334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a357580 0x55e98a360080 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f321800a470 tx=0x7f321801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d60625b903244da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3222bec640 1 -- 172.21.15.63:0/3796687334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3218057070 con 0x55e98a357580 2024-04-23T19:11:26.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3222bec640 1 -- 172.21.15.63:0/3796687334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3218052440 con 0x55e98a357580 2024-04-23T19:11:26.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3222bec640 1 -- 172.21.15.63:0/3796687334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3218052740 con 0x55e98a357580 2024-04-23T19:11:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3228610800 1 -- 172.21.15.63:0/3796687334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a357580 msgr2=0x55e98a360080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3228610800 1 --2- 172.21.15.63:0/3796687334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a357580 0x55e98a360080 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f321800a470 tx=0x7f321801ce90 comp rx=0 tx=0).stop 2024-04-23T19:11:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3228610800 1 -- 172.21.15.63:0/3796687334 shutdown_connections 2024-04-23T19:11:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3228610800 1 --2- 172.21.15.63:0/3796687334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a357580 0x55e98a360080 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.059+0000 7f3228610800 1 -- 172.21.15.63:0/3796687334 >> 172.21.15.63:0/3796687334 conn(0x55e98a351600 msgr2=0x55e98a353a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.060+0000 7f3228610800 1 -- 172.21.15.63:0/3796687334 shutdown_connections 2024-04-23T19:11:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.060+0000 7f3228610800 1 -- 172.21.15.63:0/3796687334 wait complete. 2024-04-23T19:11:26.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.072+0000 7f3228610800 1 Processor -- start 2024-04-23T19:11:26.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.072+0000 7f3228610800 1 -- start start 2024-04-23T19:11:26.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.072+0000 7f3228610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a44a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.072+0000 7f3228610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e98a2a26f0 con 0x55e98a355a40 2024-04-23T19:11:26.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3223bee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a44a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3223bee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a44a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38110/0 (socket says 172.21.15.63:38110) 2024-04-23T19:11:26.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3223bee640 1 -- 172.21.15.63:0/2177668183 learned_addr learned my addr 172.21.15.63:0/2177668183 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3223bee640 1 -- 172.21.15.63:0/2177668183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98a44ad20 con 0x55e98a355a40 2024-04-23T19:11:26.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3223bee640 1 --2- 172.21.15.63:0/2177668183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a44a750 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3214001d50 tx=0x7f321400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0333bca066b5e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3111e0b640 1 -- 172.21.15.63:0/2177668183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3214013030 con 0x55e98a355a40 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3111e0b640 1 -- 172.21.15.63:0/2177668183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321400e470 con 0x55e98a355a40 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.073+0000 7f3111e0b640 1 -- 172.21.15.63:0/2177668183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321400e810 con 0x55e98a355a40 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 -- 172.21.15.63:0/2177668183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 msgr2=0x55e98a44a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 --2- 172.21.15.63:0/2177668183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a44a750 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3214001d50 tx=0x7f321400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 -- 172.21.15.63:0/2177668183 shutdown_connections 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 --2- 172.21.15.63:0/2177668183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a44a750 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 -- 172.21.15.63:0/2177668183 >> 172.21.15.63:0/2177668183 conn(0x55e98a351600 msgr2=0x55e98a446070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 -- 172.21.15.63:0/2177668183 shutdown_connections 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 -- 172.21.15.63:0/2177668183 wait complete. 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.074+0000 7f3228610800 1 Processor -- start 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3228610800 1 -- start start 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3228610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a452360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3228610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e98a3628a0 con 0x55e98a355a40 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3223bee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a452360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3223bee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a452360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38120/0 (socket says 172.21.15.63:38120) 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3223bee640 1 -- 172.21.15.63:0/1584690551 learned_addr learned my addr 172.21.15.63:0/1584690551 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3223bee640 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98a4528a0 con 0x55e98a355a40 2024-04-23T19:11:26.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.075+0000 7f3223bee640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a452360 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55e98a44b4e0 tx=0x7f3214000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.076+0000 7f310bfff640 1 -- 172.21.15.63:0/1584690551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3214027070 con 0x55e98a355a40 2024-04-23T19:11:26.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.076+0000 7f310bfff640 1 -- 172.21.15.63:0/1584690551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214019900 con 0x55e98a355a40 2024-04-23T19:11:26.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.076+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e98a452b30 con 0x55e98a355a40 2024-04-23T19:11:26.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.076+0000 7f310bfff640 1 -- 172.21.15.63:0/1584690551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3214019c40 con 0x55e98a355a40 2024-04-23T19:11:26.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.076+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98a455740 con 0x55e98a355a40 2024-04-23T19:11:26.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f310bfff640 1 -- 172.21.15.63:0/1584690551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f321402a070 con 0x55e98a355a40 2024-04-23T19:11:26.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f310bfff640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f310c036eb0 0x7f310c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f310bfff640 1 -- 172.21.15.63:0/1584690551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f321401f070 con 0x55e98a355a40 2024-04-23T19:11:26.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f32233ed640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f310c036eb0 0x7f310c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a357580 0x55e98a459500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a459a80 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f32243ef640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a357580 0x55e98a459500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f32233ed640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f310c036eb0 0x7f310c039370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3218054000 tx=0x7f321804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.077+0000 7f32243ef640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a357580 0x55e98a459500 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.078+0000 7f32243ef640 1 -- 172.21.15.63:0/1584690551 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.078+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a456ad0 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.078+0000 7f32243ef640 1 -- 172.21.15.63:0/1584690551 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.078+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a45b880 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.078+0000 7f32243ef640 1 -- 172.21.15.63:0/1584690551 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.078+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a45abc0 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.079+0000 7f32243ef640 1 -- 172.21.15.63:0/1584690551 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a357580 2024-04-23T19:11:26.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.079+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e98a45cc30 0x55e98a45f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.079+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a45f5a0 con 0x55e98a45cc30 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.079+0000 7f3223bee640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e98a45cc30 0x55e98a45f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.079+0000 7f3223bee640 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e98a45cc30 0x55e98a45f020 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.079+0000 7f3223bee640 1 -- 172.21.15.63:0/1584690551 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3214029030 con 0x55e98a45cc30 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f3228610800 1 Processor -- start 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f3228610800 1 -- start start 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f3228610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a46d9c0 0x55e98a46dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f3228610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e98a248e30 con 0x55e98a46d9c0 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f32243ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a46d9c0 0x55e98a46dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f32243ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a46d9c0 0x55e98a46dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38136/0 (socket says 172.21.15.63:38136) 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f32243ef640 1 -- 172.21.15.63:0/72370000 learned_addr learned my addr 172.21.15.63:0/72370000 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.080+0000 7f32243ef640 1 -- 172.21.15.63:0/72370000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98a46e320 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.081+0000 7f32243ef640 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a46d9c0 0x55e98a46dde0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55e98a459e40 tx=0x7f321c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.081+0000 7f31097fa640 1 -- 172.21.15.63:0/72370000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c013440 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.081+0000 7f31097fa640 1 -- 172.21.15.63:0/72370000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c0135e0 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.081+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e98a46ed90 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.081+0000 7f31097fa640 1 -- 172.21.15.63:0/72370000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c0138c0 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.081+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98a46f290 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.081+0000 7f31097fa640 1 -- 172.21.15.63:0/72370000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f321c011040 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f31097fa640 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30fc036eb0 0x7f30fc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f31097fa640 1 -- 172.21.15.63:0/72370000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f321c01d070 con 0x55e98a46d9c0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f3223bee640 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30fc036eb0 0x7f30fc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f3228610800 1 Processor -- start 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f3223bee640 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30fc036eb0 0x7f30fc039370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f321400c500 tx=0x7f321404b0c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f3228610800 1 -- start start 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f3228610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a47b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.082+0000 7f3228610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e98a44b0a0 con 0x55e98a47acb0 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f32243ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a47b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f32243ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a47b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38144/0 (socket says 172.21.15.63:38144) 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f32243ef640 1 -- 172.21.15.63:0/739561956 learned_addr learned my addr 172.21.15.63:0/739561956 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f32243ef640 1 -- 172.21.15.63:0/739561956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98a47b6a0 con 0x55e98a47acb0 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f32243ef640 1 --2- 172.21.15.63:0/739561956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a47b0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f321c04ccf0 tx=0x7f321c02b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924d495bd6e3aa35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f3102ffd640 1 -- 172.21.15.63:0/739561956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c02bc90 con 0x55e98a47acb0 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f3102ffd640 1 -- 172.21.15.63:0/739561956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c02be30 con 0x55e98a47acb0 2024-04-23T19:11:26.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f3102ffd640 1 -- 172.21.15.63:0/739561956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c02b4f0 con 0x55e98a47acb0 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f3228610800 1 -- 172.21.15.63:0/739561956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 msgr2=0x55e98a47b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.083+0000 7f3228610800 1 --2- 172.21.15.63:0/739561956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a47b0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f321c04ccf0 tx=0x7f321c02b8e0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 -- 172.21.15.63:0/739561956 shutdown_connections 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 --2- 172.21.15.63:0/739561956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a47b0d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 -- 172.21.15.63:0/739561956 >> 172.21.15.63:0/739561956 conn(0x55e98a476f70 msgr2=0x55e98a475020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 -- 172.21.15.63:0/739561956 shutdown_connections 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 -- 172.21.15.63:0/739561956 wait complete. 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 Processor -- start 2024-04-23T19:11:26.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 -- start start 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a482270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f3228610800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e98a47be20 con 0x55e98a47acb0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f32243ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a482270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f32243ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a482270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38150/0 (socket says 172.21.15.63:38150) 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.084+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 learned_addr learned my addr 172.21.15.63:0/1005225235 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.085+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98a4827b0 con 0x55e98a47acb0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.085+0000 7f32243ef640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a482270 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f321c02aff0 tx=0x7f321c04cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.085+0000 7f31017fa640 1 -- 172.21.15.63:0/1005225235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c013a60 con 0x55e98a47acb0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.085+0000 7f31017fa640 1 -- 172.21.15.63:0/1005225235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c013c00 con 0x55e98a47acb0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.085+0000 7f31017fa640 1 -- 172.21.15.63:0/1005225235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c02a350 con 0x55e98a47acb0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.085+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e98a482a40 con 0x55e98a47acb0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.085+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98a4856f0 con 0x55e98a47acb0 2024-04-23T19:11:26.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.086+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.086+0000 7f31017fa640 1 -- 172.21.15.63:0/1005225235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f321c02a4f0 con 0x55e98a47acb0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.086+0000 7f31017fa640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30ec036e60 0x7f30ec039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.086+0000 7f31017fa640 1 -- 172.21.15.63:0/1005225235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f321c032070 con 0x55e98a47acb0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3223bee640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30ec036e60 0x7f30ec039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a48dd70 0x55e98a490230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a490980 con 0x55e98a48dd70 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f32233ed640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a48dd70 0x55e98a490230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3223bee640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30ec036e60 0x7f30ec039320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3214017a60 tx=0x7f321403d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f32233ed640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a48dd70 0x55e98a490230 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3228610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3228610800 20 realm 2024-04-23T19:11:26.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.087+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a490d50 con 0x55e98a48dd70 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4915f0 con 0x55e98a48dd70 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a491990 con 0x55e98a48dd70 2024-04-23T19:11:26.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a491200 con 0x55e98a48dd70 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.088+0000 7f3228610800 20 searching for the correct realm 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e98a493e00 0x55e98a496290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f32243ef640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e98a493e00 0x55e98a496290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f32243ef640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e98a493e00 0x55e98a496290 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f32243ef640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f321c045780 0x7f321c0089b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321c045780 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.089+0000 7f3223bee640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f321c045780 0x7f321c0089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.090+0000 7f3223bee640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f321c045780 0x7f321c0089b0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.090+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.090+0000 7f3223bee640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f321402c060 0x7f321404c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.090+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.090+0000 7f32233ed640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f321402c060 0x7f321404c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.091+0000 7f32233ed640 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f321402c060 0x7f321404c7f0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.091+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.091+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.091+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.091+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.091+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.091+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321c045780 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.092+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.093+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.093+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.093+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.093+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.093+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.093+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321c045780 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.094+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.095+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321c045780 2024-04-23T19:11:26.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.096+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321c045780 2024-04-23T19:11:26.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.097+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.097+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.097+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.097+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.097+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.097+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321c045780 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.098+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.098+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a493e00 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.098+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.098+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321402c060 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.098+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.098+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x55e98a48dd70 2024-04-23T19:11:26.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496960 con 0x7f321c045780 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4928a0 con 0x55e98a48dd70 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a493140 con 0x55e98a48dd70 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.099+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4924e0 con 0x7f321402c060 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 zone default found 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 4 Realm: () 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:26.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 zonegroup default 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a496e30 con 0x7f321c045780 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a499060 con 0x55e98a493e00 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.100+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a499400 con 0x55e98a493e00 2024-04-23T19:11:26.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.101+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.101+0000 7f3228610800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.101+0000 7f3228610800 20 started sync module instance, tier type = 2024-04-23T19:11:26.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.101+0000 7f3228610800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:26.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.101+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4984a0 con 0x7f321c045780 2024-04-23T19:11:26.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.102+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.102+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94461535815872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a498100 con 0x7f321c045780 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.102+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49a160 con 0x55e98a48dd70 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.103+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94461535815872] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.103+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.103+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94461535819680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49a580 con 0x55e98a48dd70 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.103+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49afa0 con 0x55e98a493e00 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.104+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94461535819680] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.104+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.104+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94461535824528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49af60 con 0x55e98a493e00 2024-04-23T19:11:26.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.104+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49ba80 con 0x7f321c045780 2024-04-23T19:11:26.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.105+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94461535824528] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.105+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.105+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94461535827280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49ba40 con 0x7f321c045780 2024-04-23T19:11:26.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.105+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49c530 con 0x55e98a48dd70 2024-04-23T19:11:26.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.106+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.107+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94461535830080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49c530 con 0x55e98a48dd70 2024-04-23T19:11:26.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.107+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49d4d0 con 0x7f321402c060 2024-04-23T19:11:26.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.107+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94461535827280] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.108+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.108+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94461535834016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49d490 con 0x7f321402c060 2024-04-23T19:11:26.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.108+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49dee0 con 0x7f321402c060 2024-04-23T19:11:26.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.108+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94461535830080] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.109+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94461535834016] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.109+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.109+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94461535820736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49da30 con 0x7f321402c060 2024-04-23T19:11:26.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.109+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49e320 con 0x7f321402c060 2024-04-23T19:11:26.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94461535820736] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94461535832400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49dee0 con 0x7f321402c060 2024-04-23T19:11:26.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 20 add_watcher() i=0 2024-04-23T19:11:26.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 20 add_watcher() i=1 2024-04-23T19:11:26.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 20 add_watcher() i=2 2024-04-23T19:11:26.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 20 add_watcher() i=3 2024-04-23T19:11:26.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 20 add_watcher() i=4 2024-04-23T19:11:26.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 20 add_watcher() i=5 2024-04-23T19:11:26.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.110+0000 7f3228610800 20 add_watcher() i=6 2024-04-23T19:11:26.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.111+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94461535832400] v32'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.111+0000 7f3228610800 20 add_watcher() i=7 2024-04-23T19:11:26.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.111+0000 7f3228610800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:26.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.111+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4a1770 con 0x55e98a48dd70 2024-04-23T19:11:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.111+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.111+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94461535837136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49ec30 con 0x55e98a48dd70 2024-04-23T19:11:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.112+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94461535837136] v32'208 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.112+0000 7f30d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:26.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 5 note: GC not initialized 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f30d8fe1640 20 reqs_thread_entry: start 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 20 Filter name: none 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f30527fc640 20 reqs_thread_entry: start 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4d4660 con 0x55e98a48dd70 2024-04-23T19:11:26.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.113+0000 7f3228610800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:11:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.114+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4d5a50 con 0x7f321402c060 2024-04-23T19:11:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.114+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.115+0000 7f30d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.115+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94461535837136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4a0ec0 con 0x55e98a48dd70 2024-04-23T19:11:26.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.116+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94461535837136] v32'209 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.116+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94461535815872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49dba0 con 0x7f321c045780 2024-04-23T19:11:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.117+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94461535815872] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.117+0000 7f3228610800 20 remove_watcher() i=0 2024-04-23T19:11:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.117+0000 7f3228610800 2 removed watcher, disabling cache 2024-04-23T19:11:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.117+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94461535819680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4988c0 con 0x55e98a48dd70 2024-04-23T19:11:26.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.119+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94461535819680] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.119+0000 7f3228610800 20 remove_watcher() i=1 2024-04-23T19:11:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.119+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94461535824528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4988c0 con 0x55e98a493e00 2024-04-23T19:11:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.120+0000 7f32243ef640 1 -- 172.21.15.63:0/1005225235 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94461535824528] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321c00a4a0 con 0x55e98a493e00 2024-04-23T19:11:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.120+0000 7f3228610800 20 remove_watcher() i=2 2024-04-23T19:11:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.120+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94461535827280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49aa90 con 0x7f321c045780 2024-04-23T19:11:26.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.121+0000 7f3223bee640 1 -- 172.21.15.63:0/1005225235 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94461535827280] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321403b030 con 0x7f321c045780 2024-04-23T19:11:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.121+0000 7f3228610800 20 remove_watcher() i=3 2024-04-23T19:11:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.121+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94461535830080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a4988c0 con 0x55e98a48dd70 2024-04-23T19:11:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.122+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94461535830080] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x55e98a48dd70 2024-04-23T19:11:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.122+0000 7f3228610800 20 remove_watcher() i=4 2024-04-23T19:11:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.122+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94461535834016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49b550 con 0x7f321402c060 2024-04-23T19:11:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.123+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94461535834016] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.123+0000 7f3228610800 20 remove_watcher() i=5 2024-04-23T19:11:26.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.123+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94461535820736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49aa90 con 0x7f321402c060 2024-04-23T19:11:26.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.123+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94461535820736] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.124+0000 7f3228610800 20 remove_watcher() i=6 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.124+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94461535832400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e98a49aa90 con 0x7f321402c060 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.124+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94461535832400] v32'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f321805a040 con 0x7f321402c060 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.124+0000 7f3228610800 20 remove_watcher() i=7 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f321402c060 msgr2=0x7f321404c7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f321402c060 0x7f321404c7f0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a48dd70 msgr2=0x55e98a490230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a48dd70 0x55e98a490230 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f321c045780 msgr2=0x7f321c0089b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f321c045780 0x7f321c0089b0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e98a493e00 msgr2=0x55e98a496290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e98a493e00 0x55e98a496290 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30ec036e60 msgr2=0x7f30ec039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30ec036e60 0x7f30ec039320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3214017a60 tx=0x7f321403d000 comp rx=0 tx=0).stop 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 msgr2=0x55e98a482270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/1005225235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a47acb0 0x55e98a482270 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f321c02aff0 tx=0x7f321c04cc80 comp rx=0 tx=0).stop 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 reap_dead start 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f32233ed640 1 -- 172.21.15.63:0/1005225235 reap_dead start 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 shutdown_connections 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 >> 172.21.15.63:0/1005225235 conn(0x55e98a476f70 msgr2=0x55e98a474620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 shutdown_connections 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/1005225235 wait complete. 2024-04-23T19:11:26.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30fc036eb0 msgr2=0x7f30fc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30fc036eb0 0x7f30fc039370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f321400c500 tx=0x7f321404b0c0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a46d9c0 msgr2=0x55e98a46dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a46d9c0 0x55e98a46dde0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55e98a459e40 tx=0x7f321c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 shutdown_connections 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f30fc036eb0 0x7f30fc039370 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 --2- 172.21.15.63:0/72370000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a46d9c0 0x55e98a46dde0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 >> 172.21.15.63:0/72370000 conn(0x55e98a461a60 msgr2=0x55e98a461e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 shutdown_connections 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.125+0000 7f3228610800 1 -- 172.21.15.63:0/72370000 wait complete. 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e98a45cc30 msgr2=0x55e98a45f020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e98a45cc30 0x55e98a45f020 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a357580 msgr2=0x55e98a459500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a357580 0x55e98a459500 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f310c036eb0 msgr2=0x7f310c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f310c036eb0 0x7f310c039370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3218054000 tx=0x7f321804e000 comp rx=0 tx=0).stop 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 msgr2=0x55e98a452360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a452360 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55e98a44b4e0 tx=0x7f3214000bc0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 shutdown_connections 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e98a45cc30 0x55e98a45f020 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e98a357580 0x55e98a459500 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f310c036eb0 0x7f310c039370 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 --2- 172.21.15.63:0/1584690551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e98a355a40 0x55e98a452360 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 >> 172.21.15.63:0/1584690551 conn(0x55e98a351600 msgr2=0x55e98a444890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 shutdown_connections 2024-04-23T19:11:26.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.126+0000 7f3228610800 1 -- 172.21.15.63:0/1584690551 wait complete. 2024-04-23T19:11:26.150 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo2","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo3","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo4","time":"2024-04-23T19:00:00.000000Z","epoch":1713898800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4}],"total":{"bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-04-23T19:11:26.151 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:11:26.151 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-23T19:11:26.151 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-23T19:11:26.151 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-04-23T19:11:26.203 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:26.204 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:26.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7effd2d800 1 Processor -- start 2024-04-23T19:11:26.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7effd2d800 1 -- start start 2024-04-23T19:11:26.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7effd2d800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559f6947ca70 0x559f6947ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7effd2d800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559f693307f0 con 0x559f6947d3d0 2024-04-23T19:11:26.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7effd2d800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x559f6931fe20 con 0x559f6947ca70 2024-04-23T19:11:26.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7efb97f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559f6947ca70 0x559f6947ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7efb97f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x559f6947ca70 0x559f6947ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38160/0 (socket says 172.21.15.63:38160) 2024-04-23T19:11:26.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7efb97f640 1 -- 172.21.15.63:0/3899617811 learned_addr learned my addr 172.21.15.63:0/3899617811 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.203+0000 7f7efb17e640 1 --1- 172.21.15.63:0/3899617811 >> v1:172.21.15.16:6789/0 conn(0x559f6947d3d0 0x559f6947d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:59028/0 (socket says 172.21.15.63:59028) 2024-04-23T19:11:26.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.204+0000 7f7efb97f640 1 -- 172.21.15.63:0/3899617811 >> v1:172.21.15.16:6789/0 conn(0x559f6947d3d0 legacy=0x559f6947d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.204+0000 7f7efb97f640 1 -- 172.21.15.63:0/3899617811 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6933eb50 con 0x559f6947ca70 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.205+0000 7f7efb97f640 1 --2- 172.21.15.63:0/3899617811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f6947ce90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00a520 tx=0x7f7eec01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9909999fa665b0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7efa97d640 1 -- 172.21.15.63:0/3899617811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eec057070 con 0x559f6947ca70 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7efa97d640 1 -- 172.21.15.63:0/3899617811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec052570 con 0x559f6947ca70 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7efa97d640 1 -- 172.21.15.63:0/3899617811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eec052870 con 0x559f6947ca70 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7effd2d800 1 -- 172.21.15.63:0/3899617811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 msgr2=0x559f6947ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3899617811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f6947ce90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00a520 tx=0x7f7eec01d010 comp rx=0 tx=0).stop 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7effd2d800 1 -- 172.21.15.63:0/3899617811 shutdown_connections 2024-04-23T19:11:26.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3899617811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f6947ce90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7effd2d800 1 -- 172.21.15.63:0/3899617811 >> 172.21.15.63:0/3899617811 conn(0x559f69478630 msgr2=0x559f6947aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7effd2d800 1 -- 172.21.15.63:0/3899617811 shutdown_connections 2024-04-23T19:11:26.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.206+0000 7f7effd2d800 1 -- 172.21.15.63:0/3899617811 wait complete. 2024-04-23T19:11:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.219+0000 7f7effd2d800 1 Processor -- start 2024-04-23T19:11:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.219+0000 7f7effd2d800 1 -- start start 2024-04-23T19:11:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.219+0000 7f7effd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f695716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.219+0000 7f7effd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f693c9720 con 0x559f6947ca70 2024-04-23T19:11:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7efb97f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f695716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7efb97f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f695716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38162/0 (socket says 172.21.15.63:38162) 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7efb97f640 1 -- 172.21.15.63:0/243720136 learned_addr learned my addr 172.21.15.63:0/243720136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7efb97f640 1 -- 172.21.15.63:0/243720136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f69571cc0 con 0x559f6947ca70 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7efb97f640 1 --2- 172.21.15.63:0/243720136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f695716f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7eec01d050 tx=0x7f7eec04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7212fa0faf73f43c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7de9e0b640 1 -- 172.21.15.63:0/243720136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eec057040 con 0x559f6947ca70 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7de9e0b640 1 -- 172.21.15.63:0/243720136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec052850 con 0x559f6947ca70 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.220+0000 7f7de9e0b640 1 -- 172.21.15.63:0/243720136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eec052b70 con 0x559f6947ca70 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 -- 172.21.15.63:0/243720136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 msgr2=0x559f695716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 --2- 172.21.15.63:0/243720136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f695716f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7eec01d050 tx=0x7f7eec04c000 comp rx=0 tx=0).stop 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 -- 172.21.15.63:0/243720136 shutdown_connections 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 --2- 172.21.15.63:0/243720136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f695716f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 -- 172.21.15.63:0/243720136 >> 172.21.15.63:0/243720136 conn(0x559f69478630 msgr2=0x559f6956d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 -- 172.21.15.63:0/243720136 shutdown_connections 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 -- 172.21.15.63:0/243720136 wait complete. 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 Processor -- start 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.221+0000 7f7effd2d800 1 -- start start 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7effd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f69579090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7effd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f6947e230 con 0x559f6947ca70 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7efb97f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f69579090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7efb97f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f69579090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38178/0 (socket says 172.21.15.63:38178) 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7efb97f640 1 -- 172.21.15.63:0/3787158355 learned_addr learned my addr 172.21.15.63:0/3787158355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7efb97f640 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f695795d0 con 0x559f6947ca70 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7efb97f640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f69579090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005930 tx=0x7f7eec003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7de3fff640 1 -- 172.21.15.63:0/3787158355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eec073070 con 0x559f6947ca70 2024-04-23T19:11:26.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7de3fff640 1 -- 172.21.15.63:0/3787158355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec05d650 con 0x559f6947ca70 2024-04-23T19:11:26.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f69579860 con 0x559f6947ca70 2024-04-23T19:11:26.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.222+0000 7f7de3fff640 1 -- 172.21.15.63:0/3787158355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eec057070 con 0x559f6947ca70 2024-04-23T19:11:26.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.223+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6957c510 con 0x559f6947ca70 2024-04-23T19:11:26.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.223+0000 7f7de3fff640 1 -- 172.21.15.63:0/3787158355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7eec076070 con 0x559f6947ca70 2024-04-23T19:11:26.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.223+0000 7f7de3fff640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7de4036e60 0x7f7de4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.223+0000 7f7de3fff640 1 -- 172.21.15.63:0/3787158355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f7eec08e5d0 con 0x559f6947ca70 2024-04-23T19:11:26.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7efb17e640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7de4036e60 0x7f7de4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f6947d3d0 0x559f695802d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695808a0 con 0x559f6947d3d0 2024-04-23T19:11:26.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7efc180640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f6947d3d0 0x559f695802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7efb17e640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7de4036e60 0x7f7de4039320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0000f60 tx=0x7f7ef0001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7efc180640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f6947d3d0 0x559f695802d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7efc180640 1 -- 172.21.15.63:0/3787158355 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef400a4a0 con 0x559f6947d3d0 2024-04-23T19:11:26.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.224+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f6957d8a0 con 0x559f6947d3d0 2024-04-23T19:11:26.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.225+0000 7f7efc180640 1 -- 172.21.15.63:0/3787158355 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7ef400a4a0 con 0x559f6947d3d0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.225+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f69582710 con 0x559f6947d3d0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.225+0000 7f7efc180640 1 -- 172.21.15.63:0/3787158355 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7ef400a4a0 con 0x559f6947d3d0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.225+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f69581a50 con 0x559f6947d3d0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.225+0000 7f7efc180640 1 -- 172.21.15.63:0/3787158355 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7ef400a4a0 con 0x559f6947d3d0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.225+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f69583ac0 0x559f69585eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.225+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f69586430 con 0x559f69583ac0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.226+0000 7f7efb97f640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f69583ac0 0x559f69585eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.226+0000 7f7efb97f640 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f69583ac0 0x559f69585eb0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.226+0000 7f7efb97f640 1 -- 172.21.15.63:0/3787158355 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7eec075030 con 0x559f69583ac0 2024-04-23T19:11:26.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7effd2d800 1 Processor -- start 2024-04-23T19:11:26.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7effd2d800 1 -- start start 2024-04-23T19:11:26.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7effd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f69594850 0x559f69594c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7effd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f6936fe30 con 0x559f69594850 2024-04-23T19:11:26.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7efc180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f69594850 0x559f69594c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7efc180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f69594850 0x559f69594c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38182/0 (socket says 172.21.15.63:38182) 2024-04-23T19:11:26.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7efc180640 1 -- 172.21.15.63:0/2171032391 learned_addr learned my addr 172.21.15.63:0/2171032391 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7efc180640 1 -- 172.21.15.63:0/2171032391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f695951b0 con 0x559f69594850 2024-04-23T19:11:26.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.227+0000 7f7efc180640 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f69594850 0x559f69594c70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x559f69580c80 tx=0x7f7ef40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.228+0000 7f7de17fa640 1 -- 172.21.15.63:0/2171032391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef4013440 con 0x559f69594850 2024-04-23T19:11:26.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.228+0000 7f7de17fa640 1 -- 172.21.15.63:0/2171032391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef40135e0 con 0x559f69594850 2024-04-23T19:11:26.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.228+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f69595c20 con 0x559f69594850 2024-04-23T19:11:26.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.228+0000 7f7de17fa640 1 -- 172.21.15.63:0/2171032391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef40138c0 con 0x559f69594850 2024-04-23T19:11:26.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.228+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f69596120 con 0x559f69594850 2024-04-23T19:11:26.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.228+0000 7f7de17fa640 1 -- 172.21.15.63:0/2171032391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7ef4011040 con 0x559f69594850 2024-04-23T19:11:26.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7de17fa640 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dd4036e60 0x7f7dd4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7de17fa640 1 -- 172.21.15.63:0/2171032391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f7ef401d070 con 0x559f69594850 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7efb97f640 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dd4036e60 0x7f7dd4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7effd2d800 1 Processor -- start 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7effd2d800 1 -- start start 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7efb97f640 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dd4036e60 0x7f7dd4039320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7eec05dac0 tx=0x7f7eec06ff20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7effd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.229+0000 7f7effd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f69571f80 con 0x559f695a1bc0 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7efc180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7efc180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38196/0 (socket says 172.21.15.63:38196) 2024-04-23T19:11:26.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7efc180640 1 -- 172.21.15.63:0/1475546085 learned_addr learned my addr 172.21.15.63:0/1475546085 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7efc180640 1 -- 172.21.15.63:0/1475546085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f695a25b0 con 0x559f695a1bc0 2024-04-23T19:11:26.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7efc180640 1 --2- 172.21.15.63:0/1475546085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a1fe0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40013e0 tx=0x7f7ef4012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddf03e1e1757e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7ddaffd640 1 -- 172.21.15.63:0/1475546085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef4043070 con 0x559f695a1bc0 2024-04-23T19:11:26.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7ddaffd640 1 -- 172.21.15.63:0/1475546085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef402ac50 con 0x559f695a1bc0 2024-04-23T19:11:26.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.230+0000 7f7ddaffd640 1 -- 172.21.15.63:0/1475546085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef40014f0 con 0x559f695a1bc0 2024-04-23T19:11:26.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 -- 172.21.15.63:0/1475546085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 msgr2=0x559f695a1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 --2- 172.21.15.63:0/1475546085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a1fe0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40013e0 tx=0x7f7ef4012c90 comp rx=0 tx=0).stop 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 -- 172.21.15.63:0/1475546085 shutdown_connections 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 --2- 172.21.15.63:0/1475546085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a1fe0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 -- 172.21.15.63:0/1475546085 >> 172.21.15.63:0/1475546085 conn(0x559f6959ded0 msgr2=0x559f6959bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 -- 172.21.15.63:0/1475546085 shutdown_connections 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 -- 172.21.15.63:0/1475546085 wait complete. 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 Processor -- start 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 -- start start 2024-04-23T19:11:26.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7effd2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f695a2d00 con 0x559f695a1bc0 2024-04-23T19:11:26.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7efc180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7efc180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38198/0 (socket says 172.21.15.63:38198) 2024-04-23T19:11:26.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.231+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 learned_addr learned my addr 172.21.15.63:0/840890982 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.232+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f695a9640 con 0x559f695a1bc0 2024-04-23T19:11:26.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.232+0000 7f7efc180640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a9100 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4001da0 tx=0x7f7ef404cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.232+0000 7f7dd97fa640 1 -- 172.21.15.63:0/840890982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef4043040 con 0x559f695a1bc0 2024-04-23T19:11:26.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.232+0000 7f7dd97fa640 1 -- 172.21.15.63:0/840890982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef4030030 con 0x559f695a1bc0 2024-04-23T19:11:26.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.232+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f695a98d0 con 0x559f695a1bc0 2024-04-23T19:11:26.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.232+0000 7f7dd97fa640 1 -- 172.21.15.63:0/840890982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef40123a0 con 0x559f695a1bc0 2024-04-23T19:11:26.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.232+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f695ac580 con 0x559f695a1bc0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7dd97fa640 1 -- 172.21.15.63:0/840890982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7ef4012610 con 0x559f695a1bc0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7dd97fa640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dc4036e60 0x7f7dc4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7dd97fa640 1 -- 172.21.15.63:0/840890982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f7ef400e040 con 0x559f695a1bc0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7efb97f640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dc4036e60 0x7f7dc4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7efb97f640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dc4036e60 0x7f7dc4039320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006b30 tx=0x7f7eec091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f695b4df0 0x559f695b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.233+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695b7a00 con 0x559f695b4df0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7efb17e640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f695b4df0 0x559f695b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7efb17e640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f695b4df0 0x559f695b72b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 20 realm 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695b4240 con 0x559f695b4df0 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:26.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.234+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695b84f0 con 0x559f695b4df0 2024-04-23T19:11:26.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695b8dc0 con 0x559f695b4df0 2024-04-23T19:11:26.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:26.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695b9160 con 0x559f695b4df0 2024-04-23T19:11:26.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 20 searching for the correct realm 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f695bb040 0x559f695bd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.235+0000 7f7efc180640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f695bb040 0x559f695bd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.236+0000 7f7efc180640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f695bb040 0x559f695bd470 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.236+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.236+0000 7f7efc180640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7ef4038120 0x7f7ef402b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.236+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7ef4038120 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.236+0000 7f7efb97f640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7ef4038120 0x7f7ef402b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.236+0000 7f7efb97f640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7ef4038120 0x7f7ef402b7c0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.237+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.237+0000 7f7efb97f640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7eec078060 0x7f7eec077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.237+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.237+0000 7f7efb17e640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7eec078060 0x7f7eec077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.237+0000 7f7efb17e640 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7eec078060 0x7f7eec077a50 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.238+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7ef4038120 2024-04-23T19:11:26.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.239+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7ef4038120 2024-04-23T19:11:26.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.240+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.241+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.242+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.242+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.242+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.242+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.242+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.242+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7ef4038120 2024-04-23T19:11:26.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7ef4038120 2024-04-23T19:11:26.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.243+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7ef4038120 2024-04-23T19:11:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695bb040 2024-04-23T19:11:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.244+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7eec078060 2024-04-23T19:11:26.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.245+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.245+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x559f695b4df0 2024-04-23T19:11:26.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdb40 con 0x7f7ef4038120 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695b9ae0 con 0x559f695b4df0 2024-04-23T19:11:26.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.246+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695ba380 con 0x559f695b4df0 2024-04-23T19:11:26.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:26.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695bdfc0 con 0x7f7eec078060 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 zone default found 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 4 Realm: () 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 zonegroup default 2024-04-23T19:11:26.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695b9fe0 con 0x7f7ef4038120 2024-04-23T19:11:26.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695c0380 con 0x559f695bb040 2024-04-23T19:11:26.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.247+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.248+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.248+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695c0720 con 0x559f695bb040 2024-04-23T19:11:26.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.248+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.248+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.248+0000 7f7effd2d800 20 started sync module instance, tier type = 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.248+0000 7f7effd2d800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.248+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695bf810 con 0x7f7ef4038120 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.249+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.249+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94143155797040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c0fc0 con 0x7f7ef4038120 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.249+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c1810 con 0x559f695b4df0 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.250+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94143155797040] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.250+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.250+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94143155800848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c1850 con 0x559f695b4df0 2024-04-23T19:11:26.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.250+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c2360 con 0x559f695bb040 2024-04-23T19:11:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.251+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94143155800848] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.251+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.251+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94143155805712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c2750 con 0x559f695bb040 2024-04-23T19:11:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.251+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c3190 con 0x7f7ef4038120 2024-04-23T19:11:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.252+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94143155805712] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.252+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.252+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94143155809376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c3580 con 0x7f7ef4038120 2024-04-23T19:11:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.252+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c3fa0 con 0x559f695b4df0 2024-04-23T19:11:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.253+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.253+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94143155813008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c3fa0 con 0x559f695b4df0 2024-04-23T19:11:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.253+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c4fa0 con 0x7f7eec078060 2024-04-23T19:11:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.253+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94143155809376] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.255+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.255+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94143155817104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c4f60 con 0x7f7eec078060 2024-04-23T19:11:26.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.255+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5980 con 0x7f7eec078060 2024-04-23T19:11:26.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.255+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94143155813008] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.256+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94143155817104] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.256+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.256+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94143155802976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c4340 con 0x7f7eec078060 2024-04-23T19:11:26.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.256+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5f60 con 0x7f7eec078060 2024-04-23T19:11:26.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94143155802976] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94143155819632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5f60 con 0x7f7eec078060 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 20 add_watcher() i=0 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 20 add_watcher() i=1 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 20 add_watcher() i=2 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 20 add_watcher() i=3 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 20 add_watcher() i=4 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 20 add_watcher() i=5 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.257+0000 7f7effd2d800 20 add_watcher() i=6 2024-04-23T19:11:26.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.258+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94143155819632] v32'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.258+0000 7f7effd2d800 20 add_watcher() i=7 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.258+0000 7f7effd2d800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.258+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695c91d0 con 0x559f695b4df0 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.258+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.258+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94143155821600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c64f0 con 0x559f695b4df0 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94143155821600] v32'210 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7db1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 5 note: GC not initialized 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7db0fe1640 20 reqs_thread_entry: start 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 20 Filter name: none 2024-04-23T19:11:26.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7d2a7fc640 20 reqs_thread_entry: start 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.260+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695fc800 con 0x7f7eec078060 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695fafa0 con 0x559f695b4df0 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.261+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695fb870 con 0x7f7eec078060 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.262+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~380] v32'3 uv3 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.262+0000 7f7effd2d800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.262+0000 7f7effd2d800 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.262+0000 7f7effd2d800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94143156040640 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f695fb860 con 0x7f7eec078060 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== watch-notify(notify (1) cookie 94143155819632 notify 137438953501 ret 0) ==== 598+0+0 (crc 0 0 0) 0x559f695fbc10 con 0x7f7eec078060 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7ddaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953501 cookie 94143155819632 notifier 4723 bl.length()=556 2024-04-23T19:11:26.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7ddaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7ddaffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(65 notify.7 [notify cookie 94143156040640 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7ddaffd640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f7dc0003ea0 con 0x7f7eec078060 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== watch-notify(notify_complete (2) cookie 94143156040640 notify 137438953501 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559f695fb080 con 0x7f7eec078060 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.263+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695fdc70 con 0x559f695b4df0 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.264+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v32'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.264+0000 7f7effd2d800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.264+0000 7f7effd2d800 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.264+0000 7f7effd2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.264+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94143156040560 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x559f69600140 con 0x7f7eec078060 2024-04-23T19:11:26.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== watch-notify(notify (1) cookie 94143155817104 notify 137438953502 ret 0) ==== 219+0+0 (crc 0 0 0) 0x559f695fb080 con 0x7f7eec078060 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7dda7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953502 cookie 94143155817104 notifier 4723 bl.length()=177 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7dda7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7dda7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(68 notify.5 [notify cookie 94143156040560 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7dda7fc640 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f7d30003400 con 0x7f7eec078060 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.265+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== watch-notify(notify_complete (2) cookie 94143156040560 notify 137438953502 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f7ef0018040 con 0x7f7eec078060 2024-04-23T19:11:26.304 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"bar","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bar","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-04-23T19:11:26.262930Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:11:26.262913Z","tags":[],"group_ids":[]} 2024-04-23T19:11:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.266+0000 7f7db1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.266+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94143155821600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c8960 con 0x559f695b4df0 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.267+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94143155821600] v32'211 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.267+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94143155797040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5c20 con 0x7f7ef4038120 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.268+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94143155797040] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.268+0000 7f7effd2d800 20 remove_watcher() i=0 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.268+0000 7f7effd2d800 2 removed watcher, disabling cache 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.268+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94143155800848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5c20 con 0x559f695b4df0 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.269+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94143155800848] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.269+0000 7f7effd2d800 20 remove_watcher() i=1 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.269+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94143155805712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5c20 con 0x559f695bb040 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.270+0000 7f7efc180640 1 -- 172.21.15.63:0/840890982 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94143155805712] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef4029070 con 0x559f695bb040 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.270+0000 7f7effd2d800 20 remove_watcher() i=2 2024-04-23T19:11:26.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.270+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94143155809376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5c20 con 0x7f7ef4038120 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.271+0000 7f7efb97f640 1 -- 172.21.15.63:0/840890982 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94143155809376] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7eec075030 con 0x7f7ef4038120 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.271+0000 7f7effd2d800 20 remove_watcher() i=3 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.271+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94143155813008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5c20 con 0x559f695b4df0 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.272+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94143155813008] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x559f695b4df0 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.272+0000 7f7effd2d800 20 remove_watcher() i=4 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.272+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94143155817104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5c20 con 0x7f7eec078060 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.273+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94143155817104] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.273+0000 7f7effd2d800 20 remove_watcher() i=5 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.273+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94143155802976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c4a90 con 0x7f7eec078060 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.274+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94143155802976] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.274+0000 7f7effd2d800 20 remove_watcher() i=6 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.274+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94143155819632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x559f695c5c20 con 0x7f7eec078060 2024-04-23T19:11:26.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94143155819632] v32'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef000f610 con 0x7f7eec078060 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 20 remove_watcher() i=7 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7eec078060 msgr2=0x7f7eec077a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7eec078060 0x7f7eec077a50 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f695b4df0 msgr2=0x559f695b72b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f695b4df0 0x559f695b72b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7ef4038120 msgr2=0x7f7ef402b7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7ef4038120 0x7f7ef402b7c0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f695bb040 msgr2=0x559f695bd470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x559f695bb040 0x559f695bd470 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dc4036e60 msgr2=0x7f7dc4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dc4036e60 0x7f7dc4039320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006b30 tx=0x7f7eec091000 comp rx=0 tx=0).stop 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 msgr2=0x559f695a9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7efb17e640 1 -- 172.21.15.63:0/840890982 reap_dead start 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.275+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a9100 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4001da0 tx=0x7f7ef404cbb0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 shutdown_connections 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/840890982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f695a1bc0 0x559f695a9100 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 >> 172.21.15.63:0/840890982 conn(0x559f6959ded0 msgr2=0x559f6959b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 shutdown_connections 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/840890982 wait complete. 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dd4036e60 msgr2=0x7f7dd4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dd4036e60 0x7f7dd4039320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7eec05dac0 tx=0x7f7eec06ff20 comp rx=0 tx=0).stop 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f69594850 msgr2=0x559f69594c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f69594850 0x559f69594c70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x559f69580c80 tx=0x7f7ef40178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 shutdown_connections 2024-04-23T19:11:26.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7dd4036e60 0x7f7dd4039320 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/2171032391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f69594850 0x559f69594c70 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 >> 172.21.15.63:0/2171032391 conn(0x559f695888f0 msgr2=0x559f69588cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 shutdown_connections 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/2171032391 wait complete. 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f69583ac0 msgr2=0x559f69585eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f69583ac0 0x559f69585eb0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f6947d3d0 msgr2=0x559f695802d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f6947d3d0 0x559f695802d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7de4036e60 msgr2=0x7f7de4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.276+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7de4036e60 0x7f7de4039320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0000f60 tx=0x7f7ef0001370 comp rx=0 tx=0).stop 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 msgr2=0x559f69579090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f69579090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005930 tx=0x7f7eec003020 comp rx=0 tx=0).stop 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 shutdown_connections 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x559f69583ac0 0x559f69585eb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x559f6947d3d0 0x559f695802d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7de4036e60 0x7f7de4039320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 --2- 172.21.15.63:0/3787158355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f6947ca70 0x559f69579090 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 >> 172.21.15.63:0/3787158355 conn(0x559f69478630 msgr2=0x559f6956b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 shutdown_connections 2024-04-23T19:11:26.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.277+0000 7f7effd2d800 1 -- 172.21.15.63:0/3787158355 wait complete. 2024-04-23T19:11:26.316 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-04-23T19:11:26.262930Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:11:26.262913Z', 'tags': [], 'group_ids': []} 2024-04-23T19:11:26.328 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:11:26.333 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-04-23T19:11:26.333 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-23T19:11:26.333 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-23T19:11:26.333 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-04-23T19:11:26.379 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:26.380 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:26.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390ff35800 1 Processor -- start 2024-04-23T19:11:26.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390ff35800 1 -- start start 2024-04-23T19:11:26.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390ff35800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5645772e5740 0x5645772e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390ff35800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5645771937f0 con 0x5645772e1c30 2024-04-23T19:11:26.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390ff35800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x564577182e20 con 0x5645772e5740 2024-04-23T19:11:26.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390b3b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5645772e5740 0x5645772e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390b3b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5645772e5740 0x5645772e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38200/0 (socket says 172.21.15.63:38200) 2024-04-23T19:11:26.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390bbb5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5645772e1c30 0x5645772e4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:59032/0 (socket says 172.21.15.63:59032) 2024-04-23T19:11:26.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.379+0000 7f390b3b4640 1 -- 172.21.15.63:0/3501534380 learned_addr learned my addr 172.21.15.63:0/3501534380 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.380+0000 7f390b3b4640 1 -- 172.21.15.63:0/3501534380 >> v1:172.21.15.16:6789/0 conn(0x5645772e1c30 legacy=0x5645772e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.380+0000 7f390b3b4640 1 -- 172.21.15.63:0/3501534380 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645771a1b50 con 0x5645772e5740 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390b3b4640 1 --2- 172.21.15.63:0/3501534380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e5740 0x5645772e7b30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f38f400a520 tx=0x7f38f401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdae8fc5280a6f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390abb3640 1 -- 172.21.15.63:0/3501534380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f4057070 con 0x5645772e5740 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390abb3640 1 -- 172.21.15.63:0/3501534380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f4052570 con 0x5645772e5740 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390abb3640 1 -- 172.21.15.63:0/3501534380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f404ae60 con 0x5645772e5740 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390ff35800 1 -- 172.21.15.63:0/3501534380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e5740 msgr2=0x5645772e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390ff35800 1 --2- 172.21.15.63:0/3501534380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e5740 0x5645772e7b30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f38f400a520 tx=0x7f38f401d010 comp rx=0 tx=0).stop 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390ff35800 1 -- 172.21.15.63:0/3501534380 shutdown_connections 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390ff35800 1 --2- 172.21.15.63:0/3501534380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e5740 0x5645772e7b30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390ff35800 1 -- 172.21.15.63:0/3501534380 >> 172.21.15.63:0/3501534380 conn(0x5645772db600 msgr2=0x5645772dda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390ff35800 1 -- 172.21.15.63:0/3501534380 shutdown_connections 2024-04-23T19:11:26.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.382+0000 7f390ff35800 1 -- 172.21.15.63:0/3501534380 wait complete. 2024-04-23T19:11:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.395+0000 7f390ff35800 1 Processor -- start 2024-04-23T19:11:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.395+0000 7f390ff35800 1 -- start start 2024-04-23T19:11:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.395+0000 7f390ff35800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.395+0000 7f390ff35800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56457722c6f0 con 0x5645772e1c30 2024-04-23T19:11:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.395+0000 7f390bbb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.395+0000 7f390bbb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38206/0 (socket says 172.21.15.63:38206) 2024-04-23T19:11:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.395+0000 7f390bbb5640 1 -- 172.21.15.63:0/91925353 learned_addr learned my addr 172.21.15.63:0/91925353 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.396+0000 7f390bbb5640 1 -- 172.21.15.63:0/91925353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645773d2950 con 0x5645772e1c30 2024-04-23T19:11:26.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.396+0000 7f390bbb5640 1 --2- 172.21.15.63:0/91925353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d2380 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3904001680 tx=0x7f3904001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad579137eab5de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.396+0000 7f37f9e0b640 1 -- 172.21.15.63:0/91925353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904013420 con 0x5645772e1c30 2024-04-23T19:11:26.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.396+0000 7f37f9e0b640 1 -- 172.21.15.63:0/91925353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39040135c0 con 0x5645772e1c30 2024-04-23T19:11:26.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.396+0000 7f37f9e0b640 1 -- 172.21.15.63:0/91925353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904013900 con 0x5645772e1c30 2024-04-23T19:11:26.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 -- 172.21.15.63:0/91925353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 msgr2=0x5645773d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 --2- 172.21.15.63:0/91925353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d2380 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3904001680 tx=0x7f3904001b40 comp rx=0 tx=0).stop 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 -- 172.21.15.63:0/91925353 shutdown_connections 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 --2- 172.21.15.63:0/91925353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d2380 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 -- 172.21.15.63:0/91925353 >> 172.21.15.63:0/91925353 conn(0x5645772db600 msgr2=0x5645773cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 -- 172.21.15.63:0/91925353 shutdown_connections 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 -- 172.21.15.63:0/91925353 wait complete. 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 Processor -- start 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 -- start start 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.397+0000 7f390ff35800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645772ea380 con 0x5645772e1c30 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f390bbb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f390bbb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38212/0 (socket says 172.21.15.63:38212) 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f390bbb5640 1 -- 172.21.15.63:0/3135855533 learned_addr learned my addr 172.21.15.63:0/3135855533 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f390bbb5640 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645773da460 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f390bbb5640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d9f20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3904001680 tx=0x7f3904001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f37f3fff640 1 -- 172.21.15.63:0/3135855533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904022020 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f37f3fff640 1 -- 172.21.15.63:0/3135855533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3904006800 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f37f3fff640 1 -- 172.21.15.63:0/3135855533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904004c30 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5645773da6f0 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.398+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645773dd3a0 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.399+0000 7f37f3fff640 1 -- 172.21.15.63:0/3135855533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3904025020 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.399+0000 7f37f3fff640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37f4036e60 0x7f37f4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.399+0000 7f37f3fff640 1 -- 172.21.15.63:0/3135855533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f3904018030 con 0x5645772e1c30 2024-04-23T19:11:26.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.399+0000 7f390b3b4640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37f4036e60 0x7f37f4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.400+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5645772e5740 0x5645773e1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.400+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645773e16a0 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.400+0000 7f390c3b6640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5645772e5740 0x5645773e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.400+0000 7f390b3b4640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37f4036e60 0x7f37f4039320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f38f4054000 tx=0x7f38f404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.400+0000 7f390c3b6640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5645772e5740 0x5645773e1160 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.400+0000 7f390c3b6640 1 -- 172.21.15.63:0/3135855533 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38fc0083e0 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.400+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645773de710 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390c3b6640 1 -- 172.21.15.63:0/3135855533 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f38fc0083e0 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645773e3480 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390c3b6640 1 -- 172.21.15.63:0/3135855533 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f38fc0083e0 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645773e27c0 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390c3b6640 1 -- 172.21.15.63:0/3135855533 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f38fc0083e0 con 0x5645772e5740 2024-04-23T19:11:26.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5645773e4830 0x5645773e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645773e71a0 con 0x5645773e4830 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.401+0000 7f390bbb5640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5645773e4830 0x5645773e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.402+0000 7f390bbb5640 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5645773e4830 0x5645773e6c20 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.402+0000 7f390bbb5640 1 -- 172.21.15.63:0/3135855533 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f390401b070 con 0x5645773e4830 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.402+0000 7f390ff35800 1 Processor -- start 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.402+0000 7f390ff35800 1 -- start start 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390ff35800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645773f55e0 0x5645773f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390ff35800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645771d2e30 con 0x5645773f55e0 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390c3b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645773f55e0 0x5645773f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390c3b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645773f55e0 0x5645773f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38228/0 (socket says 172.21.15.63:38228) 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390c3b6640 1 -- 172.21.15.63:0/2271788829 learned_addr learned my addr 172.21.15.63:0/2271788829 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390c3b6640 1 -- 172.21.15.63:0/2271788829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645773f5f40 con 0x5645773f55e0 2024-04-23T19:11:26.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390c3b6640 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645773f55e0 0x5645773f5a00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5645773dadf0 tx=0x7f38fc00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f37f17fa640 1 -- 172.21.15.63:0/2271788829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc01c070 con 0x5645773f55e0 2024-04-23T19:11:26.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f37f17fa640 1 -- 172.21.15.63:0/2271788829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc018410 con 0x5645773f55e0 2024-04-23T19:11:26.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5645773f69b0 con 0x5645773f55e0 2024-04-23T19:11:26.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f37f17fa640 1 -- 172.21.15.63:0/2271788829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc018730 con 0x5645773f55e0 2024-04-23T19:11:26.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.403+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645773f6eb0 con 0x5645773f55e0 2024-04-23T19:11:26.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.404+0000 7f37f17fa640 1 -- 172.21.15.63:0/2271788829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f38fc01f070 con 0x5645773f55e0 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.404+0000 7f37f17fa640 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37e4036e60 0x7f37e4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.404+0000 7f37f17fa640 1 -- 172.21.15.63:0/2271788829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f38fc04ea10 con 0x5645773f55e0 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.404+0000 7f390bbb5640 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37e4036e60 0x7f37e4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390ff35800 1 Processor -- start 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390ff35800 1 -- start start 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390bbb5640 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37e4036e60 0x7f37e4039320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3904010fd0 tx=0x7f3904026ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390ff35800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577402e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390ff35800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645773d2cd0 con 0x564577402a60 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390c3b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577402e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390c3b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577402e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38238/0 (socket says 172.21.15.63:38238) 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390c3b6640 1 -- 172.21.15.63:0/3306199310 learned_addr learned my addr 172.21.15.63:0/3306199310 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390c3b6640 1 -- 172.21.15.63:0/3306199310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564577403450 con 0x564577402a60 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.405+0000 7f390c3b6640 1 --2- 172.21.15.63:0/3306199310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577402e80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f38fc04ccc0 tx=0x7f38fc029350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8624e7aca93a9281 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f37eaffd640 1 -- 172.21.15.63:0/3306199310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc01fc30 con 0x564577402a60 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f37eaffd640 1 -- 172.21.15.63:0/3306199310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc01fdd0 con 0x564577402a60 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f37eaffd640 1 -- 172.21.15.63:0/3306199310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc046bb0 con 0x564577402a60 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 -- 172.21.15.63:0/3306199310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 msgr2=0x564577402e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 --2- 172.21.15.63:0/3306199310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577402e80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f38fc04ccc0 tx=0x7f38fc029350 comp rx=0 tx=0).stop 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 -- 172.21.15.63:0/3306199310 shutdown_connections 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 --2- 172.21.15.63:0/3306199310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577402e80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 -- 172.21.15.63:0/3306199310 >> 172.21.15.63:0/3306199310 conn(0x5645773fece0 msgr2=0x5645773fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 -- 172.21.15.63:0/3306199310 shutdown_connections 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 -- 172.21.15.63:0/3306199310 wait complete. 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 Processor -- start 2024-04-23T19:11:26.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.406+0000 7f390ff35800 1 -- start start 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390ff35800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577409f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390ff35800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564577403b80 con 0x564577402a60 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390c3b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577409f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390c3b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577409f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:38248/0 (socket says 172.21.15.63:38248) 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 learned_addr learned my addr 172.21.15.63:0/2829295365 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56457740a460 con 0x564577402a60 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390c3b6640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577409f20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f38fc029390 tx=0x7f38fc04cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f37e97fa640 1 -- 172.21.15.63:0/2829295365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc039070 con 0x564577402a60 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f37e97fa640 1 -- 172.21.15.63:0/2829295365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc018bb0 con 0x564577402a60 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f37e97fa640 1 -- 172.21.15.63:0/2829295365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc01f360 con 0x564577402a60 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.407+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56457740a6f0 con 0x564577402a60 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.408+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457740d3a0 con 0x564577402a60 2024-04-23T19:11:26.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.408+0000 7f37e97fa640 1 -- 172.21.15.63:0/2829295365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f38fc01f500 con 0x564577402a60 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.408+0000 7f37e97fa640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37d4036e60 0x7f37d4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.408+0000 7f37e97fa640 1 -- 172.21.15.63:0/2829295365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f38fc031070 con 0x564577402a60 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.409+0000 7f390bbb5640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37d4036e60 0x7f37d4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.409+0000 7f390bbb5640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37d4036e60 0x7f37d4039320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39040069b0 tx=0x7f3904033000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.409+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.409+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564577415a80 0x564577417f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.409+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577418690 con 0x564577415a80 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.409+0000 7f390b3b4640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564577415a80 0x564577417f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.409+0000 7f390b3b4640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564577415a80 0x564577417f40 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 realm 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577418a60 con 0x564577415a80 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577419330 con 0x564577415a80 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645774196d0 con 0x564577415a80 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.410+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577419f20 con 0x564577415a80 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390ff35800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390ff35800 20 searching for the correct realm 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56457741bc90 0x56457741e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390c3b6640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56457741bc90 0x56457741e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390c3b6640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56457741bc90 0x56457741e080 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390c3b6640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f38fc021070 0x7f38fc02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.411+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f38fc021070 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.412+0000 7f390bbb5640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f38fc021070 0x7f38fc02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.412+0000 7f390bbb5640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f38fc021070 0x7f38fc02b7a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.412+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.412+0000 7f390bbb5640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3904027020 0x7f390401d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.412+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.412+0000 7f390b3b4640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3904027020 0x7f390401d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.413+0000 7f390b3b4640 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3904027020 0x7f390401d550 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.413+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.413+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.413+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.413+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.413+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.413+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f38fc021070 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.414+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f38fc021070 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.415+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.416+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.417+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.417+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.417+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.417+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.417+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.417+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f38fc021070 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f38fc021070 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.418+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.419+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.419+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.419+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.419+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.419+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f38fc021070 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.419+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.419+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x56457741bc90 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f3904027020 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x564577415a80 2024-04-23T19:11:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56457741e610 con 0x7f38fc021070 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.420+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56457741a8a0 con 0x564577415a80 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56457741b140 con 0x564577415a80 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56457741ea90 con 0x7f3904027020 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 zone default found 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 4 Realm: () 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 zonegroup default 2024-04-23T19:11:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56457741ef40 con 0x7f38fc021070 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.421+0000 7f390ff35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577420cc0 con 0x56457741bc90 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645774211e0 con 0x56457741bc90 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 20 started sync module instance, tier type = 2024-04-23T19:11:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.422+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577420160 con 0x7f38fc021070 2024-04-23T19:11:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.423+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.423+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94856353545392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577421640 con 0x7f38fc021070 2024-04-23T19:11:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.423+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577421e90 con 0x564577415a80 2024-04-23T19:11:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.424+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94856353545392] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.424+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.424+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94856353552864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5645774222b0 con 0x564577415a80 2024-04-23T19:11:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.424+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577422b60 con 0x56457741bc90 2024-04-23T19:11:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.425+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94856353552864] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.426+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.426+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94856353553920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5645774226d0 con 0x56457741bc90 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.426+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577423630 con 0x7f38fc021070 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.427+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94856353553920] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.427+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.427+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94856353558816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5645774226d0 con 0x7f38fc021070 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.427+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5645774240b0 con 0x564577415a80 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.428+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.428+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94856353561536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5645774240a0 con 0x564577415a80 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.428+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94856353558816] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.428+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577422260 con 0x7f3904027020 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.429+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.429+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94856353562784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577425080 con 0x7f3904027020 2024-04-23T19:11:26.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.429+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577425ac0 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.429+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94856353561536] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.430+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94856353562784] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.430+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.430+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94856353568144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577424bd0 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.430+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577426150 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94856353568144] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94856353563856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577424c30 con 0x7f3904027020 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 20 add_watcher() i=0 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 20 add_watcher() i=1 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 20 add_watcher() i=2 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 20 add_watcher() i=3 2024-04-23T19:11:26.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 20 add_watcher() i=4 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 20 add_watcher() i=5 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.431+0000 7f390ff35800 20 add_watcher() i=6 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.432+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94856353563856] v32'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.432+0000 7f390ff35800 20 add_watcher() i=7 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.432+0000 7f390ff35800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.432+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645774292f0 con 0x564577415a80 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.433+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.433+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94856353569952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577426810 con 0x564577415a80 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94856353569952] v32'212 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f37c1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:26.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 5 note: GC not initialized 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f37c0fe1640 20 reqs_thread_entry: start 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 20 Filter name: none 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f373a7fc640 20 reqs_thread_entry: start 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.434+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56457745c780 con 0x7f3904027020 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.435+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 bar [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.435+0000 7f390ff35800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.435+0000 7f390ff35800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.435+0000 7f390ff35800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.435+0000 7f390ff35800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:11:26.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.435+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460840 con 0x7f38fc021070 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.436+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(63 bud [create,call,writefull 0~206] v32'1 uv1 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f390401b070 con 0x7f38fc021070 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.436+0000 7f390ff35800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.436+0000 7f390ff35800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.436+0000 7f390ff35800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.436+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 94856353805088 in=394b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645774613b0 con 0x7f3904027020 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== watch-notify(notify (1) cookie 94856353562784 notify 137438953503 ret 0) ==== 424+0+0 (crc 0 0 0) 0x5645771a1b50 con 0x7f3904027020 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f37eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953503 cookie 94856353562784 notifier 4730 bl.length()=382 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f37eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(64 notify.5 [notify cookie 94856353805088 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f37eaffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f37eaffd640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f37d0003ea0 con 0x7f3904027020 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x7f3904027020 2024-04-23T19:11:26.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== watch-notify(notify_complete (2) cookie 94856353805088 notify 137438953503 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f38f405daa0 con 0x7f3904027020 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.437+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645774612b0 con 0x564577415a80 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 : miss 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 20 rados->read ofs=0 len=0 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 8.12 8:4a286bde:root::.bucket.meta.myfoo6%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56457745e970 con 0x56457741bc90 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 .bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 [call out=48b,stat out=16b,read 0~380 out=380b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 333+0+662 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 info.flags=0x17 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 to cache LRU end 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:11:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.438+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 8.12 8:4a286bde:root::.bucket.meta.myfoo6%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~380 in=380b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5645774680c0 con 0x56457741bc90 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.439+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 .bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 [delete,create,call,call,writefull 0~380,setxattr (117)] v32'3 uv3 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.439+0000 7f390ff35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 info.flags=0x17 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.439+0000 7f390ff35800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 to cache LRU end 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.439+0000 7f390ff35800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.439+0000 7f390ff35800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9, ofs0, ns] 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.439+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94856353802608 in=760b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645774627f0 con 0x7f38fc021070 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== watch-notify(notify (1) cookie 94856353545392 notify 137438953494 ret 0) ==== 790+0+0 (crc 0 0 0) 0x5645773dd3a0 con 0x7f38fc021070 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f37eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953494 cookie 94856353545392 notifier 4730 bl.length()=748 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f37eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 info.flags=0x17 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f37eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 to cache LRU end 2024-04-23T19:11:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(69 notify.0 [notify cookie 94856353802608 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f37eaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f37eaffd640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f37d0003ea0 con 0x7f38fc021070 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(70 notify.0 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== watch-notify(notify_complete (2) cookie 94856353802608 notify 137438953494 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5645773da6f0 con 0x7f38fc021070 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): orig_sources= new_sources= 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): potential sources added=[] removed=[] 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): orig_dests= new_dests= 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): potential dests added=[] removed=[] 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.440+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56457745e970 con 0x564577415a80 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.441+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(71 bud.buckets [call] v32'3 uv0 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.441+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56457745e970 con 0x564577415a80 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.442+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(72 bud.buckets [call] v32'4 uv4 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.442+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:73 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56457745e970 con 0x564577415a80 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(73 myfoo6 [delete,create,call,writefull 0~166] v32'17 uv17 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f390ff35800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f390ff35800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f390ff35800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [notify cookie 94856353802608 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5645774627f0 con 0x7f38fc021070 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== watch-notify(notify (1) cookie 94856353558816 notify 137438953495 ret 0) ==== 358+0+0 (crc 0 0 0) 0x5645773da460 con 0x7f38fc021070 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f37eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953495 cookie 94856353558816 notifier 4730 bl.length()=316 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f37eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f37eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(74 notify.3 [notify cookie 94856353802608 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.443+0000 7f37eaffd640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:75 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f37d0003ea0 con 0x7f38fc021070 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.444+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== watch-notify(notify_complete (2) cookie 94856353802608 notify 137438953495 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5645773d2950 con 0x7f38fc021070 2024-04-23T19:11:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.444+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(75 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.444+0000 7f390ff35800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 : hit (requested=0x17, cached=0x17) 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.444+0000 7f390ff35800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.444+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:76 8.12 8:4a286bde:root::.bucket.meta.myfoo6%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~380 in=380b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577466120 con 0x56457741bc90 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(76 .bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 [delete,create,call,call,writefull 0~380,setxattr (117)] v32'4 uv4 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390ff35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 info.flags=0x17 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390ff35800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 to cache LRU end 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390ff35800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390ff35800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9, ofs0, ns] 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:77 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94856353814496 in=760b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577462130 con 0x7f38fc021070 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== watch-notify(notify (1) cookie 94856353545392 notify 137438953496 ret 0) ==== 790+0+0 (crc 0 0 0) 0x5645773d2950 con 0x7f38fc021070 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(77 notify.0 [notify cookie 94856353814496 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f37ea7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953496 cookie 94856353545392 notifier 4730 bl.length()=748 2024-04-23T19:11:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f37ea7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 info.flags=0x17 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f37ea7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 to cache LRU end 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f37ea7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.445+0000 7f37ea7fc640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f3740003470 con 0x7f38fc021070 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== watch-notify(notify_complete (2) cookie 94856353814496 notify 137438953496 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5645771937f0 con 0x7f38fc021070 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(78 notify.0 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): orig_sources= new_sources= 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): potential sources added=[] removed=[] 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): orig_dests= new_dests= 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390ff35800 20 handle_bi_update(): bucket=:myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]): potential dests added=[] removed=[] 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:79 4.35 4:aea617cc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x564577415a80 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(79 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 4.2b 4:d630da47:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x564577415a80 2024-04-23T19:11:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(80 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.446+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:81 4.a 4:52db42c4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x7f38fc021070 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.447+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(81 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.447+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 4.15 4:aa91196c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x56457741bc90 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.447+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(82 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3 [call out=85b] v0'0 uv6 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.447+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:83 4.2 4:42dd7ebf:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x564577415a80 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(83 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f38f405a040 con 0x564577415a80 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 4.39 4:9d2fe1d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x56457741bc90 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(84 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:85 4.3f 4:fc2e1960:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x7f38fc021070 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(85 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 4.39 4:9c4112ef:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x56457741bc90 2024-04-23T19:11:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(86 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.448+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:87 4.27 4:e4002aad:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x564577415a80 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.449+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(87 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f38f4095040 con 0x564577415a80 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.449+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:88 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x56457741bc90 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.449+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(88 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [call out=377b] v0'0 uv5 ondisk = 0) ==== 194+0+377 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.449+0000 7f390ff35800 10 cls_bucket_list_unordered: got thirteen 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.449+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:89 4.18 4:1b88699c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577461c90 con 0x7f38fc021070 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(89 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 get_obj_state: octx=0x56457745eee0 obj=myfoo6:thirteen state=0x56457745f1d8 s->prefetch_data=0 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 3.27s0 3:e606443a:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577466900 con 0x7f3904027020 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen [getxattrs out=854b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 238+0+870 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 10 manifest: total_size = 8 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5312102473928247874 2024-04-23T19:11:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.acl 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.content_type 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.etag 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.idtag 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.manifest 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.source_zone 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 get_obj_state: octx=0x56457745eee0 obj=myfoo6:thirteen state=0x56457745f1d8 s->prefetch_data=0 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 : hit (requested=0x17, cached=0x17) 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 2024-04-23T19:11:26.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.450+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56457746b790 con 0x56457741bc90 2024-04-23T19:11:26.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.452+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(91 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [stat,call,call] v32'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.452+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 3.27s0 3:e606443a:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen:head [cmpxattr user.rgw.idtag (62) op 1 mode 1 in=76b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577469c20 con 0x7f3904027020 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.453+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen [cmpxattr (62) op 1 mode 1,setxattr (117),setxattr (33)] v32'2 uv2 ondisk = 0) ==== 280+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.453+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:93 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=267b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56457746b160 con 0x56457741bc90 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:1 objects processed in :myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9]). Next marker thirteen 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.453+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f38fc021070 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.454+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(93 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [stat,call,call] v32'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.454+0000 7f37ea7fc640 20 handle_completion(): completion ok for obj=thirteen 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.454+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(94 bud [delete,create,call,writefull 0~380] v32'2 uv2 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.454+0000 7f390ff35800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.454+0000 7f390ff35800 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.454+0000 7f390ff35800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.454+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [notify cookie 94856353852592 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f3904027020 2024-04-23T19:11:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== watch-notify(notify (1) cookie 94856353562784 notify 137438953504 ret 0) ==== 598+0+0 (crc 0 0 0) 0x564577468200 con 0x7f3904027020 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f37eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953504 cookie 94856353562784 notifier 4730 bl.length()=556 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f37eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f37eaffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(95 notify.5 [notify cookie 94856353852592 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f37eaffd640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f37d0003ea0 con 0x7f3904027020 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== watch-notify(notify_complete (2) cookie 94856353852592 notify 137438953504 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564577460260 con 0x7f3904027020 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(96 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.455+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x564577415a80 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.456+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(97 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v32'2 uv2 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x564577415a80 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.456+0000 7f390ff35800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.456+0000 7f390ff35800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.456+0000 7f390ff35800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.456+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify cookie 94856353852592 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f3904027020 2024-04-23T19:11:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== watch-notify(notify (1) cookie 94856353562784 notify 137438953505 ret 0) ==== 219+0+0 (crc 0 0 0) 0x564577460260 con 0x7f3904027020 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f37eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953505 cookie 94856353562784 notifier 4730 bl.length()=177 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f37eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f37eaffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f37eaffd640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:99 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f37d0003ea0 con 0x7f3904027020 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(98 notify.5 [notify cookie 94856353852592 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== watch-notify(notify_complete (2) cookie 94856353852592 notify 137438953505 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f38f405daa0 con 0x7f3904027020 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(99 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f390ff35800 10 removing user index: bar 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.457+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f3904027020 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.458+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(100 bar [delete] v32'4 uv4 ondisk = 0) ==== 147+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.458+0000 7f390ff35800 10 removing default.rgw.meta+users.uid+bar from cache 2024-04-23T19:11:26.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.458+0000 7f390ff35800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.458+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:101 7.9 7:93e5b521:::notify.7:head [notify cookie 94856353852592 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f3904027020 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.459+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== watch-notify(notify (1) cookie 94856353563856 notify 137438953506 ret 0) ==== 194+0+0 (crc 0 0 0) 0x7f38f405daa0 con 0x7f3904027020 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.459+0000 7f37ea7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953506 cookie 94856353563856 notifier 4730 bl.length()=152 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.459+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(101 notify.7 [notify cookie 94856353852592 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.459+0000 7f37ea7fc640 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f3740003450 con 0x7f3904027020 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.459+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== watch-notify(notify_complete (2) cookie 94856353852592 notify 137438953506 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f38f4098770 con 0x7f3904027020 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.459+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(102 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.474 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"bud","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bud","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-04-23T19:11:26.262930Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:11:26.262913Z","tags":[],"group_ids":[]} 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.460+0000 7f37c1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.460+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94856353569952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x564577415a80 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.461+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94856353569952] v32'213 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x564577415a80 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.461+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94856353545392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f38fc021070 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.462+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94856353545392] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.462+0000 7f390ff35800 20 remove_watcher() i=0 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.462+0000 7f390ff35800 2 removed watcher, disabling cache 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.462+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94856353552864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x564577415a80 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.464+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94856353552864] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x564577415a80 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.464+0000 7f390ff35800 20 remove_watcher() i=1 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.464+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94856353553920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x56457741bc90 2024-04-23T19:11:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.465+0000 7f390c3b6640 1 -- 172.21.15.63:0/2829295365 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94856353553920] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc03b030 con 0x56457741bc90 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.465+0000 7f390ff35800 20 remove_watcher() i=2 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.465+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94856353558816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f38fc021070 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.466+0000 7f390bbb5640 1 -- 172.21.15.63:0/2829295365 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94856353558816] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f390406a040 con 0x7f38fc021070 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.466+0000 7f390ff35800 20 remove_watcher() i=3 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.466+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94856353561536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x564577415a80 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.467+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94856353561536] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x564577415a80 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.467+0000 7f390ff35800 20 remove_watcher() i=4 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.467+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94856353562784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f3904027020 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.468+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94856353562784] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.468+0000 7f390ff35800 20 remove_watcher() i=5 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.468+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94856353568144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f3904027020 2024-04-23T19:11:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.469+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94856353568144] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.469+0000 7f390ff35800 20 remove_watcher() i=6 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.469+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94856353563856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x564577460d00 con 0x7f3904027020 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94856353563856] v32'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38f4095040 con 0x7f3904027020 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 20 remove_watcher() i=7 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3904027020 msgr2=0x7f390401d550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f3904027020 0x7f390401d550 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564577415a80 msgr2=0x564577417f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x564577415a80 0x564577417f40 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f38fc021070 msgr2=0x7f38fc02b7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f38fc021070 0x7f38fc02b7a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56457741bc90 msgr2=0x56457741e080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56457741bc90 0x56457741e080 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37d4036e60 msgr2=0x7f37d4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37d4036e60 0x7f37d4039320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39040069b0 tx=0x7f3904033000 comp rx=0 tx=0).stop 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 msgr2=0x564577409f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 --2- 172.21.15.63:0/2829295365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564577402a60 0x564577409f20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f38fc029390 tx=0x7f38fc04cc20 comp rx=0 tx=0).stop 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 reap_dead start 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390b3b4640 1 -- 172.21.15.63:0/2829295365 reap_dead start 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 shutdown_connections 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 >> 172.21.15.63:0/2829295365 conn(0x5645773fece0 msgr2=0x5645773fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 shutdown_connections 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.470+0000 7f390ff35800 1 -- 172.21.15.63:0/2829295365 wait complete. 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37e4036e60 msgr2=0x7f37e4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37e4036e60 0x7f37e4039320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3904010fd0 tx=0x7f3904026ed0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645773f55e0 msgr2=0x5645773f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645773f55e0 0x5645773f5a00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5645773dadf0 tx=0x7f38fc00ef30 comp rx=0 tx=0).stop 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 shutdown_connections 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37e4036e60 0x7f37e4039320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/2271788829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645773f55e0 0x5645773f5a00 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 >> 172.21.15.63:0/2271788829 conn(0x5645773e9680 msgr2=0x5645773e9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 shutdown_connections 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/2271788829 wait complete. 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5645773e4830 msgr2=0x5645773e6c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5645773e4830 0x5645773e6c20 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5645772e5740 msgr2=0x5645773e1160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5645772e5740 0x5645773e1160 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37f4036e60 msgr2=0x7f37f4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37f4036e60 0x7f37f4039320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f38f4054000 tx=0x7f38f404e000 comp rx=0 tx=0).stop 2024-04-23T19:11:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 msgr2=0x5645773d9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d9f20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3904001680 tx=0x7f3904001250 comp rx=0 tx=0).stop 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 shutdown_connections 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5645773e4830 0x5645773e6c20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5645772e5740 0x5645773e1160 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f37f4036e60 0x7f37f4039320 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 --2- 172.21.15.63:0/3135855533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645772e1c30 0x5645773d9f20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 >> 172.21.15.63:0/3135855533 conn(0x5645772db600 msgr2=0x5645773cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 shutdown_connections 2024-04-23T19:11:26.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:26.471+0000 7f390ff35800 1 -- 172.21.15.63:0/3135855533 wait complete. 2024-04-23T19:11:26.486 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-04-23T19:11:26.262930Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:11:26.262913Z', 'tags': [], 'group_ids': []} 2024-04-23T19:11:31.495 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:11:31.496 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-23T19:11:31.496 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-23T19:11:31.496 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-23T19:11:31.496 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-04-23T19:11:31.556 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:31.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.556+0000 7f4b7fa94800 1 Processor -- start 2024-04-23T19:11:31.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.556+0000 7f4b7fa94800 1 -- start start 2024-04-23T19:11:31.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.556+0000 7f4b7fa94800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557616d3b6d0 0x557616d3baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.556+0000 7f4b7fa94800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557616bed7f0 con 0x557616d3c030 2024-04-23T19:11:31.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.556+0000 7f4b7fa94800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x557616bdce20 con 0x557616d3b6d0 2024-04-23T19:11:31.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.556+0000 7f4b7d593640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x557616d3b6d0 0x557616d3baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.556+0000 7f4b7cd92640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x557616d3c030 0x557616d42980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:41544/0 (socket says 172.21.15.63:41544) 2024-04-23T19:11:31.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.557+0000 7f4b7cd92640 1 -- 172.21.15.63:0/3019249965 learned_addr learned my addr 172.21.15.63:0/3019249965 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:31.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.557+0000 7f4b7d593640 1 -- 172.21.15.63:0/3019249965 >> v1:172.21.15.16:6789/0 conn(0x557616d3c030 legacy=0x557616d42980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.557+0000 7f4b7d593640 1 -- 172.21.15.63:0/3019249965 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557616bfbb50 con 0x557616d3b6d0 2024-04-23T19:11:31.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.559+0000 7f4b7d593640 1 --2- 172.21.15.63:0/3019249965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616d3baf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000a470 tx=0x7f4b7001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5a496251a6faa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.559+0000 7f4b6ffff640 1 -- 172.21.15.63:0/3019249965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b70057070 con 0x557616d3b6d0 2024-04-23T19:11:31.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.559+0000 7f4b6ffff640 1 -- 172.21.15.63:0/3019249965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b70052570 con 0x557616d3b6d0 2024-04-23T19:11:31.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.559+0000 7f4b6ffff640 1 -- 172.21.15.63:0/3019249965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b70052870 con 0x557616d3b6d0 2024-04-23T19:11:31.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.560+0000 7f4b7fa94800 1 -- 172.21.15.63:0/3019249965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 msgr2=0x557616d3baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.560+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/3019249965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616d3baf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000a470 tx=0x7f4b7001ce90 comp rx=0 tx=0).stop 2024-04-23T19:11:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.560+0000 7f4b7fa94800 1 -- 172.21.15.63:0/3019249965 shutdown_connections 2024-04-23T19:11:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.560+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/3019249965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616d3baf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.560+0000 7f4b7fa94800 1 -- 172.21.15.63:0/3019249965 >> 172.21.15.63:0/3019249965 conn(0x557616d34de0 msgr2=0x557616d37200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.560+0000 7f4b7fa94800 1 -- 172.21.15.63:0/3019249965 shutdown_connections 2024-04-23T19:11:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.560+0000 7f4b7fa94800 1 -- 172.21.15.63:0/3019249965 wait complete. 2024-04-23T19:11:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.574+0000 7f4b7fa94800 1 Processor -- start 2024-04-23T19:11:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.574+0000 7f4b7fa94800 1 -- start start 2024-04-23T19:11:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.574+0000 7f4b7fa94800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.574+0000 7f4b7fa94800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557616c86720 con 0x557616d3b6d0 2024-04-23T19:11:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4b7d593640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4b7d593640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e2e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58716/0 (socket says 172.21.15.63:58716) 2024-04-23T19:11:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4b7d593640 1 -- 172.21.15.63:0/733440605 learned_addr learned my addr 172.21.15.63:0/733440605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4b7d593640 1 -- 172.21.15.63:0/733440605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557616e2ec60 con 0x557616d3b6d0 2024-04-23T19:11:31.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4b7d593640 1 --2- 172.21.15.63:0/733440605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e2e690 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4b70055040 tx=0x7f4b7004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1ecca979a33926 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4a6b606640 1 -- 172.21.15.63:0/733440605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b70057040 con 0x557616d3b6d0 2024-04-23T19:11:31.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4a6b606640 1 -- 172.21.15.63:0/733440605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b70052990 con 0x557616d3b6d0 2024-04-23T19:11:31.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.575+0000 7f4a6b606640 1 -- 172.21.15.63:0/733440605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b70052c70 con 0x557616d3b6d0 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 -- 172.21.15.63:0/733440605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 msgr2=0x557616e2e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/733440605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e2e690 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4b70055040 tx=0x7f4b7004c000 comp rx=0 tx=0).stop 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 -- 172.21.15.63:0/733440605 shutdown_connections 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/733440605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e2e690 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 -- 172.21.15.63:0/733440605 >> 172.21.15.63:0/733440605 conn(0x557616d34de0 msgr2=0x557616e29fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 -- 172.21.15.63:0/733440605 shutdown_connections 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 -- 172.21.15.63:0/733440605 wait complete. 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.576+0000 7f4b7fa94800 1 Processor -- start 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.577+0000 7f4b7fa94800 1 -- start start 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.577+0000 7f4b7fa94800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e36110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.577+0000 7f4b7fa94800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557616d443f0 con 0x557616d3b6d0 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.577+0000 7f4b7d593640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e36110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.577+0000 7f4b7d593640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e36110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58724/0 (socket says 172.21.15.63:58724) 2024-04-23T19:11:31.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.577+0000 7f4b7d593640 1 -- 172.21.15.63:0/2305260534 learned_addr learned my addr 172.21.15.63:0/2305260534 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.577+0000 7f4b7d593640 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557616e36650 con 0x557616d3b6d0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.578+0000 7f4b7d593640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e36110 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000a470 tx=0x7f4b70003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.578+0000 7f4a69e03640 1 -- 172.21.15.63:0/2305260534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b70073070 con 0x557616d3b6d0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.578+0000 7f4a69e03640 1 -- 172.21.15.63:0/2305260534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7005d780 con 0x557616d3b6d0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.578+0000 7f4a69e03640 1 -- 172.21.15.63:0/2305260534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b70057070 con 0x557616d3b6d0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.578+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557616e368e0 con 0x557616d3b6d0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.578+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557616e39540 con 0x557616d3b6d0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.579+0000 7f4a69e03640 1 -- 172.21.15.63:0/2305260534 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4b70076070 con 0x557616d3b6d0 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.579+0000 7f4a69e03640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a64036eb0 0x7f4a64039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.579+0000 7f4a69e03640 1 -- 172.21.15.63:0/2305260534 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4b7008e6f0 con 0x557616d3b6d0 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.579+0000 7f4b7cd92640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a64036eb0 0x7f4a64039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.579+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616d3c030 0x557616e3d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.579+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e3d880 con 0x557616d3c030 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.579+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616d3c030 0x557616e3d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.580+0000 7f4b7cd92640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a64036eb0 0x7f4a64039370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4b74000f60 tx=0x7f4b740012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.580+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616d3c030 0x557616e3d300 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.580+0000 7f4b7dd94640 1 -- 172.21.15.63:0/2305260534 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4b7800a4a0 con 0x557616d3c030 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.580+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e3a8b0 con 0x557616d3c030 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.580+0000 7f4b7dd94640 1 -- 172.21.15.63:0/2305260534 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4b7800a4a0 con 0x557616d3c030 2024-04-23T19:11:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.580+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e3f680 con 0x557616d3c030 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.581+0000 7f4b7dd94640 1 -- 172.21.15.63:0/2305260534 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4b7800a4a0 con 0x557616d3c030 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.581+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e3e9c0 con 0x557616d3c030 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.581+0000 7f4b7dd94640 1 -- 172.21.15.63:0/2305260534 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4b7800a4a0 con 0x557616d3c030 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.581+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557616e40a30 0x557616e42e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.581+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e433a0 con 0x557616e40a30 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.581+0000 7f4b7d593640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557616e40a30 0x557616e42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.582+0000 7f4b7d593640 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557616e40a30 0x557616e42e20 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.582+0000 7f4b7d593640 1 -- 172.21.15.63:0/2305260534 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4b70075030 con 0x557616e40a30 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.583+0000 7f4b7fa94800 1 Processor -- start 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.583+0000 7f4b7fa94800 1 -- start start 2024-04-23T19:11:31.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.583+0000 7f4b7fa94800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e517c0 0x557616e51be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.583+0000 7f4b7fa94800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557616c2ce30 con 0x557616e517c0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.583+0000 7f4b7dd94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e517c0 0x557616e51be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.583+0000 7f4b7dd94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e517c0 0x557616e51be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58732/0 (socket says 172.21.15.63:58732) 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.583+0000 7f4b7dd94640 1 -- 172.21.15.63:0/2201930320 learned_addr learned my addr 172.21.15.63:0/2201930320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.584+0000 7f4b7dd94640 1 -- 172.21.15.63:0/2201930320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557616e52120 con 0x557616e517c0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.584+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e517c0 0x557616e51be0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x557616e374c0 tx=0x7f4b780178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.584+0000 7f4a62ffd640 1 -- 172.21.15.63:0/2201930320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78013440 con 0x557616e517c0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.584+0000 7f4a62ffd640 1 -- 172.21.15.63:0/2201930320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b780135e0 con 0x557616e517c0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.584+0000 7f4a62ffd640 1 -- 172.21.15.63:0/2201930320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b780138c0 con 0x557616e517c0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.584+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557616e52b90 con 0x557616e517c0 2024-04-23T19:11:31.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.584+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557616e53090 con 0x557616e517c0 2024-04-23T19:11:31.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.585+0000 7f4a62ffd640 1 -- 172.21.15.63:0/2201930320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4b78011040 con 0x557616e517c0 2024-04-23T19:11:31.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.585+0000 7f4a62ffd640 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a58036eb0 0x7f4a58039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.585+0000 7f4a62ffd640 1 -- 172.21.15.63:0/2201930320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4b7801d070 con 0x557616e517c0 2024-04-23T19:11:31.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.585+0000 7f4b7d593640 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a58036eb0 0x7f4a58039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7fa94800 1 Processor -- start 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7fa94800 1 -- start start 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7d593640 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a58036eb0 0x7f4a58039370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b7005b490 tx=0x7f4b70052440 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7fa94800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e5eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7fa94800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557616e2ef20 con 0x557616e5eaa0 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7dd94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e5eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7dd94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e5eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58740/0 (socket says 172.21.15.63:58740) 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7dd94640 1 -- 172.21.15.63:0/775200121 learned_addr learned my addr 172.21.15.63:0/775200121 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.586+0000 7f4b7dd94640 1 -- 172.21.15.63:0/775200121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557616e5f490 con 0x557616e5eaa0 2024-04-23T19:11:31.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/775200121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e5eec0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4b7804cca0 tx=0x7f4b780127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58803f4c3577d2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4a60ff9640 1 -- 172.21.15.63:0/775200121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78042020 con 0x557616e5eaa0 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4a60ff9640 1 -- 172.21.15.63:0/775200121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b78012cc0 con 0x557616e5eaa0 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4a60ff9640 1 -- 172.21.15.63:0/775200121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78027040 con 0x557616e5eaa0 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7fa94800 1 -- 172.21.15.63:0/775200121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 msgr2=0x557616e5eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/775200121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e5eec0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4b7804cca0 tx=0x7f4b780127a0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7fa94800 1 -- 172.21.15.63:0/775200121 shutdown_connections 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/775200121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e5eec0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7fa94800 1 -- 172.21.15.63:0/775200121 >> 172.21.15.63:0/775200121 conn(0x557616e5ae00 msgr2=0x557616e58e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7fa94800 1 -- 172.21.15.63:0/775200121 shutdown_connections 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.587+0000 7f4b7fa94800 1 -- 172.21.15.63:0/775200121 wait complete. 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7fa94800 1 Processor -- start 2024-04-23T19:11:31.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7fa94800 1 -- start start 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7fa94800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7fa94800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557616e5fc00 con 0x557616e5eaa0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7dd94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7dd94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e660a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58750/0 (socket says 172.21.15.63:58750) 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 learned_addr learned my addr 172.21.15.63:0/1942606167 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.588+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557616e665e0 con 0x557616e5eaa0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.589+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e660a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4b780127e0 tx=0x7f4b7804cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.589+0000 7f4a4effd640 1 -- 172.21.15.63:0/1942606167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78027040 con 0x557616e5eaa0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.589+0000 7f4a4effd640 1 -- 172.21.15.63:0/1942606167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b78021c60 con 0x557616e5eaa0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.589+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557616e66870 con 0x557616e5eaa0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.589+0000 7f4a4effd640 1 -- 172.21.15.63:0/1942606167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7802a350 con 0x557616e5eaa0 2024-04-23T19:11:31.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.589+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557616e69520 con 0x557616e5eaa0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.590+0000 7f4a4effd640 1 -- 172.21.15.63:0/1942606167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4b78045020 con 0x557616e5eaa0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.590+0000 7f4a4effd640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a48036eb0 0x7f4a48039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.590+0000 7f4a4effd640 1 -- 172.21.15.63:0/1942606167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4b78033030 con 0x557616e5eaa0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.590+0000 7f4b7d593640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a48036eb0 0x7f4a48039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.590+0000 7f4b7d593640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a48036eb0 0x7f4a48039370 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b7005cfd0 tx=0x7f4b70083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.590+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.590+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616e71b40 0x557616e74000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e74750 con 0x557616e71b40 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7cd92640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616e71b40 0x557616e74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7cd92640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616e71b40 0x557616e74000 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4b74009050 con 0x557616e71b40 2024-04-23T19:11:31.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 20 realm 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e70f90 con 0x557616e71b40 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4b74009050 con 0x557616e71b40 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.591+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e75240 con 0x557616e71b40 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4b74009050 con 0x557616e71b40 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e75b10 con 0x557616e71b40 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4b74009050 con 0x557616e71b40 2024-04-23T19:11:31.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e75eb0 con 0x557616e71b40 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4b74009050 con 0x557616e71b40 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 20 searching for the correct realm 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557616e77d90 0x557616e7a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.592+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x557616e77d90 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.593+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557616e77d90 0x557616e7a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.593+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557616e77d90 0x557616e7a180 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.593+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.593+0000 7f4b7dd94640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4b780123a0 0x7f4b78022640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.593+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x7f4b780123a0 2024-04-23T19:11:31.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.593+0000 7f4b7d593640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4b780123a0 0x7f4b78022640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.594+0000 7f4b7d593640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4b780123a0 0x7f4b78022640 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.594+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.594+0000 7f4b7d593640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4b70078060 0x7f4b700773b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.594+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x7f4b70078060 2024-04-23T19:11:31.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.594+0000 7f4b7cd92640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4b70078060 0x7f4b700773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.595+0000 7f4b7cd92640 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4b70078060 0x7f4b700773b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.595+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b74009050 con 0x7f4b70078060 2024-04-23T19:11:31.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.595+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x557616e71b40 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.595+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b74009050 con 0x557616e71b40 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.595+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x557616e71b40 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.595+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.595+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x7f4b70078060 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.596+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.596+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x7f4b70078060 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.596+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.596+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x7f4b780123a0 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.596+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.596+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x557616e77d90 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.597+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.597+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x557616e71b40 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.597+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.597+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x7f4b70078060 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.597+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.597+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x557616e71b40 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x557616e7a830 con 0x7f4b780123a0 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e77d90 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e77d90 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.598+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b70078060 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.599+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.599+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e77d90 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.599+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.599+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e77d90 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.599+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.599+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e71b40 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.599+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.600+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e71b40 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.600+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.600+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b70078060 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.600+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.600+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e77d90 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.600+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.600+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b780123a0 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.601+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.601+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e71b40 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.601+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.601+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b780123a0 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.601+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.601+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e77d90 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.601+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.602+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b70078060 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.602+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.602+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b780123a0 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.602+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.602+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e77d90 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.602+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.602+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b70078060 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x557616e71b40 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f4b700a9060 con 0x7f4b780123a0 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:31.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.603+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e76830 con 0x557616e71b40 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e770d0 con 0x557616e71b40 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e7acb0 con 0x7f4b70078060 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.604+0000 7f4b7fa94800 20 zone default found 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 4 Realm: () 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 zonegroup default 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e7b160 con 0x7f4b780123a0 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e7cee0 con 0x557616e77d90 2024-04-23T19:11:31.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e7d360 con 0x557616e77d90 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 started sync module instance, tier type = 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.605+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7c380 con 0x7f4b780123a0 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.606+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.606+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93965678790512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7dc70 con 0x7f4b780123a0 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.607+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7e4c0 con 0x557616e71b40 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.608+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93965678790512] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.608+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.608+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93965678794688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7e5c0 con 0x557616e71b40 2024-04-23T19:11:31.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.608+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7f0e0 con 0x557616e77d90 2024-04-23T19:11:31.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.609+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93965678794688] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.609+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.609+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93965678799792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7f4d0 con 0x557616e77d90 2024-04-23T19:11:31.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.609+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7ff10 con 0x7f4b780123a0 2024-04-23T19:11:31.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.611+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93965678799792] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.611+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.611+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93965678803424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e80300 con 0x7f4b780123a0 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.611+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e80d40 con 0x557616e71b40 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.612+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93965678803424] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.612+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.612+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93965678807056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e80d00 con 0x557616e71b40 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.612+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e817b0 con 0x7f4b70078060 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.613+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93965678807056] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.613+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.613+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93965678796816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e817b0 con 0x7f4b70078060 2024-04-23T19:11:31.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.613+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e821f0 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.614+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93965678796816] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.614+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.614+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93965678809856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e823d0 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.614+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e82c20 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.615+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.615+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93965678814640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e82fc0 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7fa94800 20 add_watcher() i=0 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7fa94800 20 add_watcher() i=1 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7fa94800 20 add_watcher() i=2 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7fa94800 20 add_watcher() i=3 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7fa94800 20 add_watcher() i=4 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7fa94800 20 add_watcher() i=5 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93965678809856] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.616+0000 7f4b7fa94800 20 add_watcher() i=6 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.617+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93965678814640] v32'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.617+0000 7f4b7fa94800 20 add_watcher() i=7 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.617+0000 7f4b7fa94800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.617+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616e81f90 con 0x557616e71b40 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.617+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.617+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93965678815824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e81f90 con 0x557616e71b40 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.618+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93965678815824] v32'214 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.618+0000 7f4a337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.618+0000 7f4b7fa94800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.618+0000 7f4b7fa94800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.618+0000 7f4b7fa94800 5 note: GC not initialized 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.619+0000 7f4a327e4640 20 reqs_thread_entry: start 2024-04-23T19:11:31.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.619+0000 7f4b7fa94800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.619+0000 7f4b7fa94800 20 Filter name: none 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.619+0000 7f49a3fff640 20 reqs_thread_entry: start 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.619+0000 7f4b7fa94800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.619+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.619+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616eb9610 con 0x7f4b70078060 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616eb7db0 con 0x7f4b70078060 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.620+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616eb82b0 con 0x7f4b70078060 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.621+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v32'5 uv5 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.621+0000 7f4b7fa94800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.621+0000 7f4b7fa94800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.621+0000 7f4b7fa94800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.621+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 93965679038560 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616ebcb40 con 0x7f4b70078060 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== watch-notify(notify (1) cookie 93965678796816 notify 137438953507 ret 0) ==== 598+0+0 (crc 0 0 0) 0x557616eb8650 con 0x7f4b70078060 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4a60ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953507 cookie 93965678796816 notifier 4737 bl.length()=556 2024-04-23T19:11:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4a60ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4a60ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(65 notify.5 [notify cookie 93965679038560 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4a60ff9640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f4a40003ea0 con 0x7f4b70078060 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== watch-notify(notify_complete (2) cookie 93965679038560 notify 137438953507 ret 0) ==== 42+0+48 (crc 0 0 0) 0x557616eb7e10 con 0x7f4b70078060 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.622+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616eb9060 con 0x7f4b70078060 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.623+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v32'9 uv9 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7fa94800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7fa94800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7fa94800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 93965679038560 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x557616ebcaa0 con 0x557616e77d90 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== watch-notify(notify (1) cookie 93965678799792 notify 137438953478 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7f4b78053030 con 0x557616e77d90 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4a4ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953478 cookie 93965678799792 notifier 4737 bl.length()=177 2024-04-23T19:11:31.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4a4ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-23T19:11:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4a4ffff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-23T19:11:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 notify.2 [notify cookie 93965679038560 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4a4ffff640 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f49b0003400 con 0x557616e77d90 2024-04-23T19:11:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.624+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== watch-notify(notify_complete (2) cookie 93965679038560 notify 137438953478 ret 0) ==== 42+0+48 (crc 0 0 0) 0x557616e66870 con 0x557616e77d90 2024-04-23T19:11:31.633 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-23T19:11:31.622019Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:11:31.622003Z","tags":[],"group_ids":[]} 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.625+0000 7f4a337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.625+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93965678815824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e85bc0 con 0x557616e71b40 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.626+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93965678815824] v32'215 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.626+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93965678790512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e85710 con 0x7f4b780123a0 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.627+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93965678790512] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.627+0000 7f4b7fa94800 20 remove_watcher() i=0 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.627+0000 7f4b7fa94800 2 removed watcher, disabling cache 2024-04-23T19:11:31.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.627+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93965678794688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7c770 con 0x557616e71b40 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.629+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93965678794688] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.629+0000 7f4b7fa94800 20 remove_watcher() i=1 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.629+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93965678799792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7c770 con 0x557616e77d90 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.630+0000 7f4b7dd94640 1 -- 172.21.15.63:0/1942606167 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93965678799792] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b78029070 con 0x557616e77d90 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.630+0000 7f4b7fa94800 20 remove_watcher() i=2 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.630+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93965678803424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7ebb0 con 0x7f4b780123a0 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.631+0000 7f4b7d593640 1 -- 172.21.15.63:0/1942606167 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93965678803424] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b70089030 con 0x7f4b780123a0 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.631+0000 7f4b7fa94800 20 remove_watcher() i=3 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.631+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93965678807056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7c770 con 0x557616e71b40 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.632+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93965678807056] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x557616e71b40 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.632+0000 7f4b7fa94800 20 remove_watcher() i=4 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.632+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93965678796816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7f9e0 con 0x7f4b70078060 2024-04-23T19:11:31.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.633+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93965678796816] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.633+0000 7f4b7fa94800 20 remove_watcher() i=5 2024-04-23T19:11:31.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.633+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93965678809856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e7ebb0 con 0x7f4b70078060 2024-04-23T19:11:31.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.634+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93965678809856] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.634+0000 7f4b7fa94800 20 remove_watcher() i=6 2024-04-23T19:11:31.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.634+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93965678814640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x557616e80810 con 0x7f4b70078060 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93965678814640] v32'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4b7402a040 con 0x7f4b70078060 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 20 remove_watcher() i=7 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4b70078060 msgr2=0x7f4b700773b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4b70078060 0x7f4b700773b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616e71b40 msgr2=0x557616e74000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616e71b40 0x557616e74000 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4b780123a0 msgr2=0x7f4b78022640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4b780123a0 0x7f4b78022640 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557616e77d90 msgr2=0x557616e7a180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.635+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x557616e77d90 0x557616e7a180 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a48036eb0 msgr2=0x7f4a48039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a48036eb0 0x7f4a48039370 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b7005cfd0 tx=0x7f4b70083000 comp rx=0 tx=0).stop 2024-04-23T19:11:31.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 msgr2=0x557616e660a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7cd92640 1 -- 172.21.15.63:0/1942606167 reap_dead start 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e660a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4b780127e0 tx=0x7f4b7804cc00 comp rx=0 tx=0).stop 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 shutdown_connections 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/1942606167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e5eaa0 0x557616e660a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 >> 172.21.15.63:0/1942606167 conn(0x557616e5ae00 msgr2=0x557616e58460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 shutdown_connections 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/1942606167 wait complete. 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a58036eb0 msgr2=0x7f4a58039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a58036eb0 0x7f4a58039370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b7005b490 tx=0x7f4b70052440 comp rx=0 tx=0).stop 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e517c0 msgr2=0x557616e51be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e517c0 0x557616e51be0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x557616e374c0 tx=0x7f4b780178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 shutdown_connections 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a58036eb0 0x7f4a58039370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2201930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616e517c0 0x557616e51be0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 >> 172.21.15.63:0/2201930320 conn(0x557616e45860 msgr2=0x557616e45c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 shutdown_connections 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.636+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2201930320 wait complete. 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557616e40a30 msgr2=0x557616e42e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557616e40a30 0x557616e42e20 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616d3c030 msgr2=0x557616e3d300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616d3c030 0x557616e3d300 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a64036eb0 msgr2=0x7f4a64039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a64036eb0 0x7f4a64039370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4b74000f60 tx=0x7f4b740012b0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 msgr2=0x557616e36110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e36110 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000a470 tx=0x7f4b70003020 comp rx=0 tx=0).stop 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 shutdown_connections 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x557616e40a30 0x557616e42e20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x557616d3c030 0x557616e3d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4a64036eb0 0x7f4a64039370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 --2- 172.21.15.63:0/2305260534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557616d3b6d0 0x557616e36110 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 >> 172.21.15.63:0/2305260534 conn(0x557616d34de0 msgr2=0x557616e287f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 shutdown_connections 2024-04-23T19:11:31.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.637+0000 7f4b7fa94800 1 -- 172.21.15.63:0/2305260534 wait complete. 2024-04-23T19:11:31.647 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-23T19:11:31.622019Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:11:31.622003Z', 'tags': [], 'group_ids': []} 2024-04-23T19:11:31.660 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:11:31.665 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-04-23T19:11:31.665 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-23T19:11:31.665 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-23T19:11:31.665 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-04-23T19:11:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.710+0000 7f4dc9728800 1 Processor -- start 2024-04-23T19:11:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.710+0000 7f4dc9728800 1 -- start start 2024-04-23T19:11:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc9728800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55dca24b1570 0x55dca24b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc9728800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55dca235f7f0 con 0x55dca24ad180 2024-04-23T19:11:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc9728800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55dca234ee20 con 0x55dca24b1570 2024-04-23T19:11:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc4c27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55dca24b1570 0x55dca24b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc5428640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55dca24ad180 0x55dca24ad5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:41550/0 (socket says 172.21.15.63:41550) 2024-04-23T19:11:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc5428640 1 -- 172.21.15.63:0/1735548705 learned_addr learned my addr 172.21.15.63:0/1735548705 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc4c27640 1 -- 172.21.15.63:0/1735548705 >> v1:172.21.15.16:6789/0 conn(0x55dca24ad180 legacy=0x55dca24ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.711+0000 7f4dc4c27640 1 -- 172.21.15.63:0/1735548705 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca236db50 con 0x55dca24b1570 2024-04-23T19:11:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.713+0000 7f4dc4c27640 1 --2- 172.21.15.63:0/1735548705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24b1570 0x55dca24b3960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00a470 tx=0x7f4dbc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98acef00a086311 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.713+0000 7f4db7fff640 1 -- 172.21.15.63:0/1735548705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dbc057070 con 0x55dca24b1570 2024-04-23T19:11:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.713+0000 7f4db7fff640 1 -- 172.21.15.63:0/1735548705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dbc052570 con 0x55dca24b1570 2024-04-23T19:11:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.713+0000 7f4db7fff640 1 -- 172.21.15.63:0/1735548705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dbc052870 con 0x55dca24b1570 2024-04-23T19:11:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.714+0000 7f4dc9728800 1 -- 172.21.15.63:0/1735548705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24b1570 msgr2=0x55dca24b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.714+0000 7f4dc9728800 1 --2- 172.21.15.63:0/1735548705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24b1570 0x55dca24b3960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00a470 tx=0x7f4dbc01ce90 comp rx=0 tx=0).stop 2024-04-23T19:11:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.714+0000 7f4dc9728800 1 -- 172.21.15.63:0/1735548705 shutdown_connections 2024-04-23T19:11:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.714+0000 7f4dc9728800 1 --2- 172.21.15.63:0/1735548705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24b1570 0x55dca24b3960 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.714+0000 7f4dc9728800 1 -- 172.21.15.63:0/1735548705 >> 172.21.15.63:0/1735548705 conn(0x55dca24a87e0 msgr2=0x55dca24aac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.714+0000 7f4dc9728800 1 -- 172.21.15.63:0/1735548705 shutdown_connections 2024-04-23T19:11:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.714+0000 7f4dc9728800 1 -- 172.21.15.63:0/1735548705 wait complete. 2024-04-23T19:11:31.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc9728800 1 Processor -- start 2024-04-23T19:11:31.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc9728800 1 -- start start 2024-04-23T19:11:31.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc9728800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca2422a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc9728800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dca23f86f0 con 0x55dca24ad180 2024-04-23T19:11:31.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc5428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca2422a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc5428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca2422a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58770/0 (socket says 172.21.15.63:58770) 2024-04-23T19:11:31.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc5428640 1 -- 172.21.15.63:0/3148223279 learned_addr learned my addr 172.21.15.63:0/3148223279 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.727+0000 7f4dc5428640 1 -- 172.21.15.63:0/3148223279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca2421050 con 0x55dca24ad180 2024-04-23T19:11:31.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.728+0000 7f4dc5428640 1 --2- 172.21.15.63:0/3148223279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca2422a90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001d80 tx=0x7f4db8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f063c1dc891f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.728+0000 7f4cb3606640 1 -- 172.21.15.63:0/3148223279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db8013030 con 0x55dca24ad180 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.728+0000 7f4cb3606640 1 -- 172.21.15.63:0/3148223279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db800e470 con 0x55dca24ad180 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.728+0000 7f4cb3606640 1 -- 172.21.15.63:0/3148223279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db800e810 con 0x55dca24ad180 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.728+0000 7f4dc9728800 1 -- 172.21.15.63:0/3148223279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 msgr2=0x55dca2422a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.728+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3148223279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca2422a90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001d80 tx=0x7f4db8001e70 comp rx=0 tx=0).stop 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 -- 172.21.15.63:0/3148223279 shutdown_connections 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3148223279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca2422a90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 -- 172.21.15.63:0/3148223279 >> 172.21.15.63:0/3148223279 conn(0x55dca24a87e0 msgr2=0x55dca2599c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 -- 172.21.15.63:0/3148223279 shutdown_connections 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 -- 172.21.15.63:0/3148223279 wait complete. 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 Processor -- start 2024-04-23T19:11:31.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 -- start start 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca25a8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc9728800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dca24b4230 con 0x55dca24ad180 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc5428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca25a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc5428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca25a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58772/0 (socket says 172.21.15.63:58772) 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.729+0000 7f4dc5428640 1 -- 172.21.15.63:0/3252677016 learned_addr learned my addr 172.21.15.63:0/3252677016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.730+0000 7f4dc5428640 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca25a8650 con 0x55dca24ad180 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.730+0000 7f4dc5428640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca25a8110 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4db8004000 tx=0x7f4db8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.730+0000 7f4cb1e03640 1 -- 172.21.15.63:0/3252677016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db8027070 con 0x55dca24ad180 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.730+0000 7f4cb1e03640 1 -- 172.21.15.63:0/3252677016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db8019900 con 0x55dca24ad180 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.730+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dca25a88e0 con 0x55dca24ad180 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.730+0000 7f4cb1e03640 1 -- 172.21.15.63:0/3252677016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db8019c60 con 0x55dca24ad180 2024-04-23T19:11:31.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.730+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca25ab590 con 0x55dca24ad180 2024-04-23T19:11:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.731+0000 7f4cb1e03640 1 -- 172.21.15.63:0/3252677016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4db802a070 con 0x55dca24ad180 2024-04-23T19:11:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.731+0000 7f4cb1e03640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4cac036e60 0x7f4cac039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.731+0000 7f4cb1e03640 1 -- 172.21.15.63:0/3252677016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4db801f070 con 0x55dca24ad180 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.731+0000 7f4dc4c27640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4cac036e60 0x7f4cac039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.731+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca24b1570 0x55dca25af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.731+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25af8d0 con 0x55dca24b1570 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.731+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca24b1570 0x55dca25af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.732+0000 7f4dc4c27640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4cac036e60 0x7f4cac039320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc054000 tx=0x7f4dbc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.732+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca24b1570 0x55dca25af350 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.732+0000 7f4dc5c29640 1 -- 172.21.15.63:0/3252677016 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4dc000a4a0 con 0x55dca24b1570 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.732+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ac920 con 0x55dca24b1570 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.732+0000 7f4dc5c29640 1 -- 172.21.15.63:0/3252677016 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4dc000a4a0 con 0x55dca24b1570 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.732+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25b1760 con 0x55dca24b1570 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.733+0000 7f4dc5c29640 1 -- 172.21.15.63:0/3252677016 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4dc000a4a0 con 0x55dca24b1570 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.733+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25b0aa0 con 0x55dca24b1570 2024-04-23T19:11:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.733+0000 7f4dc5c29640 1 -- 172.21.15.63:0/3252677016 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4dc000a4a0 con 0x55dca24b1570 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.733+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dca25b2b10 0x55dca25b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.733+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25b5480 con 0x55dca25b2b10 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.733+0000 7f4dc5428640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dca25b2b10 0x55dca25b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.734+0000 7f4dc5428640 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dca25b2b10 0x55dca25b4f00 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.734+0000 7f4dc5428640 1 -- 172.21.15.63:0/3252677016 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4db8029030 con 0x55dca25b2b10 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.734+0000 7f4dc9728800 1 Processor -- start 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.734+0000 7f4dc9728800 1 -- start start 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.735+0000 7f4dc9728800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25c3830 0x55dca25c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.735+0000 7f4dc9728800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dca239ee30 con 0x55dca25c3830 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.735+0000 7f4dc5c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25c3830 0x55dca25c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.735+0000 7f4dc5c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25c3830 0x55dca25c3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58776/0 (socket says 172.21.15.63:58776) 2024-04-23T19:11:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.735+0000 7f4dc5c29640 1 -- 172.21.15.63:0/317695590 learned_addr learned my addr 172.21.15.63:0/317695590 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.735+0000 7f4dc5c29640 1 -- 172.21.15.63:0/317695590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca25c4190 con 0x55dca25c3830 2024-04-23T19:11:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.735+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25c3830 0x55dca25c3c50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55dca25a8e20 tx=0x7f4dc00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.736+0000 7f4caaffd640 1 -- 172.21.15.63:0/317695590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0013440 con 0x55dca25c3830 2024-04-23T19:11:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.736+0000 7f4caaffd640 1 -- 172.21.15.63:0/317695590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc00135e0 con 0x55dca25c3830 2024-04-23T19:11:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.736+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dca25c4c00 con 0x55dca25c3830 2024-04-23T19:11:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.736+0000 7f4caaffd640 1 -- 172.21.15.63:0/317695590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc00138c0 con 0x55dca25c3830 2024-04-23T19:11:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.736+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca25c5100 con 0x55dca25c3830 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.736+0000 7f4caaffd640 1 -- 172.21.15.63:0/317695590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4dc0011040 con 0x55dca25c3830 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4caaffd640 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c9c036eb0 0x7f4c9c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4caaffd640 1 -- 172.21.15.63:0/317695590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4dc001d070 con 0x55dca25c3830 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4dc5428640 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c9c036eb0 0x7f4c9c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4dc9728800 1 Processor -- start 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4dc5428640 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c9c036eb0 0x7f4c9c039370 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4db800c010 tx=0x7f4db802ae80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4dc9728800 1 -- start start 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4dc9728800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.737+0000 7f4dc9728800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dca24213d0 con 0x55dca25d0bd0 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.738+0000 7f4dc5c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.738+0000 7f4dc5c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58782/0 (socket says 172.21.15.63:58782) 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.738+0000 7f4dc5c29640 1 -- 172.21.15.63:0/136186262 learned_addr learned my addr 172.21.15.63:0/136186262 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.738+0000 7f4dc5c29640 1 -- 172.21.15.63:0/136186262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca25d15c0 con 0x55dca25d0bd0 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.738+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/136186262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d0ff0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4dc004cc70 tx=0x7f4dc00127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4073dc19eb61dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.740+0000 7f4ca8ff9640 1 -- 172.21.15.63:0/136186262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0042020 con 0x55dca25d0bd0 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.740+0000 7f4ca8ff9640 1 -- 172.21.15.63:0/136186262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc0012cc0 con 0x55dca25d0bd0 2024-04-23T19:11:31.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.740+0000 7f4ca8ff9640 1 -- 172.21.15.63:0/136186262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0027040 con 0x55dca25d0bd0 2024-04-23T19:11:31.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 -- 172.21.15.63:0/136186262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 msgr2=0x55dca25d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 --2- 172.21.15.63:0/136186262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d0ff0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4dc004cc70 tx=0x7f4dc00127a0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 -- 172.21.15.63:0/136186262 shutdown_connections 2024-04-23T19:11:31.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 --2- 172.21.15.63:0/136186262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d0ff0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 -- 172.21.15.63:0/136186262 >> 172.21.15.63:0/136186262 conn(0x55dca25cce50 msgr2=0x55dca25caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 -- 172.21.15.63:0/136186262 shutdown_connections 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 -- 172.21.15.63:0/136186262 wait complete. 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 Processor -- start 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 -- start start 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.741+0000 7f4dc9728800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dca25d1cf0 con 0x55dca25d0bd0 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4dc5c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4dc5c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58798/0 (socket says 172.21.15.63:58798) 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 learned_addr learned my addr 172.21.15.63:0/2687699287 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca25d8630 con 0x55dca25d0bd0 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d80f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0001bc0 tx=0x7f4dc004cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4c9affd640 1 -- 172.21.15.63:0/2687699287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0027070 con 0x55dca25d0bd0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4c9affd640 1 -- 172.21.15.63:0/2687699287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc0021c60 con 0x55dca25d0bd0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dca25d88c0 con 0x55dca25d0bd0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4c9affd640 1 -- 172.21.15.63:0/2687699287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc002a350 con 0x55dca25d0bd0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.742+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca25db570 con 0x55dca25d0bd0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.743+0000 7f4c9affd640 1 -- 172.21.15.63:0/2687699287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4dc0045020 con 0x55dca25d0bd0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.743+0000 7f4c9affd640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c90036e60 0x7f4c90039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.743+0000 7f4c9affd640 1 -- 172.21.15.63:0/2687699287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f4dc0032070 con 0x55dca25d0bd0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.743+0000 7f4dc5428640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c90036e60 0x7f4c90039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.744+0000 7f4dc5428640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c90036e60 0x7f4c90039320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4db8017a60 tx=0x7f4db8042000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.744+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.744+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca25e3d80 0x55dca25e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.744+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25e6990 con 0x55dca25e3d80 2024-04-23T19:11:31.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.744+0000 7f4dc4c27640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca25e3d80 0x55dca25e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc4c27640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca25e3d80 0x55dca25e6240 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 realm 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25e6d60 con 0x55dca25e3d80 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25e7630 con 0x55dca25e3d80 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25e79d0 con 0x55dca25e3d80 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.745+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25e8220 con 0x55dca25e3d80 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc9728800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc9728800 20 searching for the correct realm 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dca25e9f90 0x55dca25ec380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dca25e9f90 0x55dca25ec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dca25e9f90 0x55dca25ec380 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc5c29640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4dc00123a0 0x7f4dc002c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.746+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4dc00123a0 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.747+0000 7f4dc5428640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4dc00123a0 0x7f4dc002c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.747+0000 7f4dc5428640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4dc00123a0 0x7f4dc002c480 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.747+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.747+0000 7f4dc5428640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4db802c060 0x7f4db804b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.747+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.747+0000 7f4dc4c27640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4db802c060 0x7f4db804b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.748+0000 7f4dc4c27640 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4db802c060 0x7f4db804b5b0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.748+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.748+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.748+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.748+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4dc00123a0 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.749+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.750+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.750+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.750+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.750+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.750+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.750+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4dc00123a0 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.751+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.752+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.753+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.753+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.753+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.753+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.753+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.753+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4dc00123a0 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4dc00123a0 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.754+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.755+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.755+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4dc00123a0 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.755+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.755+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e9f90 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.755+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.755+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4db802c060 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x55dca25e3d80 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ec910 con 0x7f4dc00123a0 2024-04-23T19:11:31.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25e8ba0 con 0x55dca25e3d80 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.756+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25e9440 con 0x55dca25e3d80 2024-04-23T19:11:31.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ecd90 con 0x7f4db802c060 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 zone default found 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 4 Realm: () 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 zonegroup default 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ed240 con 0x7f4dc00123a0 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.757+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25eefc0 con 0x55dca25e9f90 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc9728800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ef460 con 0x55dca25e9f90 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc9728800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc9728800 20 started sync module instance, tier type = 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc9728800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:31.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.758+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ee460 con 0x7f4dc00123a0 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.759+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.759+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94406105294928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25efd70 con 0x7f4dc00123a0 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.759+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f05c0 con 0x55dca25e3d80 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.760+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94406105294928] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.760+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.760+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94406105299136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f06a0 con 0x55dca25e3d80 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.760+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f11c0 con 0x55dca25e9f90 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.761+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94406105299136] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.761+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.761+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94406105304208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f15b0 con 0x55dca25e9f90 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.761+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f1ff0 con 0x7f4dc00123a0 2024-04-23T19:11:31.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.762+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.763+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94406105307840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f1ff0 con 0x7f4dc00123a0 2024-04-23T19:11:31.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.763+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f2ff0 con 0x55dca25e3d80 2024-04-23T19:11:31.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.763+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94406105304208] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.764+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.764+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94406105311968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f2fb0 con 0x55dca25e3d80 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.764+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f3b20 con 0x7f4db802c060 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.764+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94406105307840] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.765+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.765+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94406105314864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f3b20 con 0x7f4db802c060 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.765+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f4530 con 0x7f4db802c060 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.765+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94406105311968] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.766+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94406105314864] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.766+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.766+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94406105310176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f4080 con 0x7f4db802c060 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.766+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f4a40 con 0x7f4db802c060 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94406105318800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f4e60 con 0x7f4db802c060 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 20 add_watcher() i=0 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 20 add_watcher() i=1 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 20 add_watcher() i=2 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 20 add_watcher() i=3 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 20 add_watcher() i=4 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 20 add_watcher() i=5 2024-04-23T19:11:31.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94406105310176] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.767+0000 7f4dc9728800 20 add_watcher() i=6 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.768+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94406105318800] v32'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.768+0000 7f4dc9728800 20 add_watcher() i=7 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.768+0000 7f4dc9728800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.768+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f4e60 con 0x55dca25e3d80 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.768+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.768+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94406105319856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f4dd0 con 0x55dca25e3d80 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.769+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94406105319856] v32'216 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.769+0000 7f4c7b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 5 note: GC not initialized 2024-04-23T19:11:31.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4c7a7e4640 20 reqs_thread_entry: start 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 20 Filter name: none 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4bebfff640 20 reqs_thread_entry: start 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.770+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55dca262b220 con 0x7f4dc00123a0 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.771+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(62 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.771+0000 7f4dc9728800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.771+0000 7f4dc9728800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.771+0000 7f4dc9728800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.771+0000 7f4dc9728800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.771+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca262f360 con 0x7f4db802c060 2024-04-23T19:11:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.772+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 fud [create,call,writefull 0~206] v32'6 uv5 ondisk = -17 ((17) File exists)) ==== 231+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.773+0000 7f4c7b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.773+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94406105319856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca262f3b0 con 0x55dca25e3d80 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.774+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94406105319856] v32'217 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.774+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94406105294928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f49b0 con 0x7f4dc00123a0 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.775+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94406105294928] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.775+0000 7f4dc9728800 20 remove_watcher() i=0 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.775+0000 7f4dc9728800 2 removed watcher, disabling cache 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.775+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94406105299136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f49b0 con 0x55dca25e3d80 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.776+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94406105299136] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.776+0000 7f4dc9728800 20 remove_watcher() i=1 2024-04-23T19:11:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.776+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94406105304208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ef8c0 con 0x55dca25e9f90 2024-04-23T19:11:31.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.777+0000 7f4dc5c29640 1 -- 172.21.15.63:0/2687699287 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94406105304208] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dc0029070 con 0x55dca25e9f90 2024-04-23T19:11:31.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.777+0000 7f4dc9728800 20 remove_watcher() i=2 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.777+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94406105307840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f49b0 con 0x7f4dc00123a0 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.778+0000 7f4dc5428640 1 -- 172.21.15.63:0/2687699287 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94406105307840] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4db803e070 con 0x7f4dc00123a0 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.778+0000 7f4dc9728800 20 remove_watcher() i=3 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.778+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94406105311968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f0c90 con 0x55dca25e3d80 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.779+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94406105311968] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x55dca25e3d80 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.779+0000 7f4dc9728800 20 remove_watcher() i=4 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.779+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94406105314864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25ef8c0 con 0x7f4db802c060 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.780+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94406105314864] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.780+0000 7f4dc9728800 20 remove_watcher() i=5 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.780+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94406105310176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f49b0 con 0x7f4db802c060 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.781+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94406105310176] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.781+0000 7f4dc9728800 20 remove_watcher() i=6 2024-04-23T19:11:31.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.781+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94406105318800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55dca25f2ae0 con 0x7f4db802c060 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94406105318800] v32'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4dbc05a040 con 0x7f4db802c060 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 20 remove_watcher() i=7 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4db802c060 msgr2=0x7f4db804b5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f4db802c060 0x7f4db804b5b0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca25e3d80 msgr2=0x55dca25e6240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca25e3d80 0x55dca25e6240 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4dc00123a0 msgr2=0x7f4dc002c480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f4dc00123a0 0x7f4dc002c480 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dca25e9f90 msgr2=0x55dca25ec380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.782+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55dca25e9f90 0x55dca25ec380 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c90036e60 msgr2=0x7f4c90039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c90036e60 0x7f4c90039320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4db8017a60 tx=0x7f4db8042000 comp rx=0 tx=0).stop 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc4c27640 1 -- 172.21.15.63:0/2687699287 reap_dead start 2024-04-23T19:11:31.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 msgr2=0x55dca25d80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d80f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0001bc0 tx=0x7f4dc004cc00 comp rx=0 tx=0).stop 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 shutdown_connections 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 --2- 172.21.15.63:0/2687699287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25d0bd0 0x55dca25d80f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 >> 172.21.15.63:0/2687699287 conn(0x55dca25cce50 msgr2=0x55dca25ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 shutdown_connections 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/2687699287 wait complete. 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c9c036eb0 msgr2=0x7f4c9c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c9c036eb0 0x7f4c9c039370 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4db800c010 tx=0x7f4db802ae80 comp rx=0 tx=0).stop 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25c3830 msgr2=0x55dca25c3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25c3830 0x55dca25c3c50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55dca25a8e20 tx=0x7f4dc00178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 shutdown_connections 2024-04-23T19:11:31.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4c9c036eb0 0x7f4c9c039370 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 --2- 172.21.15.63:0/317695590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca25c3830 0x55dca25c3c50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 >> 172.21.15.63:0/317695590 conn(0x55dca25b78d0 msgr2=0x55dca25b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 shutdown_connections 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.783+0000 7f4dc9728800 1 -- 172.21.15.63:0/317695590 wait complete. 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dca25b2b10 msgr2=0x55dca25b4f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dca25b2b10 0x55dca25b4f00 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca24b1570 msgr2=0x55dca25af350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca24b1570 0x55dca25af350 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4cac036e60 msgr2=0x7f4cac039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4cac036e60 0x7f4cac039320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc054000 tx=0x7f4dbc04e000 comp rx=0 tx=0).stop 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 msgr2=0x55dca25a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca25a8110 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4db8004000 tx=0x7f4db8000bc0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 shutdown_connections 2024-04-23T19:11:31.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55dca25b2b10 0x55dca25b4f00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55dca24b1570 0x55dca25af350 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4cac036e60 0x7f4cac039320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 --2- 172.21.15.63:0/3252677016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dca24ad180 0x55dca25a8110 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 >> 172.21.15.63:0/3252677016 conn(0x55dca24a87e0 msgr2=0x55dca2421930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 shutdown_connections 2024-04-23T19:11:31.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.784+0000 7f4dc9728800 1 -- 172.21.15.63:0/3252677016 wait complete. 2024-04-23T19:11:31.793 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-23T19:11:31.799 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:11:31.799 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-23T19:11:31.804 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-23T19:11:31.804 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-04-23T19:11:31.804 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-23T19:11:31.804 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-23T19:11:31.805 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-04-23T19:11:31.850 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:31.850 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:31.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.849+0000 7fe143f2d800 1 Processor -- start 2024-04-23T19:11:31.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.849+0000 7fe143f2d800 1 -- start start 2024-04-23T19:11:31.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.849+0000 7fe143f2d800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55a861aaf6b0 0x55a861ab1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.849+0000 7fe143f2d800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a86195d7f0 con 0x55a861aabba0 2024-04-23T19:11:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.849+0000 7fe143f2d800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55a86194ce20 con 0x55a861aaf6b0 2024-04-23T19:11:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.850+0000 7fe13f37e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55a861aaf6b0 0x55a861ab1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.850+0000 7fe13fb7f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55a861aabba0 0x55a861aadf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:41556/0 (socket says 172.21.15.63:41556) 2024-04-23T19:11:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.850+0000 7fe13f37e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55a861aaf6b0 0x55a861ab1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58800/0 (socket says 172.21.15.63:58800) 2024-04-23T19:11:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.850+0000 7fe13fb7f640 1 -- 172.21.15.63:0/4284368510 learned_addr learned my addr 172.21.15.63:0/4284368510 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:11:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.850+0000 7fe13f37e640 1 -- 172.21.15.63:0/4284368510 >> v1:172.21.15.16:6789/0 conn(0x55a861aabba0 legacy=0x55a861aadf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.850+0000 7fe13f37e640 1 -- 172.21.15.63:0/4284368510 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86196bb50 con 0x55a861aaf6b0 2024-04-23T19:11:31.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.852+0000 7fe13f37e640 1 --2- 172.21.15.63:0/4284368510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aaf6b0 0x55a861ab1aa0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe13400a470 tx=0x7fe13401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b1069ca590a680 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.852+0000 7fe13eb7d640 1 -- 172.21.15.63:0/4284368510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe134057070 con 0x55a861aaf6b0 2024-04-23T19:11:31.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.852+0000 7fe13eb7d640 1 -- 172.21.15.63:0/4284368510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe134052570 con 0x55a861aaf6b0 2024-04-23T19:11:31.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.852+0000 7fe13eb7d640 1 -- 172.21.15.63:0/4284368510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe134052870 con 0x55a861aaf6b0 2024-04-23T19:11:31.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.852+0000 7fe143f2d800 1 -- 172.21.15.63:0/4284368510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aaf6b0 msgr2=0x55a861ab1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.853+0000 7fe143f2d800 1 --2- 172.21.15.63:0/4284368510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aaf6b0 0x55a861ab1aa0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe13400a470 tx=0x7fe13401ce90 comp rx=0 tx=0).stop 2024-04-23T19:11:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.853+0000 7fe143f2d800 1 -- 172.21.15.63:0/4284368510 shutdown_connections 2024-04-23T19:11:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.853+0000 7fe143f2d800 1 --2- 172.21.15.63:0/4284368510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aaf6b0 0x55a861ab1aa0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.853+0000 7fe143f2d800 1 -- 172.21.15.63:0/4284368510 >> 172.21.15.63:0/4284368510 conn(0x55a861aa5630 msgr2=0x55a861aa7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.853+0000 7fe143f2d800 1 -- 172.21.15.63:0/4284368510 shutdown_connections 2024-04-23T19:11:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.853+0000 7fe143f2d800 1 -- 172.21.15.63:0/4284368510 wait complete. 2024-04-23T19:11:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.865+0000 7fe143f2d800 1 Processor -- start 2024-04-23T19:11:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.865+0000 7fe143f2d800 1 -- start start 2024-04-23T19:11:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.866+0000 7fe143f2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861aab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.866+0000 7fe143f2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8619f66f0 con 0x55a861aabba0 2024-04-23T19:11:31.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.866+0000 7fe13fb7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861aab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.866+0000 7fe13fb7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861aab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58804/0 (socket says 172.21.15.63:58804) 2024-04-23T19:11:31.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.866+0000 7fe13fb7f640 1 -- 172.21.15.63:0/2815607220 learned_addr learned my addr 172.21.15.63:0/2815607220 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.866+0000 7fe13fb7f640 1 -- 172.21.15.63:0/2815607220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a861aa9aa0 con 0x55a861aabba0 2024-04-23T19:11:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.867+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/2815607220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861aab4e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe130001d50 tx=0x7fe13000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fcf0ec76696fad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.867+0000 7fe02de0b640 1 -- 172.21.15.63:0/2815607220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe130013030 con 0x55a861aabba0 2024-04-23T19:11:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.867+0000 7fe02de0b640 1 -- 172.21.15.63:0/2815607220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13000e470 con 0x55a861aabba0 2024-04-23T19:11:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.867+0000 7fe02de0b640 1 -- 172.21.15.63:0/2815607220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe13000e810 con 0x55a861aabba0 2024-04-23T19:11:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 -- 172.21.15.63:0/2815607220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 msgr2=0x55a861aab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2815607220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861aab4e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe130001d50 tx=0x7fe13000b6c0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 -- 172.21.15.63:0/2815607220 shutdown_connections 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2815607220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861aab4e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 -- 172.21.15.63:0/2815607220 >> 172.21.15.63:0/2815607220 conn(0x55a861aa5630 msgr2=0x55a861b97c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 -- 172.21.15.63:0/2815607220 shutdown_connections 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 -- 172.21.15.63:0/2815607220 wait complete. 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 Processor -- start 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 -- start start 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861ba60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe143f2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a861ab22e0 con 0x55a861aabba0 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe13fb7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861ba60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe13fb7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861ba60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58818/0 (socket says 172.21.15.63:58818) 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.868+0000 7fe13fb7f640 1 -- 172.21.15.63:0/2954821196 learned_addr learned my addr 172.21.15.63:0/2954821196 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.869+0000 7fe13fb7f640 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a861ba65f0 con 0x55a861aabba0 2024-04-23T19:11:31.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.869+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861ba60b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55a861aaab10 tx=0x7fe130000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.869+0000 7fe027fff640 1 -- 172.21.15.63:0/2954821196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe130027070 con 0x55a861aabba0 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.869+0000 7fe027fff640 1 -- 172.21.15.63:0/2954821196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe130019900 con 0x55a861aabba0 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.869+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a861ba6880 con 0x55a861aabba0 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.869+0000 7fe027fff640 1 -- 172.21.15.63:0/2954821196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe130019c40 con 0x55a861aabba0 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.869+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a861ba9530 con 0x55a861aabba0 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.870+0000 7fe027fff640 1 -- 172.21.15.63:0/2954821196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe13002a070 con 0x55a861aabba0 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.870+0000 7fe027fff640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe028036e60 0x7fe028039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.870+0000 7fe027fff640 1 -- 172.21.15.63:0/2954821196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe13001f070 con 0x55a861aabba0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.870+0000 7fe13f37e640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe028036e60 0x7fe028039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.870+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861aaf6b0 0x55a861bad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.870+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861bad8c0 con 0x55a861aaf6b0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.871+0000 7fe140380640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861aaf6b0 0x55a861bad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.871+0000 7fe13f37e640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe028036e60 0x7fe028039320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe134054000 tx=0x7fe13404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.871+0000 7fe140380640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861aaf6b0 0x55a861bad2f0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.871+0000 7fe140380640 1 -- 172.21.15.63:0/2954821196 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe13800a4a0 con 0x55a861aaf6b0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.871+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861baa8a0 con 0x55a861aaf6b0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.871+0000 7fe140380640 1 -- 172.21.15.63:0/2954821196 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe13800a4a0 con 0x55a861aaf6b0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.871+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861baf6c0 con 0x55a861aaf6b0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.872+0000 7fe140380640 1 -- 172.21.15.63:0/2954821196 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe13800a4a0 con 0x55a861aaf6b0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.872+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861baea00 con 0x55a861aaf6b0 2024-04-23T19:11:31.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.872+0000 7fe140380640 1 -- 172.21.15.63:0/2954821196 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe13800a4a0 con 0x55a861aaf6b0 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.872+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a861bb0a70 0x55a861bb2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.872+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861bb33a0 con 0x55a861bb0a70 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.872+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a861bb0a70 0x55a861bb2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.873+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a861bb0a70 0x55a861bb2e60 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.873+0000 7fe13fb7f640 1 -- 172.21.15.63:0/2954821196 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe130029030 con 0x55a861bb0a70 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.873+0000 7fe143f2d800 1 Processor -- start 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.873+0000 7fe143f2d800 1 -- start start 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe143f2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bc1800 0x55a861bc1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe143f2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a86199ce30 con 0x55a861bc1800 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe140380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bc1800 0x55a861bc1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe140380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bc1800 0x55a861bc1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58824/0 (socket says 172.21.15.63:58824) 2024-04-23T19:11:31.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe140380640 1 -- 172.21.15.63:0/545071974 learned_addr learned my addr 172.21.15.63:0/545071974 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe140380640 1 -- 172.21.15.63:0/545071974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a861bc2160 con 0x55a861bc1800 2024-04-23T19:11:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe140380640 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bc1800 0x55a861bc1c20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55a861badca0 tx=0x7fe1380178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.874+0000 7fe0257fa640 1 -- 172.21.15.63:0/545071974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe138013440 con 0x55a861bc1800 2024-04-23T19:11:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.875+0000 7fe0257fa640 1 -- 172.21.15.63:0/545071974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1380135e0 con 0x55a861bc1800 2024-04-23T19:11:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.875+0000 7fe0257fa640 1 -- 172.21.15.63:0/545071974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1380138c0 con 0x55a861bc1800 2024-04-23T19:11:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.875+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a861bc2bd0 con 0x55a861bc1800 2024-04-23T19:11:31.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.875+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a861bc30d0 con 0x55a861bc1800 2024-04-23T19:11:31.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.875+0000 7fe0257fa640 1 -- 172.21.15.63:0/545071974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe138011040 con 0x55a861bc1800 2024-04-23T19:11:31.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.875+0000 7fe0257fa640 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe018036eb0 0x7fe018039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe0257fa640 1 -- 172.21.15.63:0/545071974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe13801d070 con 0x55a861bc1800 2024-04-23T19:11:31.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe018036eb0 0x7fe018039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe143f2d800 1 Processor -- start 2024-04-23T19:11:31.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe143f2d800 1 -- start start 2024-04-23T19:11:31.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe018036eb0 0x7fe018039370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a861baeda0 tx=0x7fe13000ef30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe143f2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bcef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe143f2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a861aa9e20 con 0x55a861bceb30 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe140380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bcef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe140380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bcef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58834/0 (socket says 172.21.15.63:58834) 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.876+0000 7fe140380640 1 -- 172.21.15.63:0/2394318742 learned_addr learned my addr 172.21.15.63:0/2394318742 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe140380640 1 -- 172.21.15.63:0/2394318742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a861bcf520 con 0x55a861bceb30 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe140380640 1 --2- 172.21.15.63:0/2394318742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bcef50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe13804cc70 tx=0x7fe13804ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c77ed42f94ee65 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe01effd640 1 -- 172.21.15.63:0/2394318742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe138042070 con 0x55a861bceb30 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe01effd640 1 -- 172.21.15.63:0/2394318742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe138012df0 con 0x55a861bceb30 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe01effd640 1 -- 172.21.15.63:0/2394318742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe138021b80 con 0x55a861bceb30 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe143f2d800 1 -- 172.21.15.63:0/2394318742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 msgr2=0x55a861bcef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2394318742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bcef50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe13804cc70 tx=0x7fe13804ce70 comp rx=0 tx=0).stop 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe143f2d800 1 -- 172.21.15.63:0/2394318742 shutdown_connections 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2394318742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bcef50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe143f2d800 1 -- 172.21.15.63:0/2394318742 >> 172.21.15.63:0/2394318742 conn(0x55a861bcae00 msgr2=0x55a861bc8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe143f2d800 1 -- 172.21.15.63:0/2394318742 shutdown_connections 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.877+0000 7fe143f2d800 1 -- 172.21.15.63:0/2394318742 wait complete. 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe143f2d800 1 Processor -- start 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe143f2d800 1 -- start start 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe143f2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bd6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe143f2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a861bcfcb0 con 0x55a861bceb30 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe140380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bd6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe140380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bd6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58840/0 (socket says 172.21.15.63:58840) 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 learned_addr learned my addr 172.21.15.63:0/1950005619 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:31.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a861bd65b0 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.878+0000 7fe140380640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bd6070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe13804bb90 tx=0x7fe13804cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.879+0000 7fe01d7fa640 1 -- 172.21.15.63:0/1950005619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe138015430 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.879+0000 7fe01d7fa640 1 -- 172.21.15.63:0/1950005619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe138001610 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.879+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a861bd6840 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.879+0000 7fe01d7fa640 1 -- 172.21.15.63:0/1950005619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe13802a480 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.879+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a861bd94f0 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.879+0000 7fe01d7fa640 1 -- 172.21.15.63:0/1950005619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe13802a620 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe01d7fa640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036eb0 0x7fe008039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe01d7fa640 1 -- 172.21.15.63:0/1950005619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe13800e040 con 0x55a861bceb30 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036eb0 0x7fe008039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036eb0 0x7fe008039370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe130017a60 tx=0x7fe130043000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861be1d30 0x55a861be41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861be4940 con 0x55a861be1d30 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.880+0000 7fe13f37e640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861be1d30 0x55a861be41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe13f37e640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861be1d30 0x55a861be41f0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 20 realm 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861be4d10 con 0x55a861be1d30 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:31.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861be55e0 con 0x55a861be1d30 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.881+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861be5980 con 0x55a861be1d30 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe143f2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861be61d0 con 0x55a861be1d30 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe143f2d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe143f2d800 20 searching for the correct realm 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a861be7f40 0x55a861bea330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe140380640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a861be7f40 0x55a861bea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.882+0000 7fe140380640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a861be7f40 0x55a861bea330 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.883+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.883+0000 7fe140380640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe1380123a0 0x7fe138042870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.883+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe1380123a0 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.883+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe1380123a0 0x7fe138042870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.883+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe1380123a0 0x7fe138042870 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.883+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.884+0000 7fe13fb7f640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe13002c060 0x7fe13002a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.884+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.884+0000 7fe13f37e640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe13002c060 0x7fe13002a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:31.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.884+0000 7fe13f37e640 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe13002c060 0x7fe13002a580 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.885+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe1380123a0 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.886+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe1380123a0 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.887+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.888+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.889+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe1380123a0 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe1380123a0 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.890+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.891+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.891+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe1380123a0 2024-04-23T19:11:31.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.891+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.891+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be7f40 2024-04-23T19:11:31.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.891+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.891+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe13002c060 2024-04-23T19:11:31.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x55a861be1d30 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a861bea8c0 con 0x7fe1380123a0 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861be6b80 con 0x55a861be1d30 2024-04-23T19:11:31.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.892+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861be7420 con 0x55a861be1d30 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861bead40 con 0x7fe13002c060 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 20 zone default found 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 4 Realm: () 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 20 zonegroup default 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861beb1f0 con 0x7fe1380123a0 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.893+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861becf70 con 0x55a861be7f40 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861bed420 con 0x55a861be7f40 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 20 started sync module instance, tier type = 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.894+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bec410 con 0x7fe1380123a0 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.895+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.895+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94181682759680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bedd30 con 0x7fe1380123a0 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.895+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bee580 con 0x55a861be1d30 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.896+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94181682759680] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.897+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.897+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94181682763904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bee5e0 con 0x55a861be1d30 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.897+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bef0f0 con 0x55a861be7f40 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.898+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.898+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94181682768800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bef990 con 0x55a861be7f40 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.898+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf0490 con 0x7fe1380123a0 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.898+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94181682763904] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.899+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94181682768800] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.899+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.899+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94181682766032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bef990 con 0x7fe1380123a0 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.899+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf0d30 con 0x55a861be1d30 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.900+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.900+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94181682766032] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.900+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94181682776192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf11f0 con 0x55a861be1d30 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.900+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bef500 con 0x7fe13002c060 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.901+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94181682776192] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.901+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.902+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94181682777248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf11b0 con 0x7fe13002c060 2024-04-23T19:11:31.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.902+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf2180 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.903+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94181682777248] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.903+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.903+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94181682780128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf20b0 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.903+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf2b30 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94181682780128] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94181682783776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf2af0 con 0x7fe13002c060 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 20 add_watcher() i=0 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 20 add_watcher() i=1 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 20 add_watcher() i=2 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 20 add_watcher() i=3 2024-04-23T19:11:31.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 20 add_watcher() i=4 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 20 add_watcher() i=5 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.904+0000 7fe143f2d800 20 add_watcher() i=6 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.905+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94181682783776] v32'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.905+0000 7fe143f2d800 20 add_watcher() i=7 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.905+0000 7fe143f2d800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.905+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf1cf0 con 0x55a861be1d30 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.905+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.905+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94181682786000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf1cf0 con 0x55a861be1d30 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.906+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94181682786000] v32'218 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.907+0000 7fdff5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.907+0000 7fe143f2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.907+0000 7fe143f2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:31.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.907+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfe910 con 0x55a861be1d30 2024-04-23T19:11:31.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.908+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'33 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.908+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfef60 con 0x7fe13002c060 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.910+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.910+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bff3a0 con 0x55a861be7f40 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.911+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.911+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfe440 con 0x55a861be1d30 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.912+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'219 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.912+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bff9d0 con 0x7fe1380123a0 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.913+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'68 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.913+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bffe80 con 0x55a861be7f40 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.914+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'45 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.914+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bffe80 con 0x55a861be1d30 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.915+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.915+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc7e0 con 0x7fe13002c060 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.916+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'45 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.916+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc7e0 con 0x7fe1380123a0 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.917+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe13003f070 con 0x7fe1380123a0 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.917+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc7e0 con 0x55a861be7f40 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.918+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.918+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc7e0 con 0x7fe1380123a0 2024-04-23T19:11:31.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.919+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'69 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.919+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc7e0 con 0x7fe13002c060 2024-04-23T19:11:31.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.920+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.920+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe1380123a0 2024-04-23T19:11:31.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.921+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.922+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe13002c060 2024-04-23T19:11:31.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.922+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'51 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.923+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be1d30 2024-04-23T19:11:31.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.923+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'220 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.924+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.925+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'63 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.925+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe1380123a0 2024-04-23T19:11:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.926+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'70 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.926+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.926+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.927+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be1d30 2024-04-23T19:11:31.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.928+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'34 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.928+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be1d30 2024-04-23T19:11:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.929+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'221 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.929+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.930+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'64 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.930+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe1380123a0 2024-04-23T19:11:31.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.931+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'36 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.931+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.932+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.932+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.933+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.933+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.934+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'22 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.934+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.935+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'46 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.935+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe13002c060 2024-04-23T19:11:31.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.936+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.936+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.937+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.937+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe13002c060 2024-04-23T19:11:31.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.938+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'46 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.938+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x55a861be7f40 2024-04-23T19:11:31.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.939+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'65 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.939+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe13002c060 2024-04-23T19:11:31.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.940+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'52 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13405a040 con 0x7fe13002c060 2024-04-23T19:11:31.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.940+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bfc610 con 0x7fe1380123a0 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.941+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.941+0000 7fdff4fe1640 20 reqs_thread_entry: start 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.941+0000 7fe143f2d800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.941+0000 7fe143f2d800 20 Filter name: none 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fdf6e7fc640 20 reqs_thread_entry: start 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c2b0a0 con 0x7fe1380123a0 2024-04-23T19:11:31.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(94 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:95 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c2d9d0 con 0x55a861be1d30 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(95 bud.buckets [call out=184b] v0'0 uv4 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 : miss 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.942+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:96 8.12 8:4a286bde:root::.bucket.meta.myfoo6%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c2d740 con 0x55a861be7f40 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(96 .bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 [call out=48b,stat out=16b,read 0~380 out=380b,getxattrs out=212b] v0'0 uv4 ondisk = 0) ==== 333+0+656 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 info.flags=0x17 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 to cache LRU end 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 4.35 4:aea617cc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caac90 con 0x55a861be1d30 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(97 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.943+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:98 4.2b 4:d630da47:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caac90 con 0x55a861be1d30 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.944+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(98 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.944+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:99 4.a 4:52db42c4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caac90 con 0x7fe1380123a0 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.944+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(99 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.944+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:100 4.15 4:aa91196c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caac90 con 0x55a861be7f40 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.944+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(100 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3 [call out=85b] v0'0 uv6 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.944+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 4.2 4:42dd7ebf:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caad10 con 0x55a861be1d30 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.945+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(101 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13405a040 con 0x55a861be1d30 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.945+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:102 4.39 4:9d2fe1d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caad10 con 0x55a861be7f40 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.945+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(102 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.945+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:103 4.3f 4:fc2e1960:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caad10 con 0x7fe1380123a0 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.945+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(103 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.945+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:104 4.39 4:9c4112ef:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caad10 con 0x55a861be7f40 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.946+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(104 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.946+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 4.27 4:e4002aad:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caad10 con 0x55a861be1d30 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.946+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(105 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.946+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:106 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caad10 con 0x55a861be7f40 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.946+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(106 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [call out=289b] v0'0 uv7 ondisk = 0) ==== 194+0+289 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.946+0000 7fe143f2d800 10 cls_bucket_list_unordered: got thirteen 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.946+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 4.18 4:1b88699c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caad10 con 0x7fe1380123a0 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(107 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe143f2d800 20 get_obj_state: octx=0x7fe134052c00 obj=myfoo6:thirteen state=0x55a861ca8d38 s->prefetch_data=0 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:108 3.27s0 3:e606443a:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861ca9b60 con 0x7fe13002c060 2024-04-23T19:11:31.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(108 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen [getxattrs out=819b,stat out=16b] v0'0 uv2 ondisk = 0) ==== 238+0+835 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe143f2d800 10 manifest: total_size = 8 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe143f2d800 20 get_obj_state: setting s->obj_tag to _mc1wEe2-Z8w3yHXFzDMZWP5FZhSaij4 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe143f2d800 20 get_obj_state: octx=0x7fe134052c00 obj=myfoo6:thirteen state=0x55a861ca8d38 s->prefetch_data=0 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe143f2d800 20 prepare_atomic_modification: state is not atomic. state=0x55a861ca8d38 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.947+0000 7fe143f2d800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.948+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:109 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861caa420 con 0x55a861be7f40 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.949+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(109 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [stat,call,call] v32'8 uv8 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.949+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:110 3.27s0 3:e606443a:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861caa420 con 0x7fe13002c060 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.950+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(110 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9_thirteen [call] v32'3 uv3 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.950+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:111 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c31e40 con 0x55a861be7f40 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.950+0000 7fe143f2d800 20 list_objects_ordered: starting attempt 1 2024-04-23T19:11:31.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.950+0000 7fe143f2d800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.950+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:112 4.35 4:aea617cc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c33c90 con 0x55a861be1d30 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.950+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:113 4.2b 4:d630da47:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c33c90 con 0x55a861be1d30 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.950+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:114 4.a 4:52db42c4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c31e40 con 0x7fe1380123a0 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:115 4.15 4:aa91196c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c33c90 con 0x55a861be7f40 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:116 4.2 4:42dd7ebf:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c31e40 con 0x55a861be1d30 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:117 4.39 4:9d2fe1d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c3f960 con 0x55a861be7f40 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(114 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(112 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(115 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3 [call out=85b] v0'0 uv6 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(113 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:118 4.3f 4:fc2e1960:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c34080 con 0x7fe1380123a0 2024-04-23T19:11:31.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:119 4.39 4:9c4112ef:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c36a00 con 0x55a861be7f40 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(116 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:120 4.27 4:e4002aad:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c36a00 con 0x55a861be1d30 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:121 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c34080 con 0x55a861be7f40 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:122 4.18 4:1b88699c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c413e0 con 0x7fe1380123a0 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.951+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(117 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(118 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(122 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(120 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== osd_op_reply(119 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== osd_op_reply(111 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [stat,call,call] v32'9 uv9 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe01effd640 20 handle_completion(): completion ok for obj=thirteen 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 36 ==== osd_op_reply(121 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [call out=124b] v0'0 uv9 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-23T19:11:31.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9])"; uploads.size()=0; is_truncated=0 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:123 4.35 4:aea617cc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caa340 con 0x55a861be1d30 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:124 4.2b 4:d630da47:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c40dc0 con 0x55a861be1d30 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:125 4.a 4:52db42c4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caa340 con 0x7fe1380123a0 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:126 4.15 4:aa91196c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c40dc0 con 0x55a861be7f40 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:127 4.2 4:42dd7ebf:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caa340 con 0x55a861be1d30 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:128 4.39 4:9d2fe1d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c365b0 con 0x55a861be7f40 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:129 4.3f 4:fc2e1960:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c302d0 con 0x7fe1380123a0 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:130 4.39 4:9c4112ef:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861caa340 con 0x55a861be7f40 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:131 4.27 4:e4002aad:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c44d80 con 0x55a861be1d30 2024-04-23T19:11:31.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:132 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c47720 con 0x55a861be7f40 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:133 4.18 4:1b88699c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c44d80 con 0x7fe1380123a0 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.952+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(125 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(129 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(133 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 37 ==== osd_op_reply(126 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3 [call out=85b] v0'0 uv6 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(123 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(124 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(127 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== osd_op_reply(131 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 38 ==== osd_op_reply(128 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 39 ==== osd_op_reply(132 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [call out=124b] v0'0 uv9 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 40 ==== osd_op_reply(130 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.953+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:134 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c3fd50 con 0x55a861be1d30 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.954+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(134 bud.buckets [call] v32'5 uv5 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.954+0000 7fe143f2d800 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.954+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.954+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:135 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c32c80 con 0x55a861be1d30 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.955+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(135 myfoo6 [call out=48b,read 0~166 out=166b] v0'0 uv17 ondisk = 0) ==== 192+0+214 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.955+0000 7fe143f2d800 20 rados_obj.operate() r=0 bl.length=166 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.955+0000 7fe143f2d800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x11 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.955+0000 7fe143f2d800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.955+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:136 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c32c80 con 0x55a861be1d30 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== osd_op_reply(136 myfoo6 [call,call,delete] v32'18 uv18 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe143f2d800 10 removing default.rgw.meta+root+myfoo6 from cache 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe143f2d800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-23T19:11:31.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 94181683046352 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c32bf0 con 0x7fe1380123a0 2024-04-23T19:11:31.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== watch-notify(notify (1) cookie 94181682766032 notify 137438953498 ret 0) ==== 192+0+0 (crc 0 0 0) 0x55a861bff1e0 con 0x7fe1380123a0 2024-04-23T19:11:31.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe01e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953498 cookie 94181682766032 notifier 4751 bl.length()=150 2024-04-23T19:11:31.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe01e7fc640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fdf74003320 con 0x7fe1380123a0 2024-04-23T19:11:31.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(137 notify.3 [notify cookie 94181683046352 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== watch-notify(notify_complete (2) cookie 94181683046352 notify 137438953498 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fe130061060 con 0x7fe1380123a0 2024-04-23T19:11:31.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.956+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:139 8.12 8:4a286bde:root::.bucket.meta.myfoo6%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c32c80 con 0x55a861be7f40 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.957+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 41 ==== osd_op_reply(139 .bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 [call,call,delete] v32'5 uv5 ondisk = 0) ==== 291+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.957+0000 7fe143f2d800 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 from cache 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.957+0000 7fe143f2d800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9, ofs0, ns] 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.957+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:140 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94181683025584 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c32c80 con 0x7fe1380123a0 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 34 ==== watch-notify(notify (1) cookie 94181682759680 notify 137438953499 ret 0) ==== 249+0+0 (crc 0 0 0) 0x7fe130061060 con 0x7fe1380123a0 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe01effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953499 cookie 94181682759680 notifier 4751 bl.length()=207 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe01effd640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:141 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fe004003d60 con 0x7fe1380123a0 2024-04-23T19:11:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 35 ==== osd_op_reply(140 notify.0 [notify cookie 94181683025584 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 36 ==== watch-notify(notify_complete (2) cookie 94181683025584 notify 137438953499 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a861ba9530 con 0x7fe1380123a0 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 37 ==== osd_op_reply(141 notify.0 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:142 4.35 4:aea617cc:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c327d0 con 0x55a861be1d30 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:143 4.2b 4:d630da47:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c330a0 con 0x55a861be1d30 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:144 4.a 4:52db42c4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c338f0 con 0x7fe1380123a0 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:145 4.15 4:aa91196c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c3c4d0 con 0x55a861be7f40 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:146 4.2 4:42dd7ebf:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c327d0 con 0x55a861be1d30 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:147 4.39 4:9d2fe1d3:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c338f0 con 0x55a861be7f40 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:148 4.3f 4:fc2e1960:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c3c4d0 con 0x7fe1380123a0 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:149 4.39 4:9c4112ef:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c338f0 con 0x55a861be7f40 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:150 4.27 4:e4002aad:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c3c4d0 con 0x55a861be1d30 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:151 4.3b 4:dccd4f51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c406b0 con 0x55a861be7f40 2024-04-23T19:11:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.958+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:152 4.18 4:1b88699c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c41190 con 0x7fe1380123a0 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.960+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(142 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.0 [delete] v32'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.960+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 42 ==== osd_op_reply(145 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.3 [delete] v32'7 uv7 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.960+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 38 ==== osd_op_reply(152 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.10 [delete] v32'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.960+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 43 ==== osd_op_reply(147 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.5 [delete] v32'3 uv3 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.962+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 44 ==== osd_op_reply(151 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.9 [delete] v32'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 45 ==== osd_op_reply(149 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.7 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 39 ==== osd_op_reply(144 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.2 [delete] v32'3 uv3 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 40 ==== osd_op_reply(148 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.6 [delete] v32'9 uv9 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(146 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.4 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(143 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.1 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(150 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9.8 [delete] v32'5 uv5 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe143f2d800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo6/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 : miss 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe143f2d800 20 rados->read ofs=0 len=0 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:153 6.0 6:020b0fa4:::pubsub..bucket.myfoo6%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c3c8f0 con 0x7fe13002c060 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(153 pubsub..bucket.myfoo6/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe143f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe143f2d800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo6/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 info.flags=0x0 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe143f2d800 10 adding default.rgw.log++pubsub..bucket.myfoo6/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 to cache LRU end 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.963+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:154 6.0 6:020b0fa4:::pubsub..bucket.myfoo6%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c3c8f0 con 0x7fe13002c060 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.964+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(154 pubsub..bucket.myfoo6/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.9 [delete] v32'53 uv42 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.964+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:155 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c2d0f0 con 0x55a861be1d30 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.965+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(155 bud.buckets [call] v32'6 uv6 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.965+0000 7fe143f2d800 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-04-23T19:11:31.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.965+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:156 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c2d0f0 con 0x55a861be1d30 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.966+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 49 ==== osd_op_reply(156 NX5QOQKC6BH2IDN8HC7A [delete] v32'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.966+0000 7fe143f2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.966+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:157 7.3 7:c609908c:::notify.5:head [notify cookie 94181683024112 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c337e0 con 0x7fe13002c060 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== watch-notify(notify (1) cookie 94181682777248 notify 137438953508 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55a861bfef60 con 0x7fe13002c060 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe01e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953508 cookie 94181682777248 notifier 4751 bl.length()=170 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(157 notify.5 [notify cookie 94181683024112 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe01e7fc640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:158 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fdf74003320 con 0x7fe13002c060 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(158 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== watch-notify(notify_complete (2) cookie 94181683024112 notify 137438953508 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a861bd94f0 con 0x7fe13002c060 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe143f2d800 10 removing email index: 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe143f2d800 10 removing user buckets index 2024-04-23T19:11:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.967+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:159 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c2d0f0 con 0x55a861be1d30 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.968+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 50 ==== osd_op_reply(159 bud.buckets [delete] v32'7 uv7 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.968+0000 7fe143f2d800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.968+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94181683024112 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c337e0 con 0x7fe13002c060 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.968+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== watch-notify(notify (1) cookie 94181682780128 notify 137438953509 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55a861bd94f0 con 0x7fe13002c060 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.968+0000 7fe01effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953509 cookie 94181682780128 notifier 4751 bl.length()=160 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.968+0000 7fe01effd640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fe004003d60 con 0x7fe13002c060 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.969+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== osd_op_reply(160 notify.6 [notify cookie 94181683024112 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.969+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.969+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== watch-notify(notify_complete (2) cookie 94181683024112 notify 137438953509 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a861bfeaa0 con 0x7fe13002c060 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.969+0000 7fe143f2d800 10 removing user index: bud 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.969+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:162 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861c2d0f0 con 0x7fe1380123a0 2024-04-23T19:11:31.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 41 ==== osd_op_reply(162 bud [call,call,delete] v32'3 uv3 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe143f2d800 10 removing default.rgw.meta+users.uid+bud from cache 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe143f2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94181683024112 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a861c40c60 con 0x7fe13002c060 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== watch-notify(notify (1) cookie 94181682777248 notify 137438953510 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55a861bfeaa0 con 0x7fe13002c060 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe01e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953510 cookie 94181682777248 notifier 4751 bl.length()=152 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(163 notify.5 [notify cookie 94181683024112 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.970+0000 7fe01e7fc640 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fdf74003320 con 0x7fe13002c060 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.971+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.971+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== watch-notify(notify_complete (2) cookie 94181683024112 notify 137438953510 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a861bfe8d0 con 0x7fe13002c060 2024-04-23T19:11:31.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.972+0000 7fdff5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.972+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94181682786000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x55a861be1d30 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.973+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 51 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 94181682786000] v32'222 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.973+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94181682759680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x7fe1380123a0 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.974+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 42 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 94181682759680] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.974+0000 7fe143f2d800 20 remove_watcher() i=0 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.974+0000 7fe143f2d800 2 removed watcher, disabling cache 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.974+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94181682763904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x55a861be1d30 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.975+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 52 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 94181682763904] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.975+0000 7fe143f2d800 20 remove_watcher() i=1 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.975+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94181682768800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x55a861be7f40 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.976+0000 7fe140380640 1 -- 172.21.15.63:0/1950005619 <== osd.3 v2:172.21.15.63:6816/1289558434 46 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 94181682768800] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe138044070 con 0x55a861be7f40 2024-04-23T19:11:31.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.976+0000 7fe143f2d800 20 remove_watcher() i=2 2024-04-23T19:11:31.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.976+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94181682766032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x7fe1380123a0 2024-04-23T19:11:31.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.977+0000 7fe13fb7f640 1 -- 172.21.15.63:0/1950005619 <== osd.2 v2:172.21.15.63:6808/753864785 43 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 94181682766032] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe13003fd70 con 0x7fe1380123a0 2024-04-23T19:11:31.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.977+0000 7fe143f2d800 20 remove_watcher() i=3 2024-04-23T19:11:31.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.977+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94181682776192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x55a861be1d30 2024-04-23T19:11:31.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.978+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.1 v2:172.21.15.63:6800/3216879695 53 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 94181682776192] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x55a861be1d30 2024-04-23T19:11:31.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.979+0000 7fe143f2d800 20 remove_watcher() i=4 2024-04-23T19:11:31.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.979+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94181682777248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf0880 con 0x7fe13002c060 2024-04-23T19:11:31.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.980+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 94181682777248] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.980+0000 7fe143f2d800 20 remove_watcher() i=5 2024-04-23T19:11:31.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.980+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94181682780128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x7fe13002c060 2024-04-23T19:11:31.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.981+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 94181682780128] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.981+0000 7fe143f2d800 20 remove_watcher() i=6 2024-04-23T19:11:31.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.981+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94181682783776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a861bf9d00 con 0x7fe13002c060 2024-04-23T19:11:31.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 94181682783776] v32'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe134097060 con 0x7fe13002c060 2024-04-23T19:11:31.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 20 remove_watcher() i=7 2024-04-23T19:11:31.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe13002c060 msgr2=0x7fe13002a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe13002c060 0x7fe13002a580 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861be1d30 msgr2=0x55a861be41f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861be1d30 0x55a861be41f0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe1380123a0 msgr2=0x7fe138042870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe1380123a0 0x7fe138042870 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a861be7f40 msgr2=0x55a861bea330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a861be7f40 0x55a861bea330 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036eb0 msgr2=0x7fe008039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036eb0 0x7fe008039370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe130017a60 tx=0x7fe130043000 comp rx=0 tx=0).stop 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe13f37e640 1 -- 172.21.15.63:0/1950005619 reap_dead start 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 msgr2=0x55a861bd6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.982+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bd6070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe13804bb90 tx=0x7fe13804cc00 comp rx=0 tx=0).stop 2024-04-23T19:11:31.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 shutdown_connections 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/1950005619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bceb30 0x55a861bd6070 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 >> 172.21.15.63:0/1950005619 conn(0x55a861bcae00 msgr2=0x55a861bc8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 shutdown_connections 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/1950005619 wait complete. 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe018036eb0 msgr2=0x7fe018039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe018036eb0 0x7fe018039370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a861baeda0 tx=0x7fe13000ef30 comp rx=0 tx=0).stop 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bc1800 msgr2=0x55a861bc1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bc1800 0x55a861bc1c20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55a861badca0 tx=0x7fe1380178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 shutdown_connections 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe018036eb0 0x7fe018039370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/545071974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861bc1800 0x55a861bc1c20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 >> 172.21.15.63:0/545071974 conn(0x55a861bb58a0 msgr2=0x55a861bb5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 shutdown_connections 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/545071974 wait complete. 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a861bb0a70 msgr2=0x55a861bb2e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a861bb0a70 0x55a861bb2e60 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861aaf6b0 msgr2=0x55a861bad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861aaf6b0 0x55a861bad2f0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe028036e60 msgr2=0x7fe028039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe028036e60 0x7fe028039320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe134054000 tx=0x7fe13404e000 comp rx=0 tx=0).stop 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 msgr2=0x55a861ba60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.983+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861ba60b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55a861aaab10 tx=0x7fe130000bc0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 shutdown_connections 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a861bb0a70 0x55a861bb2e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a861aaf6b0 0x55a861bad2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe028036e60 0x7fe028039320 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 --2- 172.21.15.63:0/2954821196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a861aabba0 0x55a861ba60b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 >> 172.21.15.63:0/2954821196 conn(0x55a861aa5630 msgr2=0x55a861aaa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 shutdown_connections 2024-04-23T19:11:31.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:31.984+0000 7fe143f2d800 1 -- 172.21.15.63:0/2954821196 wait complete. 2024-04-23T19:11:31.993 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-23T19:11:31.993 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-23T19:11:31.994 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-04-23T19:11:32.039 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:11:32.039 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:11:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.038+0000 7f2586f40800 1 Processor -- start 2024-04-23T19:11:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.038+0000 7f2586f40800 1 -- start start 2024-04-23T19:11:32.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.038+0000 7f2586f40800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e053395180 0x55e0533955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.038+0000 7f2586f40800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e0532477f0 con 0x55e053396340 2024-04-23T19:11:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.038+0000 7f2586f40800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55e053236e20 con 0x55e053395180 2024-04-23T19:11:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.038+0000 7f2584a3f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e053395180 0x55e0533955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.039+0000 7f2584a3f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55e053395180 0x55e0533955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58848/0 (socket says 172.21.15.63:58848) 2024-04-23T19:11:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.039+0000 7f2584a3f640 1 -- 172.21.15.63:0/1450972788 learned_addr learned my addr 172.21.15.63:0/1450972788 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.039+0000 7f2584a3f640 1 -- 172.21.15.63:0/1450972788 >> v1:172.21.15.16:6789/0 conn(0x55e053396340 legacy=0x55e0533967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.039+0000 7f2584a3f640 1 -- 172.21.15.63:0/1450972788 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e053255b50 con 0x55e053395180 2024-04-23T19:11:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2584a3f640 1 --2- 172.21.15.63:0/1450972788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0533955a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f257400a520 tx=0x7f257401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239e436ac83fa41e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2583a3d640 1 -- 172.21.15.63:0/1450972788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2574057070 con 0x55e053395180 2024-04-23T19:11:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2583a3d640 1 -- 172.21.15.63:0/1450972788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574052570 con 0x55e053395180 2024-04-23T19:11:32.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2583a3d640 1 -- 172.21.15.63:0/1450972788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f257404ae60 con 0x55e053395180 2024-04-23T19:11:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2586f40800 1 -- 172.21.15.63:0/1450972788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 msgr2=0x55e0533955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2586f40800 1 --2- 172.21.15.63:0/1450972788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0533955a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f257400a520 tx=0x7f257401d010 comp rx=0 tx=0).stop 2024-04-23T19:11:32.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2586f40800 1 -- 172.21.15.63:0/1450972788 shutdown_connections 2024-04-23T19:11:32.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2586f40800 1 --2- 172.21.15.63:0/1450972788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0533955a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.041+0000 7f2586f40800 1 -- 172.21.15.63:0/1450972788 >> 172.21.15.63:0/1450972788 conn(0x55e0533907e0 msgr2=0x55e053392c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:32.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.042+0000 7f2586f40800 1 -- 172.21.15.63:0/1450972788 shutdown_connections 2024-04-23T19:11:32.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.042+0000 7f2586f40800 1 -- 172.21.15.63:0/1450972788 wait complete. 2024-04-23T19:11:32.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.054+0000 7f2586f40800 1 Processor -- start 2024-04-23T19:11:32.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.054+0000 7f2586f40800 1 -- start start 2024-04-23T19:11:32.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.054+0000 7f2586f40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0534867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.054+0000 7f2586f40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e0532e06f0 con 0x55e053395180 2024-04-23T19:11:32.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2584a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0534867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2584a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0534867e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58854/0 (socket says 172.21.15.63:58854) 2024-04-23T19:11:32.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2584a3f640 1 -- 172.21.15.63:0/1626629972 learned_addr learned my addr 172.21.15.63:0/1626629972 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:32.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2584a3f640 1 -- 172.21.15.63:0/1626629972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e053486db0 con 0x55e053395180 2024-04-23T19:11:32.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2584a3f640 1 --2- 172.21.15.63:0/1626629972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0534867e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2574055040 tx=0x7f257404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbd3d5a7b1ccf9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2472e0d640 1 -- 172.21.15.63:0/1626629972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2574057040 con 0x55e053395180 2024-04-23T19:11:32.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2472e0d640 1 -- 172.21.15.63:0/1626629972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574052720 con 0x55e053395180 2024-04-23T19:11:32.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.055+0000 7f2472e0d640 1 -- 172.21.15.63:0/1626629972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2574052a20 con 0x55e053395180 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 -- 172.21.15.63:0/1626629972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 msgr2=0x55e0534867e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 --2- 172.21.15.63:0/1626629972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0534867e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2574055040 tx=0x7f257404c000 comp rx=0 tx=0).stop 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 -- 172.21.15.63:0/1626629972 shutdown_connections 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 --2- 172.21.15.63:0/1626629972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e0534867e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 -- 172.21.15.63:0/1626629972 >> 172.21.15.63:0/1626629972 conn(0x55e0533907e0 msgr2=0x55e053482100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 -- 172.21.15.63:0/1626629972 shutdown_connections 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 -- 172.21.15.63:0/1626629972 wait complete. 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.056+0000 7f2586f40800 1 Processor -- start 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2586f40800 1 -- start start 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2586f40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e05348e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2586f40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e05339e7b0 con 0x55e053395180 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2584a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e05348e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2584a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e05348e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58866/0 (socket says 172.21.15.63:58866) 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2584a3f640 1 -- 172.21.15.63:0/1184392951 learned_addr learned my addr 172.21.15.63:0/1184392951 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2584a3f640 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05348e750 con 0x55e053395180 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.057+0000 7f2584a3f640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e05348e210 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f25740524f0 tx=0x7f2574003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.058+0000 7f247160a640 1 -- 172.21.15.63:0/1184392951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2574073070 con 0x55e053395180 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.058+0000 7f247160a640 1 -- 172.21.15.63:0/1184392951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574052d80 con 0x55e053395180 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.058+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e05348e9e0 con 0x55e053395180 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.058+0000 7f247160a640 1 -- 172.21.15.63:0/1184392951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2574057070 con 0x55e053395180 2024-04-23T19:11:32.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.058+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e053491690 con 0x55e053395180 2024-04-23T19:11:32.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.058+0000 7f247160a640 1 -- 172.21.15.63:0/1184392951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f2574076070 con 0x55e053395180 2024-04-23T19:11:32.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f247160a640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f246c036e60 0x7f246c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f247160a640 1 -- 172.21.15.63:0/1184392951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f257408e300 con 0x55e053395180 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f258423e640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f246c036e60 0x7f246c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e053396340 0x55e053495450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053495a30 con 0x55e053396340 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f2585240640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e053396340 0x55e053495450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f258423e640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f246c036e60 0x7f246c039320 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2578001e10 tx=0x7f2578001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.059+0000 7f2585240640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e053396340 0x55e053495450 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2585240640 1 -- 172.21.15.63:0/1184392951 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f257c00a4a0 con 0x55e053396340 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534962c0 con 0x55e053396340 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2585240640 1 -- 172.21.15.63:0/1184392951 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f257c00a4a0 con 0x55e053396340 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053497670 con 0x55e053396340 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2585240640 1 -- 172.21.15.63:0/1184392951 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f257c00a4a0 con 0x55e053396340 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053496b10 con 0x55e053396340 2024-04-23T19:11:32.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2585240640 1 -- 172.21.15.63:0/1184392951 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f257c00a4a0 con 0x55e053396340 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e053498a20 0x55e05349ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.060+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05349b550 con 0x55e053498a20 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.061+0000 7f2584a3f640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e053498a20 0x55e05349ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.061+0000 7f2584a3f640 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e053498a20 0x55e05349ae10 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.061+0000 7f2584a3f640 1 -- 172.21.15.63:0/1184392951 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2574075030 con 0x55e053498a20 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2586f40800 1 Processor -- start 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2586f40800 1 -- start start 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2586f40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534a99e0 0x55e0534a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2586f40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e053286e30 con 0x55e0534a99e0 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2585240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534a99e0 0x55e0534a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2585240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534a99e0 0x55e0534a9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58874/0 (socket says 172.21.15.63:58874) 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2585240640 1 -- 172.21.15.63:0/2830626722 learned_addr learned my addr 172.21.15.63:0/2830626722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:32.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.062+0000 7f2585240640 1 -- 172.21.15.63:0/2830626722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0534aa340 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.063+0000 7f2585240640 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534a99e0 0x55e0534a9e00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55e05348ef20 tx=0x7f257c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.063+0000 7f246a7fc640 1 -- 172.21.15.63:0/2830626722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f257c013440 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.063+0000 7f246a7fc640 1 -- 172.21.15.63:0/2830626722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0135e0 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.063+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e0534aadb0 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.063+0000 7f246a7fc640 1 -- 172.21.15.63:0/2830626722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f257c0138c0 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.063+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0534ab2b0 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f246a7fc640 1 -- 172.21.15.63:0/2830626722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f257c011040 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f246a7fc640 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f245c036eb0 0x7f245c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f246a7fc640 1 -- 172.21.15.63:0/2830626722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f257c01d070 con 0x55e0534a99e0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f2584a3f640 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f245c036eb0 0x7f245c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f2586f40800 1 Processor -- start 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f2584a3f640 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f245c036eb0 0x7f245c039370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2574077520 tx=0x7f257408c510 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f2586f40800 1 -- start start 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f2586f40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f2586f40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e053487070 con 0x55e0534b6cc0 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.064+0000 7f2585240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.065+0000 7f2585240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534b70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58882/0 (socket says 172.21.15.63:58882) 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.065+0000 7f2585240640 1 -- 172.21.15.63:0/576082817 learned_addr learned my addr 172.21.15.63:0/576082817 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.065+0000 7f2585240640 1 -- 172.21.15.63:0/576082817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0534b76b0 con 0x55e0534b6cc0 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.065+0000 7f2585240640 1 --2- 172.21.15.63:0/576082817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534b70e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f257c04cca0 tx=0x7f257c012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1b641d8e29ce60 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.065+0000 7f245bfff640 1 -- 172.21.15.63:0/576082817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f257c042030 con 0x55e0534b6cc0 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.065+0000 7f245bfff640 1 -- 172.21.15.63:0/576082817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c001310 con 0x55e0534b6cc0 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.065+0000 7f245bfff640 1 -- 172.21.15.63:0/576082817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f257c027040 con 0x55e0534b6cc0 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 -- 172.21.15.63:0/576082817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 msgr2=0x55e0534b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 --2- 172.21.15.63:0/576082817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534b70e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f257c04cca0 tx=0x7f257c012a00 comp rx=0 tx=0).stop 2024-04-23T19:11:32.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 -- 172.21.15.63:0/576082817 shutdown_connections 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 --2- 172.21.15.63:0/576082817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534b70e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 -- 172.21.15.63:0/576082817 >> 172.21.15.63:0/576082817 conn(0x55e0534b3020 msgr2=0x55e0534b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 -- 172.21.15.63:0/576082817 shutdown_connections 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 -- 172.21.15.63:0/576082817 wait complete. 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 Processor -- start 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 -- start start 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534be240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2586f40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e0534b7dc0 con 0x55e0534b6cc0 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2585240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2585240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:58886/0 (socket says 172.21.15.63:58886) 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.066+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 learned_addr learned my addr 172.21.15.63:0/2086995173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.067+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0534be780 con 0x55e0534b6cc0 2024-04-23T19:11:32.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.067+0000 7f2585240640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534be240 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f257c001df0 tx=0x7f257c04cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.067+0000 7f245a7fc640 1 -- 172.21.15.63:0/2086995173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f257c027030 con 0x55e0534b6cc0 2024-04-23T19:11:32.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.067+0000 7f245a7fc640 1 -- 172.21.15.63:0/2086995173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c02a350 con 0x55e0534b6cc0 2024-04-23T19:11:32.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.067+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e0534bea10 con 0x55e0534b6cc0 2024-04-23T19:11:32.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.067+0000 7f245a7fc640 1 -- 172.21.15.63:0/2086995173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f257c02a650 con 0x55e0534b6cc0 2024-04-23T19:11:32.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.067+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0534c16c0 con 0x55e0534b6cc0 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.068+0000 7f245a7fc640 1 -- 172.21.15.63:0/2086995173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f257c045070 con 0x55e0534b6cc0 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.068+0000 7f245a7fc640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2450036eb0 0x7f2450039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.068+0000 7f245a7fc640 1 -- 172.21.15.63:0/2086995173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f257c043070 con 0x55e0534b6cc0 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.068+0000 7f2584a3f640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2450036eb0 0x7f2450039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e0534c9f20 0x55e0534cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534ccb30 con 0x55e0534c9f20 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2584a3f640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2450036eb0 0x7f2450039370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2574052fd0 tx=0x7f2574087000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f258423e640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e0534c9f20 0x55e0534cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f258423e640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e0534c9f20 0x55e0534cc3e0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f257800bdb0 con 0x55e0534c9f20 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 20 realm 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534c9370 con 0x55e0534c9f20 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f257800bdb0 con 0x55e0534c9f20 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.069+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534cd620 con 0x55e0534c9f20 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f257800bdb0 con 0x55e0534c9f20 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534cdef0 con 0x55e0534c9f20 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f257800bdb0 con 0x55e0534c9f20 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534ce290 con 0x55e0534c9f20 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f257800bdb0 con 0x55e0534c9f20 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 20 searching for the correct realm 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e0534d0170 0x55e0534d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.070+0000 7f2585240640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e0534d0170 0x55e0534d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.071+0000 7f2585240640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e0534d0170 0x55e0534d2580 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.071+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.071+0000 7f2585240640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f257c0123a0 0x7f257c046790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.071+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257c0123a0 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.071+0000 7f2584a3f640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f257c0123a0 0x7f257c046790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.072+0000 7f2584a3f640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f257c0123a0 0x7f257c046790 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.072+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.072+0000 7f2584a3f640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f257409c040 0x7f257406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.072+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.072+0000 7f258423e640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f257409c040 0x7f257406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.072+0000 7f258423e640 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f257409c040 0x7f257406f580 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:11:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257800bdb0 con 0x7f257409c040 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257800bdb0 con 0x55e0534c9f20 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.073+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.074+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.074+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257c0123a0 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.074+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.074+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.074+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.074+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.075+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.075+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.075+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.075+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.075+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.075+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257c0123a0 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.076+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.076+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.076+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.076+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.076+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.076+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.077+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.078+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.078+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.078+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.078+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257c0123a0 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.078+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.078+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257c0123a0 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.079+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257c0123a0 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.080+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.080+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534d0170 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.080+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.080+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257409c040 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.080+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.080+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x55e0534c9f20 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d2c50 con 0x7f257c0123a0 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534cebe0 con 0x55e0534c9f20 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534cf480 con 0x55e0534c9f20 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.081+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d30d0 con 0x7f257409c040 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 zone default found 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 4 Realm: () 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 zonegroup default 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d3580 con 0x7f257c0123a0 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d5300 con 0x55e0534d0170 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d57c0 con 0x55e0534d0170 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 started sync module instance, tier type = 2024-04-23T19:11:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.082+0000 7f2586f40800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.083+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d47a0 con 0x7f257c0123a0 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.084+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.084+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94421958609808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d60d0 con 0x7f257c0123a0 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.084+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d6920 con 0x55e0534c9f20 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.085+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94421958609808] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.085+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.085+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94421958614048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d69a0 con 0x55e0534c9f20 2024-04-23T19:11:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.085+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d74c0 con 0x55e0534d0170 2024-04-23T19:11:32.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.086+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.086+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94421958619024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d7d60 con 0x55e0534d0170 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.086+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d8850 con 0x7f257c0123a0 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.086+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94421958614048] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.087+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94421958619024] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.087+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.087+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94421958616176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d7d60 con 0x7f257c0123a0 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.087+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d90f0 con 0x55e0534c9f20 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.088+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.088+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94421958626848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d9990 con 0x55e0534c9f20 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.088+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534da460 con 0x7f257409c040 2024-04-23T19:11:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.088+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94421958616176] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.089+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94421958626848] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.089+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.089+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94421958631248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d9490 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.089+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534dad00 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.090+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94421958631248] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.090+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.090+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94421958621360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d9490 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.090+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534db320 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94421958621360] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94421958635120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534db290 con 0x7f257409c040 2024-04-23T19:11:32.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 20 add_watcher() i=0 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 20 add_watcher() i=1 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 20 add_watcher() i=2 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 20 add_watcher() i=3 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 20 add_watcher() i=4 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 20 add_watcher() i=5 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.091+0000 7f2586f40800 20 add_watcher() i=6 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.092+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94421958635120] v32'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.092+0000 7f2586f40800 20 add_watcher() i=7 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.092+0000 7f2586f40800 2 all 8 watchers are set, enabling cache 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.092+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0534de690 con 0x55e0534c9f20 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.093+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.093+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94421958633472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534dbb80 con 0x55e0534c9f20 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.094+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94421958633472] v32'223 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.094+0000 7f243afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.094+0000 7f2586f40800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.094+0000 7f2586f40800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.094+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e7050 con 0x55e0534c9f20 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.095+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'34 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.095+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e76a0 con 0x7f257409c040 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.096+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.096+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e7ae0 con 0x55e0534d0170 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.097+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.097+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e7af0 con 0x55e0534c9f20 2024-04-23T19:11:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.098+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'224 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.098+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8060 con 0x7f257c0123a0 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.099+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'71 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.099+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8480 con 0x55e0534d0170 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.100+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'47 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.100+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8480 con 0x55e0534c9f20 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.102+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.102+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8480 con 0x7f257409c040 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.103+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'47 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.103+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8480 con 0x7f257c0123a0 2024-04-23T19:11:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.104+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.104+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8480 con 0x55e0534d0170 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.105+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.105+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8480 con 0x7f257c0123a0 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.106+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'72 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.106+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8570 con 0x7f257409c040 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.107+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.107+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e8570 con 0x7f257c0123a0 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.108+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.108+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e6ab0 con 0x7f257409c040 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.109+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'54 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.109+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e6ab0 con 0x55e0534c9f20 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.110+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'225 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.110+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e6a80 con 0x55e0534d0170 2024-04-23T19:11:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.111+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'66 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.111+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e6a80 con 0x7f257c0123a0 2024-04-23T19:11:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.112+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'73 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.112+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e6a80 con 0x55e0534d0170 2024-04-23T19:11:32.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.113+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.113+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e6a80 con 0x55e0534c9f20 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.114+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'35 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.114+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534c9f20 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.115+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'226 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.115+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534d0170 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.116+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'67 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.116+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x7f257c0123a0 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.117+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'37 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.117+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534d0170 2024-04-23T19:11:32.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.118+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.118+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534d0170 2024-04-23T19:11:32.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.119+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.119+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534d0170 2024-04-23T19:11:32.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.120+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'23 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.120+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534d0170 2024-04-23T19:11:32.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.121+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'48 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.121+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x7f257409c040 2024-04-23T19:11:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.122+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.122+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534d0170 2024-04-23T19:11:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.123+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.123+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x7f257409c040 2024-04-23T19:11:32.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.125+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'48 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.125+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x55e0534d0170 2024-04-23T19:11:32.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.126+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'68 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.126+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x7f257409c040 2024-04-23T19:11:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.127+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'55 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.127+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e69f0 con 0x7f257c0123a0 2024-04-23T19:11:32.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f2439fe3640 20 reqs_thread_entry: start 2024-04-23T19:11:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f2586f40800 20 init_complete bucket index max shards: 11 2024-04-23T19:11:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f2586f40800 20 Filter name: none 2024-04-23T19:11:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f23ab7fe640 20 reqs_thread_entry: start 2024-04-23T19:11:32.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f2586f40800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.128+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053513880 con 0x7f257409c040 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 20 rados_obj.operate() r=0 bl.length=380 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:11:32.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e0535161b0 con 0x7f257409c040 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(95 fud.buckets [call out=186b] v0'0 uv5 ondisk = 0) ==== 155+0+186 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 : miss 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.129+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:96 8.f 8:f206e60c:root::.bucket.meta.myfoo7%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053515f30 con 0x55e0534d0170 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(96 .bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 [call out=48b,stat out=16b,read 0~382 out=382b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 334+0+664 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2586f40800 20 rados_obj.operate() r=0 bl.length=382 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2586f40800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 info.flags=0x17 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2586f40800 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 to cache LRU end 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2586f40800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2586f40800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2586f40800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 2024-04-23T19:11:32.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.130+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:97 4.4 4:21037e61:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593440 con 0x7f257409c040 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.131+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(97 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.131+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:98 4.c 4:335aec59:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593480 con 0x55e0534c9f20 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.131+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(98 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.131+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:99 4.6 4:613d2a51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593480 con 0x55e0534d0170 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.131+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(99 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.131+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 4.3 4:c3be44e0:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593480 con 0x55e0534c9f20 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(100 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:101 4.a 4:51a5c25f:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593480 con 0x7f257c0123a0 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(101 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:102 4.3f 4:fcc88c01:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593480 con 0x7f257c0123a0 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(102 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5 [call out=377b] v0'0 uv8 ondisk = 0) ==== 195+0+377 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f2586f40800 10 cls_bucket_list_unordered: got fourteen 2024-04-23T19:11:32.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 4.20 4:05182548:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593480 con 0x55e0534c9f20 2024-04-23T19:11:32.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.132+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(103 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.133+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:104 4.36 4:6c6b2818:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593440 con 0x55e0534c9f20 2024-04-23T19:11:32.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.133+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(104 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.133+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:105 4.1b 4:d93e2e9b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593440 con 0x7f257c0123a0 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.133+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(105 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.133+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:106 4.1e 4:7a4a77b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593440 con 0x7f257409c040 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.134+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(106 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.134+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 4.3e 4:7c76a7d2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053593440 con 0x7f257c0123a0 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.134+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(107 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10 [call out=85b] v0'0 uv19 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.134+0000 7f2586f40800 20 get_obj_state: octx=0x7f25780105a0 obj=myfoo7:fourteen state=0x55e0535914e8 s->prefetch_data=0 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.134+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:108 3.31s0 3:8f3b6244:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053592310 con 0x7f257409c040 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.135+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(108 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10_fourteen [getxattrs out=856b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 239+0+872 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.135+0000 7f2586f40800 10 manifest: total_size = 8 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.135+0000 7f2586f40800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.6687987019608106373 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.135+0000 7f2586f40800 20 get_obj_state: octx=0x7f25780105a0 obj=myfoo7:fourteen state=0x55e0535914e8 s->prefetch_data=0 2024-04-23T19:11:32.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.135+0000 7f2586f40800 20 prepare_atomic_modification: state is not atomic. state=0x55e0535914e8 2024-04-23T19:11:32.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.135+0000 7f2586f40800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5 2024-04-23T19:11:32.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.135+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:109 4.3f 4:fcc88c01:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=188b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053592c50 con 0x7f257c0123a0 2024-04-23T19:11:32.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.136+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(109 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5 [stat,call,call] v32'10 uv10 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.136+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:110 3.31s0 3:8f3b6244:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053592c50 con 0x7f257409c040 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(110 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10_fourteen [call] v32'4 uv4 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:111 4.3f 4:fcc88c01:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=265b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05351a630 con 0x7f257c0123a0 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f2586f40800 20 list_objects_ordered: starting attempt 1 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f2586f40800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:112 4.4 4:21037e61:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351a630 con 0x7f257409c040 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:113 4.c 4:335aec59:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351e8c0 con 0x55e0534c9f20 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:114 4.6 4:613d2a51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351a630 con 0x55e0534d0170 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.138+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:115 4.3 4:c3be44e0:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053523660 con 0x55e0534c9f20 2024-04-23T19:11:32.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:116 4.a 4:51a5c25f:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053526250 con 0x7f257c0123a0 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(112 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(113 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:117 4.3f 4:fcc88c01:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053526250 con 0x7f257c0123a0 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(114 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:118 4.20 4:05182548:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351ecc0 con 0x55e0534c9f20 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:119 4.36 4:6c6b2818:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520650 con 0x55e0534c9f20 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:120 4.1b 4:d93e2e9b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351ecc0 con 0x7f257c0123a0 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:121 4.1e 4:7a4a77b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05352bc60 con 0x7f257409c040 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:122 4.3e 4:7c76a7d2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05352e650 con 0x7f257c0123a0 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(116 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(121 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(120 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.139+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(122 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10 [call out=85b] v0'0 uv19 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(115 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(118 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(111 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5 [stat,call,call] v32'11 uv11 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f245bfff640 20 handle_completion(): completion ok for obj=fourteen 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(119 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(117 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5 [call out=124b] v0'0 uv11 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-23T19:11:32.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10])"; uploads.size()=0; is_truncated=0 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:123 4.4 4:21037e61:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520af0 con 0x7f257409c040 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:124 4.c 4:335aec59:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053525b70 con 0x55e0534c9f20 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:125 4.6 4:613d2a51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351bf40 con 0x55e0534d0170 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:126 4.3 4:c3be44e0:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053525b70 con 0x55e0534c9f20 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:127 4.a 4:51a5c25f:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351bf40 con 0x7f257c0123a0 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:128 4.3f 4:fcc88c01:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05351a630 con 0x7f257c0123a0 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:129 4.20 4:05182548:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053525b70 con 0x55e0534c9f20 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:130 4.36 4:6c6b2818:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05352b410 con 0x55e0534c9f20 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:131 4.1b 4:d93e2e9b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053525b70 con 0x7f257c0123a0 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.140+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:132 4.1e 4:7a4a77b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e05352b8d0 con 0x7f257409c040 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:133 4.3e 4:7c76a7d2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053532f50 con 0x7f257c0123a0 2024-04-23T19:11:32.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(123 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(125 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(127 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(124 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(126 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(128 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5 [call out=124b] v0'0 uv11 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(129 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== osd_op_reply(131 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(130 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== osd_op_reply(133 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10 [call out=85b] v0'0 uv19 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== osd_op_reply(132 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.141+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:134 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0535201a0 con 0x7f257409c040 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.142+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(134 fud.buckets [call] v32'6 uv6 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.142+0000 7f2586f40800 10 cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.142+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.142+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:135 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x7f257c0123a0 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.143+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 34 ==== osd_op_reply(135 myfoo7 [call out=48b,read 0~168 out=168b] v0'0 uv44 ondisk = 0) ==== 192+0+216 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.143+0000 7f2586f40800 20 rados_obj.operate() r=0 bl.length=168 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.143+0000 7f2586f40800 10 cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x11 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.143+0000 7f2586f40800 10 adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.143+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:136 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053525b10 con 0x7f257c0123a0 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 35 ==== osd_op_reply(136 myfoo7 [call,call,delete] v32'45 uv45 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f2586f40800 10 removing default.rgw.meta+root+myfoo7 from cache 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f2586f40800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 94421959386912 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x7f257c0123a0 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 36 ==== watch-notify(notify (1) cookie 94421958616176 notify 137438953500 ret 0) ==== 192+0+0 (crc 0 0 0) 0x55e0534e76a0 con 0x7f257c0123a0 2024-04-23T19:11:32.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f245bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953500 cookie 94421958616176 notifier 4758 bl.length()=150 2024-04-23T19:11:32.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f245bfff640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f2448003d60 con 0x7f257c0123a0 2024-04-23T19:11:32.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.144+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 37 ==== osd_op_reply(137 notify.3 [notify cookie 94421959386912 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.145+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 38 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.145+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 39 ==== watch-notify(notify_complete (2) cookie 94421959386912 notify 137438953500 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f25740a6060 con 0x7f257c0123a0 2024-04-23T19:11:32.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.145+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:139 8.f 8:f206e60c:root::.bucket.meta.myfoo7%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053525b10 con 0x55e0534d0170 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(139 .bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 [call,call,delete] v32'2 uv2 ondisk = 0) ==== 292+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f2586f40800 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 from cache 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f2586f40800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10, ofs0, ns] 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:140 7.0 7:05bf5b68:::notify.1:head [notify cookie 94421959386912 in=220b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x55e0534c9f20 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== watch-notify(notify (1) cookie 94421958614048 notify 137438953493 ret 0) ==== 250+0+0 (crc 0 0 0) 0x55e0534e8c80 con 0x55e0534c9f20 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f245b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953493 cookie 94421958614048 notifier 4758 bl.length()=208 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(140 notify.1 [notify cookie 94421959386912 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.146+0000 7f245b7fe640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:141 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f23b80032c0 con 0x55e0534c9f20 2024-04-23T19:11:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(141 notify.1 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== watch-notify(notify_complete (2) cookie 94421959386912 notify 137438953493 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e0534c16c0 con 0x55e0534c9f20 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:142 4.4 4:21037e61:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053515ea0 con 0x7f257409c040 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:143 4.c 4:335aec59:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053514340 con 0x55e0534c9f20 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:144 4.6 4:613d2a51:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053514b90 con 0x55e0534d0170 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:145 4.3 4:c3be44e0:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053514f30 con 0x55e0534c9f20 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:146 4.a 4:51a5c25f:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05351b890 con 0x7f257c0123a0 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:147 4.3f 4:fcc88c01:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05351f190 con 0x7f257c0123a0 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:148 4.20 4:05182548:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05351f530 con 0x55e0534c9f20 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:149 4.36 4:6c6b2818:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053514b90 con 0x55e0534c9f20 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:150 4.1b 4:d93e2e9b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053514340 con 0x7f257c0123a0 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:151 4.1e 4:7a4a77b1:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e05351b890 con 0x7f257409c040 2024-04-23T19:11:32.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.147+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:152 4.3e 4:7c76a7d2:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053514340 con 0x7f257c0123a0 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.148+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 40 ==== osd_op_reply(146 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.4 [delete] v32'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.148+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(144 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.2 [delete] v32'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 41 ==== osd_op_reply(150 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.8 [delete] v32'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 42 ==== osd_op_reply(147 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.5 [delete] v32'12 uv12 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== osd_op_reply(142 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.0 [delete] v32'3 uv3 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(143 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.1 [delete] v32'6 uv6 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(148 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.6 [delete] v32'6 uv6 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(145 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.3 [delete] v32'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(151 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.9 [delete] v32'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.149+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 43 ==== osd_op_reply(152 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.10 [delete] v32'20 uv20 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(149 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10.7 [delete] v32'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f2586f40800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 : miss 2024-04-23T19:11:32.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f2586f40800 20 rados->read ofs=0 len=0 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:153 6.18 6:1b3f8b49:::pubsub..bucket.myfoo7%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x7f257409c040 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(153 pubsub..bucket.myfoo7/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f2586f40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f2586f40800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 info.flags=0x0 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f2586f40800 10 adding default.rgw.log++pubsub..bucket.myfoo7/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 to cache LRU end 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.150+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:154 6.18 6:1b3f8b49:::pubsub..bucket.myfoo7%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x7f257409c040 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.151+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(154 pubsub..bucket.myfoo7/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.10 [delete] v32'28 uv21 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.151+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:155 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=147b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0535201a0 con 0x7f257409c040 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.152+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(155 fud.buckets [call] v32'7 uv7 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.152+0000 7f2586f40800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-23T19:11:32.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.152+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:156 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x7f257409c040 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.153+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(156 p5YnriCv1nAtykxBrupQ [delete] v32'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.153+0000 7f2586f40800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.153+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:157 7.15 7:a93a5511:::notify.2:head [notify cookie 94421959386880 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x55e0534d0170 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.153+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== watch-notify(notify (1) cookie 94421958619024 notify 137438953479 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55e053515450 con 0x55e0534d0170 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.153+0000 7f245b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953479 cookie 94421958619024 notifier 4758 bl.length()=170 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.154+0000 7f245b7fe640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:158 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f23b80032a0 con 0x55e0534d0170 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.154+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(157 notify.2 [notify cookie 94421959386880 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.154+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== osd_op_reply(158 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.154+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== watch-notify(notify_complete (2) cookie 94421959386880 notify 137438953479 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e0534bea10 con 0x55e0534d0170 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.154+0000 7f2586f40800 10 removing email index: 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.154+0000 7f2586f40800 10 removing user buckets index 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.154+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:159 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x7f257409c040 2024-04-23T19:11:32.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.155+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(159 fud.buckets [delete] v32'8 uv8 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.155+0000 7f2586f40800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.155+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94421959386880 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053520a70 con 0x7f257409c040 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== watch-notify(notify (1) cookie 94421958621360 notify 137438953511 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55e0534c16c0 con 0x7f257409c040 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 42 ==== osd_op_reply(160 notify.6 [notify cookie 94421959386880 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f245bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953511 cookie 94421958621360 notifier 4758 bl.length()=160 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f245bfff640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f2448003d60 con 0x7f257409c040 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 43 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 44 ==== watch-notify(notify_complete (2) cookie 94421959386880 notify 137438953511 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e0534e7220 con 0x7f257409c040 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f2586f40800 10 removing user index: fud 2024-04-23T19:11:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.156+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:162 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0535201a0 con 0x7f257409c040 2024-04-23T19:11:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 45 ==== osd_op_reply(162 fud [call,call,delete] v32'7 uv7 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f2586f40800 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-23T19:11:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f2586f40800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-23T19:11:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94421959386880 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55e053515ea0 con 0x7f257409c040 2024-04-23T19:11:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 46 ==== watch-notify(notify (1) cookie 94421958631248 notify 137438953512 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55e0534e7220 con 0x7f257409c040 2024-04-23T19:11:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f245bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953512 cookie 94421958631248 notifier 4758 bl.length()=152 2024-04-23T19:11:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f245bfff640 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f2448003d60 con 0x7f257409c040 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 47 ==== osd_op_reply(163 notify.5 [notify cookie 94421959386880 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 48 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.158+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 49 ==== watch-notify(notify_complete (2) cookie 94421959386880 notify 137438953512 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e0534e7010 con 0x7f257409c040 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.159+0000 7f243afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.159+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94421958633472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534e4cd0 con 0x55e0534c9f20 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.161+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 94421958633472] v32'227 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.161+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94421958609808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534da800 con 0x7f257c0123a0 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.162+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 44 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 94421958609808] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.162+0000 7f2586f40800 20 remove_watcher() i=0 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.162+0000 7f2586f40800 2 removed watcher, disabling cache 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.162+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94421958614048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534da800 con 0x55e0534c9f20 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.163+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 49 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 94421958614048] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.163+0000 7f2586f40800 20 remove_watcher() i=1 2024-04-23T19:11:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.163+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94421958619024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d5c20 con 0x55e0534d0170 2024-04-23T19:11:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.164+0000 7f2585240640 1 -- 172.21.15.63:0/2086995173 <== osd.3 v2:172.21.15.63:6816/1289558434 36 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 94421958619024] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257c029070 con 0x55e0534d0170 2024-04-23T19:11:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.164+0000 7f2586f40800 20 remove_watcher() i=2 2024-04-23T19:11:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.164+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94421958616176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534da800 con 0x7f257c0123a0 2024-04-23T19:11:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.165+0000 7f2584a3f640 1 -- 172.21.15.63:0/2086995173 <== osd.2 v2:172.21.15.63:6808/753864785 45 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 94421958616176] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2574075030 con 0x7f257c0123a0 2024-04-23T19:11:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.165+0000 7f2586f40800 20 remove_watcher() i=3 2024-04-23T19:11:32.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.165+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94421958626848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d6f90 con 0x55e0534c9f20 2024-04-23T19:11:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.166+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.1 v2:172.21.15.63:6800/3216879695 50 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 94421958626848] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x55e0534c9f20 2024-04-23T19:11:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.166+0000 7f2586f40800 20 remove_watcher() i=4 2024-04-23T19:11:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.166+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94421958631248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d5c20 con 0x7f257409c040 2024-04-23T19:11:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.167+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 50 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 94421958631248] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.167+0000 7f2586f40800 20 remove_watcher() i=5 2024-04-23T19:11:32.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.167+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94421958621360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d6470 con 0x7f257409c040 2024-04-23T19:11:32.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.168+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 51 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 94421958621360] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.168+0000 7f2586f40800 20 remove_watcher() i=6 2024-04-23T19:11:32.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.168+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94421958635120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55e0534d6f90 con 0x7f257409c040 2024-04-23T19:11:32.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.169+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 <== osd.0 v2:172.21.15.16:6800/2262294208 52 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 94421958635120] v32'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257802b040 con 0x7f257409c040 2024-04-23T19:11:32.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.169+0000 7f2586f40800 20 remove_watcher() i=7 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f257409c040 msgr2=0x7f257406f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f257409c040 0x7f257406f580 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e0534c9f20 msgr2=0x55e0534cc3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e0534c9f20 0x55e0534cc3e0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f257c0123a0 msgr2=0x7f257c046790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f257c0123a0 0x7f257c046790 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e0534d0170 msgr2=0x55e0534d2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55e0534d0170 0x55e0534d2580 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2450036eb0 msgr2=0x7f2450039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2450036eb0 0x7f2450039370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2574052fd0 tx=0x7f2574087000 comp rx=0 tx=0).stop 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 msgr2=0x55e0534be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534be240 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f257c001df0 tx=0x7f257c04cc00 comp rx=0 tx=0).stop 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f258423e640 1 -- 172.21.15.63:0/2086995173 reap_dead start 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 shutdown_connections 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2086995173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534b6cc0 0x55e0534be240 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 >> 172.21.15.63:0/2086995173 conn(0x55e0534b3020 msgr2=0x55e0534b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 shutdown_connections 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2086995173 wait complete. 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f245c036eb0 msgr2=0x7f245c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f245c036eb0 0x7f245c039370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2574077520 tx=0x7f257408c510 comp rx=0 tx=0).stop 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534a99e0 msgr2=0x55e0534a9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534a99e0 0x55e0534a9e00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55e05348ef20 tx=0x7f257c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 shutdown_connections 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f245c036eb0 0x7f245c039370 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 --2- 172.21.15.63:0/2830626722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0534a99e0 0x55e0534a9e00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 >> 172.21.15.63:0/2830626722 conn(0x55e05349da80 msgr2=0x55e05349de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 shutdown_connections 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.170+0000 7f2586f40800 1 -- 172.21.15.63:0/2830626722 wait complete. 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e053498a20 msgr2=0x55e05349ae10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e053498a20 0x55e05349ae10 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e053396340 msgr2=0x55e053495450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e053396340 0x55e053495450 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f246c036e60 msgr2=0x7f246c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f246c036e60 0x7f246c039320 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2578001e10 tx=0x7f2578001390 comp rx=0 tx=0).stop 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 msgr2=0x55e05348e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e05348e210 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f25740524f0 tx=0x7f2574003020 comp rx=0 tx=0).stop 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 shutdown_connections 2024-04-23T19:11:32.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55e053498a20 0x55e05349ae10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55e053396340 0x55e053495450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f246c036e60 0x7f246c039320 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 --2- 172.21.15.63:0/1184392951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e053395180 0x55e05348e210 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:11:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 >> 172.21.15.63:0/1184392951 conn(0x55e0533907e0 msgr2=0x55e053480900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:11:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 shutdown_connections 2024-04-23T19:11:32.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:11:32.171+0000 7f2586f40800 1 -- 172.21.15.63:0/1184392951 wait complete. 2024-04-23T19:12:12.182 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-04-23T19:12:12.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-04-23T19:12:12.183 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-04-23T19:12:12.235 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:12.236 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:12.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.236+0000 7fbe1bf36800 1 Processor -- start 2024-04-23T19:12:12.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.236+0000 7fbe1bf36800 1 -- start start 2024-04-23T19:12:12.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.236+0000 7fbe1bf36800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55f483e71780 0x55f483e73b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.236+0000 7fbe1bf36800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f483d1f7f0 con 0x55f483e6dc70 2024-04-23T19:12:12.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.236+0000 7fbe1bf36800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55f483d0ee20 con 0x55f483e71780 2024-04-23T19:12:12.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.237+0000 7fbe174cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55f483e71780 0x55f483e73b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.237+0000 7fbe17ccc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55f483e6dc70 0x55f483e70060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56606/0 (socket says 172.21.15.63:56606) 2024-04-23T19:12:12.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.237+0000 7fbe17ccc640 1 -- 172.21.15.63:0/575988019 learned_addr learned my addr 172.21.15.63:0/575988019 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:12.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.237+0000 7fbe174cb640 1 -- 172.21.15.63:0/575988019 >> v1:172.21.15.16:6789/0 conn(0x55f483e6dc70 legacy=0x55f483e70060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.237+0000 7fbe174cb640 1 -- 172.21.15.63:0/575988019 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483d2db50 con 0x55f483e71780 2024-04-23T19:12:12.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.239+0000 7fbe174cb640 1 --2- 172.21.15.63:0/575988019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e71780 0x55f483e73b70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00a470 tx=0x7fbe0c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733f080a11572f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.239+0000 7fbe16cca640 1 -- 172.21.15.63:0/575988019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe0c057070 con 0x55f483e71780 2024-04-23T19:12:12.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.239+0000 7fbe16cca640 1 -- 172.21.15.63:0/575988019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0c052570 con 0x55f483e71780 2024-04-23T19:12:12.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.239+0000 7fbe16cca640 1 -- 172.21.15.63:0/575988019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe0c052870 con 0x55f483e71780 2024-04-23T19:12:12.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.240+0000 7fbe1bf36800 1 -- 172.21.15.63:0/575988019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e71780 msgr2=0x55f483e73b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.240+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/575988019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e71780 0x55f483e73b70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00a470 tx=0x7fbe0c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:12.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.240+0000 7fbe1bf36800 1 -- 172.21.15.63:0/575988019 shutdown_connections 2024-04-23T19:12:12.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.240+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/575988019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e71780 0x55f483e73b70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.240+0000 7fbe1bf36800 1 -- 172.21.15.63:0/575988019 >> 172.21.15.63:0/575988019 conn(0x55f483e67600 msgr2=0x55f483e69a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.240+0000 7fbe1bf36800 1 -- 172.21.15.63:0/575988019 shutdown_connections 2024-04-23T19:12:12.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.240+0000 7fbe1bf36800 1 -- 172.21.15.63:0/575988019 wait complete. 2024-04-23T19:12:12.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.255+0000 7fbe1bf36800 1 Processor -- start 2024-04-23T19:12:12.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.255+0000 7fbe1bf36800 1 -- start start 2024-04-23T19:12:12.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.255+0000 7fbe1bf36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f5e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.255+0000 7fbe1bf36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f483db86f0 con 0x55f483e6dc70 2024-04-23T19:12:12.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.255+0000 7fbe17ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f5e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.255+0000 7fbe17ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f5e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39896/0 (socket says 172.21.15.63:39896) 2024-04-23T19:12:12.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.255+0000 7fbe17ccc640 1 -- 172.21.15.63:0/2883843100 learned_addr learned my addr 172.21.15.63:0/2883843100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.256+0000 7fbe17ccc640 1 -- 172.21.15.63:0/2883843100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483f5e960 con 0x55f483e6dc70 2024-04-23T19:12:12.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.256+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/2883843100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f5e390 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbe08001de0 tx=0x7fbe0800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f73e29c847dfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.256+0000 7fbd05e0b640 1 -- 172.21.15.63:0/2883843100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe08012020 con 0x55f483e6dc70 2024-04-23T19:12:12.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.256+0000 7fbd05e0b640 1 -- 172.21.15.63:0/2883843100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0800e5a0 con 0x55f483e6dc70 2024-04-23T19:12:12.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.256+0000 7fbd05e0b640 1 -- 172.21.15.63:0/2883843100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe0800e8c0 con 0x55f483e6dc70 2024-04-23T19:12:12.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 -- 172.21.15.63:0/2883843100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 msgr2=0x55f483f5e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/2883843100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f5e390 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbe08001de0 tx=0x7fbe0800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 -- 172.21.15.63:0/2883843100 shutdown_connections 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/2883843100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f5e390 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 -- 172.21.15.63:0/2883843100 >> 172.21.15.63:0/2883843100 conn(0x55f483e67600 msgr2=0x55f483f59cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 -- 172.21.15.63:0/2883843100 shutdown_connections 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 -- 172.21.15.63:0/2883843100 wait complete. 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 Processor -- start 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.257+0000 7fbe1bf36800 1 -- start start 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbe1bf36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbe1bf36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f483e76360 con 0x55f483e6dc70 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbe17ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbe17ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f65ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39910/0 (socket says 172.21.15.63:39910) 2024-04-23T19:12:12.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbe17ccc640 1 -- 172.21.15.63:0/1483011083 learned_addr learned my addr 172.21.15.63:0/1483011083 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbe17ccc640 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483f66420 con 0x55f483e6dc70 2024-04-23T19:12:12.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f65ee0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c010 tx=0x7fbe08000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.258+0000 7fbcfffff640 1 -- 172.21.15.63:0/1483011083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe08012070 con 0x55f483e6dc70 2024-04-23T19:12:12.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.259+0000 7fbcfffff640 1 -- 172.21.15.63:0/1483011083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0801a7a0 con 0x55f483e6dc70 2024-04-23T19:12:12.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.259+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f483f666b0 con 0x55f483e6dc70 2024-04-23T19:12:12.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.259+0000 7fbcfffff640 1 -- 172.21.15.63:0/1483011083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe0801aa80 con 0x55f483e6dc70 2024-04-23T19:12:12.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.259+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f483f69360 con 0x55f483e6dc70 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.259+0000 7fbcfffff640 1 -- 172.21.15.63:0/1483011083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbe08015020 con 0x55f483e6dc70 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbcfffff640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbd00036eb0 0x7fbd00039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbcfffff640 1 -- 172.21.15.63:0/1483011083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fbe08020070 con 0x55f483e6dc70 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbe174cb640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbd00036eb0 0x7fbd00039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483e71780 0x55f483f6d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483f6d6a0 con 0x55f483e71780 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbe184cd640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483e71780 0x55f483f6d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbe174cb640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbd00036eb0 0x7fbd00039370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c054000 tx=0x7fbe0c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.260+0000 7fbe184cd640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483e71780 0x55f483f6d120 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.261+0000 7fbe184cd640 1 -- 172.21.15.63:0/1483011083 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe1000a4a0 con 0x55f483e71780 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.261+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483f6a6f0 con 0x55f483e71780 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.261+0000 7fbe184cd640 1 -- 172.21.15.63:0/1483011083 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbe1000a4a0 con 0x55f483e71780 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.261+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483f6f4f0 con 0x55f483e71780 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.261+0000 7fbe184cd640 1 -- 172.21.15.63:0/1483011083 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbe1000a4a0 con 0x55f483e71780 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.261+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483f6e830 con 0x55f483e71780 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.262+0000 7fbe184cd640 1 -- 172.21.15.63:0/1483011083 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbe1000a4a0 con 0x55f483e71780 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.262+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f483f708a0 0x55f483f72c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.262+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483f73210 con 0x55f483f708a0 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.262+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f483f708a0 0x55f483f72c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.263+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f483f708a0 0x55f483f72c90 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.263+0000 7fbe17ccc640 1 -- 172.21.15.63:0/1483011083 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbe0802b070 con 0x55f483f708a0 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.263+0000 7fbe1bf36800 1 Processor -- start 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.263+0000 7fbe1bf36800 1 -- start start 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.263+0000 7fbe1bf36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f81610 0x55f483f81a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.263+0000 7fbe1bf36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f483d5ee30 con 0x55f483f81610 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.264+0000 7fbe184cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f81610 0x55f483f81a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.264+0000 7fbe184cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f81610 0x55f483f81a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39914/0 (socket says 172.21.15.63:39914) 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.264+0000 7fbe184cd640 1 -- 172.21.15.63:0/307600161 learned_addr learned my addr 172.21.15.63:0/307600161 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.264+0000 7fbe184cd640 1 -- 172.21.15.63:0/307600161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483f81f70 con 0x55f483f81610 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.264+0000 7fbe184cd640 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f81610 0x55f483f81a30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55f483f6da60 tx=0x7fbe100178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.264+0000 7fbcfd7fa640 1 -- 172.21.15.63:0/307600161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe10013440 con 0x55f483f81610 2024-04-23T19:12:12.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.265+0000 7fbcfd7fa640 1 -- 172.21.15.63:0/307600161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe100135e0 con 0x55f483f81610 2024-04-23T19:12:12.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.265+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f483f829e0 con 0x55f483f81610 2024-04-23T19:12:12.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.265+0000 7fbcfd7fa640 1 -- 172.21.15.63:0/307600161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe100138c0 con 0x55f483f81610 2024-04-23T19:12:12.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.265+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f483f82ee0 con 0x55f483f81610 2024-04-23T19:12:12.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.265+0000 7fbcfd7fa640 1 -- 172.21.15.63:0/307600161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbe10011040 con 0x55f483f81610 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbcfd7fa640 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbcf0036eb0 0x7fbcf0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbcfd7fa640 1 -- 172.21.15.63:0/307600161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fbe1001d070 con 0x55f483f81610 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbcf0036eb0 0x7fbcf0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbe1bf36800 1 Processor -- start 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbe1bf36800 1 -- start start 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbcf0036eb0 0x7fbcf0039370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbe0804b0b0 tx=0x7fbe0804af80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbe1bf36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483f8ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.266+0000 7fbe1bf36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f483f5ece0 con 0x55f483f8e940 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbe184cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483f8ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbe184cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483f8ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39924/0 (socket says 172.21.15.63:39924) 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbe184cd640 1 -- 172.21.15.63:0/166503848 learned_addr learned my addr 172.21.15.63:0/166503848 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbe184cd640 1 -- 172.21.15.63:0/166503848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483f8f330 con 0x55f483f8e940 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbe184cd640 1 --2- 172.21.15.63:0/166503848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483f8ed60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbe1004cc30 tx=0x7fbe100129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8db0da0e74f97e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbcf6ffd640 1 -- 172.21.15.63:0/166503848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe100123a0 con 0x55f483f8e940 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbcf6ffd640 1 -- 172.21.15.63:0/166503848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1004a020 con 0x55f483f8e940 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.267+0000 7fbcf6ffd640 1 -- 172.21.15.63:0/166503848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe10001310 con 0x55f483f8e940 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 -- 172.21.15.63:0/166503848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 msgr2=0x55f483f8ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/166503848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483f8ed60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbe1004cc30 tx=0x7fbe100129d0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 -- 172.21.15.63:0/166503848 shutdown_connections 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/166503848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483f8ed60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 -- 172.21.15.63:0/166503848 >> 172.21.15.63:0/166503848 conn(0x55f483f8ac10 msgr2=0x55f483f88c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 -- 172.21.15.63:0/166503848 shutdown_connections 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 -- 172.21.15.63:0/166503848 wait complete. 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 Processor -- start 2024-04-23T19:12:12.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 -- start start 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483e6d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.268+0000 7fbe1bf36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f483f8fac0 con 0x55f483f8e940 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.269+0000 7fbe184cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483e6d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.269+0000 7fbe184cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483e6d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39936/0 (socket says 172.21.15.63:39936) 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.269+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 learned_addr learned my addr 172.21.15.63:0/817489056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.269+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483e6da10 con 0x55f483f8e940 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.269+0000 7fbe184cd640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483e6d4d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbe1004bb50 tx=0x7fbe1004cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.269+0000 7fbcf57fa640 1 -- 172.21.15.63:0/817489056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe1002aa60 con 0x55f483f8e940 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.270+0000 7fbcf57fa640 1 -- 172.21.15.63:0/817489056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1002ac00 con 0x55f483f8e940 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.270+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f483e6bbc0 con 0x55f483f8e940 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.270+0000 7fbcf57fa640 1 -- 172.21.15.63:0/817489056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe1002a350 con 0x55f483f8e940 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.270+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f483e6c0c0 con 0x55f483f8e940 2024-04-23T19:12:12.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbcf57fa640 1 -- 172.21.15.63:0/817489056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbe10045040 con 0x55f483f8e940 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbcf57fa640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbce0036eb0 0x7fbce0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbcf57fa640 1 -- 172.21.15.63:0/817489056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fbe10043020 con 0x55f483f8e940 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbce0036eb0 0x7fbce0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbce0036eb0 0x7fbce0039370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800bc10 tx=0x7fbe08048000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483fa3d70 0x55f483fa6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.271+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fa6980 con 0x55f483fa3d70 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe174cb640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483fa3d70 0x55f483fa6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe174cb640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483fa3d70 0x55f483fa6230 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 20 realm 2024-04-23T19:12:12.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fa6d50 con 0x55f483fa3d70 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.272+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fa7620 con 0x55f483fa3d70 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fa79c0 con 0x55f483fa3d70 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fa8210 con 0x55f483fa3d70 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 20 searching for the correct realm 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f483fa9f80 0x55f483fac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.273+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.274+0000 7fbe184cd640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f483fa9f80 0x55f483fac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.274+0000 7fbe184cd640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f483fa9f80 0x55f483fac370 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.274+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.274+0000 7fbe184cd640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbe10002380 0x7fbe10002ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.274+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe10002380 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.274+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbe10002380 0x7fbe10002ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.275+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbe10002380 0x7fbe10002ad0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.275+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.275+0000 7fbe17ccc640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbe08017050 0x7fbe08019000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.275+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.275+0000 7fbe174cb640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbe08017050 0x7fbe08019000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.276+0000 7fbe174cb640 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbe08017050 0x7fbe08019000 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.276+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.276+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.276+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.276+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.277+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.277+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.277+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.277+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.277+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.277+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe10002380 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.278+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.278+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.278+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.278+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.278+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.278+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.278+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.279+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.279+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.279+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe10002380 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.279+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.279+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.279+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.279+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.280+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.280+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.280+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.280+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.281+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.281+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.281+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.281+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.281+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.281+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.281+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.282+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.282+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.282+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.282+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.282+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe10002380 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.282+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.282+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.283+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.283+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe10002380 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.283+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.283+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.283+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.283+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.284+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.284+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe10002380 2024-04-23T19:12:12.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.284+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.284+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa9f80 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.284+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.284+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe08017050 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x55f483fa3d70 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f483fac900 con 0x7fbe10002380 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:12.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.285+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fa8b90 con 0x55f483fa3d70 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fa9430 con 0x55f483fa3d70 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483facd80 con 0x7fbe08017050 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 zone default found 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 4 Realm: () 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 zonegroup default 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.286+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fad230 con 0x7fbe10002380 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483faefb0 con 0x55f483fa9f80 2024-04-23T19:12:12.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483faf470 con 0x55f483fa9f80 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 20 started sync module instance, tier type = 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.287+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fae450 con 0x7fbe10002380 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.288+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.288+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94508674639936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fafd80 con 0x7fbe10002380 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.288+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb05d0 con 0x55f483fa3d70 2024-04-23T19:12:12.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.290+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94508674639936] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.290+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.290+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94508674644176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb0690 con 0x55f483fa3d70 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.290+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb11b0 con 0x55f483fa9f80 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.291+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94508674644176] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.291+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.292+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94508674649216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb15a0 con 0x55f483fa9f80 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.292+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb1fe0 con 0x7fbe10002380 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.293+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94508674649216] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.293+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.293+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94508674652848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb23d0 con 0x7fbe10002380 2024-04-23T19:12:12.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.293+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb2e10 con 0x55f483fa3d70 2024-04-23T19:12:12.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.295+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.295+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94508674656480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb2dd0 con 0x55f483fa3d70 2024-04-23T19:12:12.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.296+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb3df0 con 0x7fbe08017050 2024-04-23T19:12:12.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.296+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94508674652848] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.297+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94508674656480] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.297+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.297+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94508674660608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb3df0 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.297+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb4260 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.299+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94508674660608] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.299+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.299+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94508674646304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb3df0 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.299+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb4cf0 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94508674646304] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94508674662944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb4cf0 con 0x7fbe08017050 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 20 add_watcher() i=0 2024-04-23T19:12:12.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 20 add_watcher() i=1 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 20 add_watcher() i=2 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 20 add_watcher() i=3 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 20 add_watcher() i=4 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 20 add_watcher() i=5 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.300+0000 7fbe1bf36800 20 add_watcher() i=6 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.301+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94508674662944] v32'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.301+0000 7fbe1bf36800 20 add_watcher() i=7 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.301+0000 7fbe1bf36800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.301+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb7ff0 con 0x55f483fa3d70 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.302+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.302+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94508674664880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483fb5280 con 0x55f483fa3d70 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94508674664880] v32'228 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbccdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbe1bf36800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbe1bf36800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbe1bf36800 5 note: GC not initialized 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbcccfe1640 20 reqs_thread_entry: start 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbe1bf36800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbe1bf36800 20 Filter name: none 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.303+0000 7fbc467fc640 20 reqs_thread_entry: start 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f483feb4b0 con 0x55f483fa3d70 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:12:12.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:12:12.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:12:12.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:12.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.304+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x7fbe08017050 2024-04-23T19:12:12.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.305+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 usage.10 [call] v32'32 uv32 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.305+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x7fbe08017050 2024-04-23T19:12:12.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.306+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(64 usage.10 [call] v32'33 uv32 ondisk = -61 ((61) No data available)) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.307+0000 7fbccdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.307+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94508674664880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x55f483fa3d70 2024-04-23T19:12:12.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.308+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94508674664880] v32'229 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.308+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94508674639936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x7fbe10002380 2024-04-23T19:12:12.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.309+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94508674639936] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.309+0000 7fbe1bf36800 20 remove_watcher() i=0 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.309+0000 7fbe1bf36800 2 removed watcher, disabling cache 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.309+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94508674644176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x55f483fa3d70 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.310+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94508674644176] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.311+0000 7fbe1bf36800 20 remove_watcher() i=1 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.311+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94508674649216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x55f483fa9f80 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.312+0000 7fbe184cd640 1 -- 172.21.15.63:0/817489056 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94508674649216] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe1004cd70 con 0x55f483fa9f80 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.312+0000 7fbe1bf36800 20 remove_watcher() i=2 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.312+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94508674652848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x7fbe10002380 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.313+0000 7fbe17ccc640 1 -- 172.21.15.63:0/817489056 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94508674652848] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0802b070 con 0x7fbe10002380 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.313+0000 7fbe1bf36800 20 remove_watcher() i=3 2024-04-23T19:12:12.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.313+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94508674656480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x55f483fa3d70 2024-04-23T19:12:12.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.314+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94508674656480] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x55f483fa3d70 2024-04-23T19:12:12.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.314+0000 7fbe1bf36800 20 remove_watcher() i=4 2024-04-23T19:12:12.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.314+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94508674660608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x7fbe08017050 2024-04-23T19:12:12.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.315+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94508674660608] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.315+0000 7fbe1bf36800 20 remove_watcher() i=5 2024-04-23T19:12:12.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.315+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94508674646304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x7fbe08017050 2024-04-23T19:12:12.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.316+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94508674646304] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.316+0000 7fbe1bf36800 20 remove_watcher() i=6 2024-04-23T19:12:12.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.316+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94508674662944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f483feaf90 con 0x7fbe08017050 2024-04-23T19:12:12.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.317+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94508674662944] v32'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe0c05a040 con 0x7fbe08017050 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.317+0000 7fbe1bf36800 20 remove_watcher() i=7 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbe08017050 msgr2=0x7fbe08019000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fbe08017050 0x7fbe08019000 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483fa3d70 msgr2=0x55f483fa6230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483fa3d70 0x55f483fa6230 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbe10002380 msgr2=0x7fbe10002ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fbe10002380 0x7fbe10002ad0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f483fa9f80 msgr2=0x55f483fac370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f483fa9f80 0x55f483fac370 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbce0036eb0 msgr2=0x7fbce0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbce0036eb0 0x7fbce0039370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800bc10 tx=0x7fbe08048000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 msgr2=0x55f483e6d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/817489056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f8e940 0x55f483e6d4d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbe1004bb50 tx=0x7fbe1004cbc0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 reap_dead start 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe174cb640 1 -- 172.21.15.63:0/817489056 reap_dead start 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 shutdown_connections 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 >> 172.21.15.63:0/817489056 conn(0x55f483f8ac10 msgr2=0x55f483f88270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 shutdown_connections 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/817489056 wait complete. 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbcf0036eb0 msgr2=0x7fbcf0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbcf0036eb0 0x7fbcf0039370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbe0804b0b0 tx=0x7fbe0804af80 comp rx=0 tx=0).stop 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f81610 msgr2=0x55f483f81a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f81610 0x55f483f81a30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55f483f6da60 tx=0x7fbe100178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 shutdown_connections 2024-04-23T19:12:12.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbcf0036eb0 0x7fbcf0039370 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/307600161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483f81610 0x55f483f81a30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 >> 172.21.15.63:0/307600161 conn(0x55f483f756b0 msgr2=0x55f483f75ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 shutdown_connections 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.318+0000 7fbe1bf36800 1 -- 172.21.15.63:0/307600161 wait complete. 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f483f708a0 msgr2=0x55f483f72c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f483f708a0 0x55f483f72c90 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483e71780 msgr2=0x55f483f6d120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483e71780 0x55f483f6d120 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbd00036eb0 msgr2=0x7fbd00039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbd00036eb0 0x7fbd00039370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c054000 tx=0x7fbe0c04e000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 msgr2=0x55f483f65ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f65ee0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c010 tx=0x7fbe08000ed0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 shutdown_connections 2024-04-23T19:12:12.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f483f708a0 0x55f483f72c90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f483e71780 0x55f483f6d120 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbd00036eb0 0x7fbd00039370 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 --2- 172.21.15.63:0/1483011083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f483e6dc70 0x55f483f65ee0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 >> 172.21.15.63:0/1483011083 conn(0x55f483e67600 msgr2=0x55f483f584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 shutdown_connections 2024-04-23T19:12:12.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.319+0000 7fbe1bf36800 1 -- 172.21.15.63:0/1483011083 wait complete. 2024-04-23T19:12:12.328 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-23T19:12:12.328 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-23T19:12:12.328 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-23T19:12:12.373 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:12.373 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:12.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.373+0000 7ff248062800 1 Processor -- start 2024-04-23T19:12:12.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.373+0000 7ff248062800 1 -- start start 2024-04-23T19:12:12.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.373+0000 7ff248062800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563f0ec4b750 0x563f0ec4bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.373+0000 7ff248062800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563f0eafc7f0 con 0x563f0ec4c0b0 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.373+0000 7ff248062800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x563f0eaebe20 con 0x563f0ec4b750 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.374+0000 7ff245b61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563f0ec4b750 0x563f0ec4bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.374+0000 7ff245b61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563f0ec4b750 0x563f0ec4bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39948/0 (socket says 172.21.15.63:39948) 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.374+0000 7ff245b61640 1 -- 172.21.15.63:0/127587614 learned_addr learned my addr 172.21.15.63:0/127587614 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.374+0000 7ff245360640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x563f0ec4c0b0 0x563f0ec481b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56608/0 (socket says 172.21.15.63:56608) 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.374+0000 7ff245b61640 1 -- 172.21.15.63:0/127587614 >> v1:172.21.15.16:6789/0 conn(0x563f0ec4c0b0 legacy=0x563f0ec481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.374+0000 7ff245b61640 1 -- 172.21.15.63:0/127587614 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0eb0ab50 con 0x563f0ec4b750 2024-04-23T19:12:12.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff245b61640 1 --2- 172.21.15.63:0/127587614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4bb70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff23800a520 tx=0x7ff23801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06e0e96d14b4568 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff244b5f640 1 -- 172.21.15.63:0/127587614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238057070 con 0x563f0ec4b750 2024-04-23T19:12:12.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff244b5f640 1 -- 172.21.15.63:0/127587614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff238052570 con 0x563f0ec4b750 2024-04-23T19:12:12.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff244b5f640 1 -- 172.21.15.63:0/127587614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff23804ae60 con 0x563f0ec4b750 2024-04-23T19:12:12.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff248062800 1 -- 172.21.15.63:0/127587614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 msgr2=0x563f0ec4bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff248062800 1 --2- 172.21.15.63:0/127587614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4bb70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff23800a520 tx=0x7ff23801d010 comp rx=0 tx=0).stop 2024-04-23T19:12:12.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff248062800 1 -- 172.21.15.63:0/127587614 shutdown_connections 2024-04-23T19:12:12.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff248062800 1 --2- 172.21.15.63:0/127587614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4bb70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff248062800 1 -- 172.21.15.63:0/127587614 >> 172.21.15.63:0/127587614 conn(0x563f0ec43db0 msgr2=0x563f0ec461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.376+0000 7ff248062800 1 -- 172.21.15.63:0/127587614 shutdown_connections 2024-04-23T19:12:12.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.377+0000 7ff248062800 1 -- 172.21.15.63:0/127587614 wait complete. 2024-04-23T19:12:12.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.389+0000 7ff248062800 1 Processor -- start 2024-04-23T19:12:12.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.389+0000 7ff248062800 1 -- start start 2024-04-23T19:12:12.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.390+0000 7ff248062800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ed3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.390+0000 7ff248062800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563f0eb956f0 con 0x563f0ec4b750 2024-04-23T19:12:12.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.390+0000 7ff245b61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ed3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.390+0000 7ff245b61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ed3ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39962/0 (socket says 172.21.15.63:39962) 2024-04-23T19:12:12.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.390+0000 7ff245b61640 1 -- 172.21.15.63:0/2815604108 learned_addr learned my addr 172.21.15.63:0/2815604108 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.390+0000 7ff245b61640 1 -- 172.21.15.63:0/2815604108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0ed3c020 con 0x563f0ec4b750 2024-04-23T19:12:12.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff245b61640 1 --2- 172.21.15.63:0/2815604108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ed3ba50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff238055040 tx=0x7ff23804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bca28cedb7ee71 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff133e07640 1 -- 172.21.15.63:0/2815604108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238057040 con 0x563f0ec4b750 2024-04-23T19:12:12.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff133e07640 1 -- 172.21.15.63:0/2815604108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff238052720 con 0x563f0ec4b750 2024-04-23T19:12:12.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff133e07640 1 -- 172.21.15.63:0/2815604108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238052a20 con 0x563f0ec4b750 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff248062800 1 -- 172.21.15.63:0/2815604108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 msgr2=0x563f0ed3ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff248062800 1 --2- 172.21.15.63:0/2815604108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ed3ba50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff238055040 tx=0x7ff23804c000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff248062800 1 -- 172.21.15.63:0/2815604108 shutdown_connections 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff248062800 1 --2- 172.21.15.63:0/2815604108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ed3ba50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.391+0000 7ff248062800 1 -- 172.21.15.63:0/2815604108 >> 172.21.15.63:0/2815604108 conn(0x563f0ec43db0 msgr2=0x563f0ed37370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff248062800 1 -- 172.21.15.63:0/2815604108 shutdown_connections 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff248062800 1 -- 172.21.15.63:0/2815604108 wait complete. 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff248062800 1 Processor -- start 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff248062800 1 -- start start 2024-04-23T19:12:12.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff248062800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff248062800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563f0ec53a10 con 0x563f0ec4b750 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff245b61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff245b61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39964/0 (socket says 172.21.15.63:39964) 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.392+0000 7ff245b61640 1 -- 172.21.15.63:0/3576348827 learned_addr learned my addr 172.21.15.63:0/3576348827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.393+0000 7ff245b61640 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0ec494e0 con 0x563f0ec4b750 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.393+0000 7ff245b61640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4adf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff2380524f0 tx=0x7ff238003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.393+0000 7ff132604640 1 -- 172.21.15.63:0/3576348827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238073070 con 0x563f0ec4b750 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.393+0000 7ff132604640 1 -- 172.21.15.63:0/3576348827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff23805d440 con 0x563f0ec4b750 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.393+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563f0ec49770 con 0x563f0ec4b750 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.393+0000 7ff132604640 1 -- 172.21.15.63:0/3576348827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238057070 con 0x563f0ec4b750 2024-04-23T19:12:12.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.393+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0ec49c90 con 0x563f0ec4b750 2024-04-23T19:12:12.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.394+0000 7ff132604640 1 -- 172.21.15.63:0/3576348827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff238076070 con 0x563f0ec4b750 2024-04-23T19:12:12.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.394+0000 7ff132604640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff12c036e60 0x7ff12c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.394+0000 7ff132604640 1 -- 172.21.15.63:0/3576348827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7ff23808e440 con 0x563f0ec4b750 2024-04-23T19:12:12.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.394+0000 7ff245360640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff12c036e60 0x7ff12c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.394+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ec4c0b0 0x563f0ed4c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.394+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed4ceb0 con 0x563f0ec4c0b0 2024-04-23T19:12:12.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.394+0000 7ff246362640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ec4c0b0 0x563f0ed4c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.395+0000 7ff245360640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff12c036e60 0x7ff12c039320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff23c000f60 tx=0x7ff23c001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.395+0000 7ff246362640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ec4c0b0 0x563f0ed4c890 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.395+0000 7ff246362640 1 -- 172.21.15.63:0/3576348827 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff24000a4a0 con 0x563f0ec4c0b0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.395+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed4d750 con 0x563f0ec4c0b0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.395+0000 7ff246362640 1 -- 172.21.15.63:0/3576348827 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff24000a4a0 con 0x563f0ec4c0b0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.395+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed4eb00 con 0x563f0ec4c0b0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.396+0000 7ff246362640 1 -- 172.21.15.63:0/3576348827 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff24000a4a0 con 0x563f0ec4c0b0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.396+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed4dfa0 con 0x563f0ec4c0b0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.396+0000 7ff246362640 1 -- 172.21.15.63:0/3576348827 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff24000a4a0 con 0x563f0ec4c0b0 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.396+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563f0ed4ff90 0x563f0ed52380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.396+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed52af0 con 0x563f0ed4ff90 2024-04-23T19:12:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.396+0000 7ff245b61640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563f0ed4ff90 0x563f0ed52380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.397+0000 7ff245b61640 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563f0ed4ff90 0x563f0ed52380 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.397+0000 7ff245b61640 1 -- 172.21.15.63:0/3576348827 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff238075030 con 0x563f0ed4ff90 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.397+0000 7ff248062800 1 Processor -- start 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.397+0000 7ff248062800 1 -- start start 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.397+0000 7ff248062800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed60f70 0x563f0ed61390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.397+0000 7ff248062800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563f0ec4b330 con 0x563f0ed60f70 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff246362640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed60f70 0x563f0ed61390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff246362640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed60f70 0x563f0ed61390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39966/0 (socket says 172.21.15.63:39966) 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff246362640 1 -- 172.21.15.63:0/2568290597 learned_addr learned my addr 172.21.15.63:0/2568290597 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff246362640 1 -- 172.21.15.63:0/2568290597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0ed618d0 con 0x563f0ed60f70 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff246362640 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed60f70 0x563f0ed61390 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x563f0ed4d270 tx=0x7ff2400178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff12b7fe640 1 -- 172.21.15.63:0/2568290597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff240013440 con 0x563f0ed60f70 2024-04-23T19:12:12.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff12b7fe640 1 -- 172.21.15.63:0/2568290597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2400135e0 con 0x563f0ed60f70 2024-04-23T19:12:12.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563f0ed62340 con 0x563f0ed60f70 2024-04-23T19:12:12.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff12b7fe640 1 -- 172.21.15.63:0/2568290597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2400138c0 con 0x563f0ed60f70 2024-04-23T19:12:12.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.398+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0ed62840 con 0x563f0ed60f70 2024-04-23T19:12:12.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.399+0000 7ff12b7fe640 1 -- 172.21.15.63:0/2568290597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff240011040 con 0x563f0ed60f70 2024-04-23T19:12:12.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.399+0000 7ff12b7fe640 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff120036e60 0x7ff120039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.399+0000 7ff12b7fe640 1 -- 172.21.15.63:0/2568290597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7ff24001d070 con 0x563f0ed60f70 2024-04-23T19:12:12.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.399+0000 7ff245b61640 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff120036e60 0x7ff120039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff248062800 1 Processor -- start 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff248062800 1 -- start start 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff245b61640 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff120036e60 0x7ff120039320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x563f0ed4e340 tx=0x7ff23805d920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff248062800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed6e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff248062800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563f0eb3be30 con 0x563f0ed6e300 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff246362640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed6e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff246362640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed6e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39972/0 (socket says 172.21.15.63:39972) 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff246362640 1 -- 172.21.15.63:0/3752560147 learned_addr learned my addr 172.21.15.63:0/3752560147 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.400+0000 7ff246362640 1 -- 172.21.15.63:0/3752560147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0ed6ecf0 con 0x563f0ed6e300 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff246362640 1 --2- 172.21.15.63:0/3752560147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed6e720 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff2400129a0 tx=0x7ff240012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52db4c1dab58749d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff1297fa640 1 -- 172.21.15.63:0/3752560147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff24002ad80 con 0x563f0ed6e300 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff1297fa640 1 -- 172.21.15.63:0/3752560147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2400014f0 con 0x563f0ed6e300 2024-04-23T19:12:12.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff1297fa640 1 -- 172.21.15.63:0/3752560147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2400017d0 con 0x563f0ed6e300 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff248062800 1 -- 172.21.15.63:0/3752560147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 msgr2=0x563f0ed6e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff248062800 1 --2- 172.21.15.63:0/3752560147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed6e720 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff2400129a0 tx=0x7ff240012a90 comp rx=0 tx=0).stop 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff248062800 1 -- 172.21.15.63:0/3752560147 shutdown_connections 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff248062800 1 --2- 172.21.15.63:0/3752560147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed6e720 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff248062800 1 -- 172.21.15.63:0/3752560147 >> 172.21.15.63:0/3752560147 conn(0x563f0ed6a5d0 msgr2=0x563f0ed68630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff248062800 1 -- 172.21.15.63:0/3752560147 shutdown_connections 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.401+0000 7ff248062800 1 -- 172.21.15.63:0/3752560147 wait complete. 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff248062800 1 Processor -- start 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff248062800 1 -- start start 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff248062800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed755f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff248062800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563f0ed6f2c0 con 0x563f0ed6e300 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff246362640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff246362640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed755f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:39974/0 (socket says 172.21.15.63:39974) 2024-04-23T19:12:12.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 learned_addr learned my addr 172.21.15.63:0/2611163666 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.402+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0ed75b30 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.403+0000 7ff246362640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed755f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff2400129a0 tx=0x7ff24004cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.403+0000 7ff1177fe640 1 -- 172.21.15.63:0/2611163666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2400124d0 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.403+0000 7ff1177fe640 1 -- 172.21.15.63:0/2611163666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff240012670 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.403+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563f0ed75e20 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.403+0000 7ff1177fe640 1 -- 172.21.15.63:0/2611163666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff24004e110 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.403+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0ed78a70 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.403+0000 7ff1177fe640 1 -- 172.21.15.63:0/2611163666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7ff240041070 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff1177fe640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff110036e60 0x7ff110039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff1177fe640 1 -- 172.21.15.63:0/2611163666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7ff24000e040 con 0x563f0ed6e300 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff245b61640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff110036e60 0x7ff110039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ed81390 0x563f0ed83850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed83fa0 con 0x563f0ed81390 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff245360640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ed81390 0x563f0ed83850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff245b61640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff110036e60 0x7ff110039320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff238077520 tx=0x7ff238089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.404+0000 7ff245360640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ed81390 0x563f0ed83850 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 realm 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed80860 con 0x563f0ed81390 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed84aa0 con 0x563f0ed81390 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed85370 con 0x563f0ed81390 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.405+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed85710 con 0x563f0ed81390 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff248062800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff248062800 20 searching for the correct realm 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563f0ed875f0 0x563f0ed899e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff246362640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563f0ed875f0 0x563f0ed899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff246362640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563f0ed875f0 0x563f0ed899e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff246362640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff24003b2a0 0x7ff24004eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.406+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff24003b2a0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.407+0000 7ff245b61640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff24003b2a0 0x7ff24004eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.407+0000 7ff245b61640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff24003b2a0 0x7ff24004eaa0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.407+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.407+0000 7ff245b61640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff23809c040 0x7ff23805c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.407+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.407+0000 7ff245360640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff23809c040 0x7ff23805c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.408+0000 7ff245360640 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff23809c040 0x7ff23805c720 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.408+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.408+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.408+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.408+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.408+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.408+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.409+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.409+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.409+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.409+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff24003b2a0 2024-04-23T19:12:12.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.409+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.409+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.410+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff24003b2a0 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.411+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.412+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff24003b2a0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.413+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff24003b2a0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff24003b2a0 2024-04-23T19:12:12.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.414+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed875f0 2024-04-23T19:12:12.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff23809c040 2024-04-23T19:12:12.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x563f0ed81390 2024-04-23T19:12:12.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a090 con 0x7ff24003b2a0 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff248062800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff248062800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff248062800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff248062800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff248062800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.415+0000 7ff248062800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed860c0 con 0x563f0ed81390 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed86960 con 0x563f0ed81390 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed85d00 con 0x7ff23809c040 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 zone default found 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 4 Realm: () 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 zonegroup default 2024-04-23T19:12:12.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.416+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8a4f0 con 0x7ff24003b2a0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8c720 con 0x563f0ed875f0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8cac0 con 0x563f0ed875f0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 20 started sync module instance, tier type = 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.417+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8bbb0 con 0x7ff24003b2a0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.418+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.418+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94828832014288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8b810 con 0x7ff24003b2a0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.418+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8d840 con 0x563f0ed81390 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.419+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94828832014288] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.420+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.420+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94828832018016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8dc30 con 0x563f0ed81390 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.420+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8e680 con 0x563f0ed875f0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.421+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94828832018016] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.421+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.421+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94828832022896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8ea20 con 0x563f0ed875f0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.421+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8f4e0 con 0x7ff24003b2a0 2024-04-23T19:12:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.422+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.422+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94828832026544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8fd80 con 0x7ff24003b2a0 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.422+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed90860 con 0x563f0ed81390 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.422+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94828832022896] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.425+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.425+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94828832031600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed90c50 con 0x563f0ed81390 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.425+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed91650 con 0x7ff23809c040 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.425+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94828832026544] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.426+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94828832031600] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.426+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.426+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94828832019072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed91160 con 0x7ff23809c040 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.426+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed91f20 con 0x7ff23809c040 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.427+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94828832019072] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.427+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.427+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94828832028880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed91cb0 con 0x7ff23809c040 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.427+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed92500 con 0x7ff23809c040 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94828832028880] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94828832037344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed92370 con 0x7ff23809c040 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 20 add_watcher() i=0 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 20 add_watcher() i=1 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 20 add_watcher() i=2 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 20 add_watcher() i=3 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 20 add_watcher() i=4 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 20 add_watcher() i=5 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.428+0000 7ff248062800 20 add_watcher() i=6 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.429+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94828832037344] v32'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.429+0000 7ff248062800 20 add_watcher() i=7 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.429+0000 7ff248062800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.429+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed958e0 con 0x563f0ed81390 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.430+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.430+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94828832038400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed92e00 con 0x563f0ed81390 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94828832038400] v32'230 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff0f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff248062800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff248062800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff248062800 5 note: GC not initialized 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff0f67fc640 20 reqs_thread_entry: start 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff248062800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff248062800 20 Filter name: none 2024-04-23T19:12:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff067fff640 20 reqs_thread_entry: start 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff248062800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.431+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0edc8970 con 0x563f0ed81390 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.432+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.432+0000 7ff248062800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.432+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563f0edc9d60 con 0x7ff23809c040 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.432+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv32 ondisk = 0) ==== 152+0+15 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.433+0000 7ff0f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.433+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94828832038400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed7d3f0 con 0x563f0ed81390 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.434+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94828832038400] v32'231 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.434+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94828832014288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed91e00 con 0x7ff24003b2a0 2024-04-23T19:12:12.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.435+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94828832014288] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.435+0000 7ff248062800 20 remove_watcher() i=0 2024-04-23T19:12:12.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.435+0000 7ff248062800 2 removed watcher, disabling cache 2024-04-23T19:12:12.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.435+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94828832018016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8bfd0 con 0x563f0ed81390 2024-04-23T19:12:12.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.436+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94828832018016] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.436+0000 7ff248062800 20 remove_watcher() i=1 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.436+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94828832022896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8bfd0 con 0x563f0ed875f0 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.437+0000 7ff246362640 1 -- 172.21.15.63:0/2611163666 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94828832022896] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff240029070 con 0x563f0ed875f0 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.438+0000 7ff248062800 20 remove_watcher() i=2 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.438+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94828832026544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8e170 con 0x7ff24003b2a0 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.438+0000 7ff245b61640 1 -- 172.21.15.63:0/2611163666 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94828832026544] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff238075030 con 0x7ff24003b2a0 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.438+0000 7ff248062800 20 remove_watcher() i=3 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.438+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94828832031600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8bfd0 con 0x563f0ed81390 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.439+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94828832031600] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x563f0ed81390 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.440+0000 7ff248062800 20 remove_watcher() i=4 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.440+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94828832019072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8efb0 con 0x7ff23809c040 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.441+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94828832019072] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.441+0000 7ff248062800 20 remove_watcher() i=5 2024-04-23T19:12:12.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.441+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94828832028880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8e170 con 0x7ff23809c040 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.442+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94828832028880] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.442+0000 7ff248062800 20 remove_watcher() i=6 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.442+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94828832037344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563f0ed8e170 con 0x7ff23809c040 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94828832037344] v32'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff23c00f5c0 con 0x7ff23809c040 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 20 remove_watcher() i=7 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff23809c040 msgr2=0x7ff23805c720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7ff23809c040 0x7ff23805c720 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ed81390 msgr2=0x563f0ed83850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ed81390 0x563f0ed83850 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff24003b2a0 msgr2=0x7ff24004eaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7ff24003b2a0 0x7ff24004eaa0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563f0ed875f0 msgr2=0x563f0ed899e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563f0ed875f0 0x563f0ed899e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff110036e60 msgr2=0x7ff110039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff110036e60 0x7ff110039320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff238077520 tx=0x7ff238089000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 msgr2=0x563f0ed755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2611163666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed6e300 0x563f0ed755f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff2400129a0 tx=0x7ff24004cbd0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 reap_dead start 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff245360640 1 -- 172.21.15.63:0/2611163666 reap_dead start 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 shutdown_connections 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 >> 172.21.15.63:0/2611163666 conn(0x563f0ed6a5d0 msgr2=0x563f0ed67c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 shutdown_connections 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2611163666 wait complete. 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff120036e60 msgr2=0x7ff120039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff120036e60 0x7ff120039320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x563f0ed4e340 tx=0x7ff23805d920 comp rx=0 tx=0).stop 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed60f70 msgr2=0x563f0ed61390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.443+0000 7ff248062800 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed60f70 0x563f0ed61390 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x563f0ed4d270 tx=0x7ff2400178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 shutdown_connections 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff120036e60 0x7ff120039320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/2568290597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ed60f70 0x563f0ed61390 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 >> 172.21.15.63:0/2568290597 conn(0x563f0ed55010 msgr2=0x563f0ed55410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 shutdown_connections 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/2568290597 wait complete. 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563f0ed4ff90 msgr2=0x563f0ed52380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563f0ed4ff90 0x563f0ed52380 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ec4c0b0 msgr2=0x563f0ed4c890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ec4c0b0 0x563f0ed4c890 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff12c036e60 msgr2=0x7ff12c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff12c036e60 0x7ff12c039320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff23c000f60 tx=0x7ff23c001370 comp rx=0 tx=0).stop 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 msgr2=0x563f0ec4adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4adf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff2380524f0 tx=0x7ff238003020 comp rx=0 tx=0).stop 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 shutdown_connections 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563f0ed4ff90 0x563f0ed52380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563f0ec4c0b0 0x563f0ed4c890 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7ff12c036e60 0x7ff12c039320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 --2- 172.21.15.63:0/3576348827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563f0ec4b750 0x563f0ec4adf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 >> 172.21.15.63:0/3576348827 conn(0x563f0ec43db0 msgr2=0x563f0ed35b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 shutdown_connections 2024-04-23T19:12:12.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.444+0000 7ff248062800 1 -- 172.21.15.63:0/3576348827 wait complete. 2024-04-23T19:12:12.454 INFO:teuthology.orchestra.run.smithi063.stdout:{"entries":[],"summary":[]} 2024-04-23T19:12:12.455 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-23T19:12:12.455 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-23T19:12:12.455 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-23T19:12:12.455 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-04-23T19:12:12.499 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:12.500 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:12.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.500+0000 7f4711c87800 1 Processor -- start 2024-04-23T19:12:12.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.500+0000 7f4711c87800 1 -- start start 2024-04-23T19:12:12.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.500+0000 7f4711c87800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55a13872b000 0x55a13872b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.500+0000 7f4711c87800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a1385de7f0 con 0x55a13872c260 2024-04-23T19:12:12.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.500+0000 7f4711c87800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55a1385cde20 con 0x55a13872b000 2024-04-23T19:12:12.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.500+0000 7f470f786640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55a13872b000 0x55a13872b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.500+0000 7f470ef85640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55a13872c260 0x55a13872c6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56618/0 (socket says 172.21.15.63:56618) 2024-04-23T19:12:12.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.501+0000 7f470ef85640 1 -- 172.21.15.63:0/2485543883 learned_addr learned my addr 172.21.15.63:0/2485543883 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:12.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.501+0000 7f470f786640 1 -- 172.21.15.63:0/2485543883 >> v1:172.21.15.16:6789/0 conn(0x55a13872c260 legacy=0x55a13872c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.501+0000 7f470f786640 1 -- 172.21.15.63:0/2485543883 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1385ecb50 con 0x55a13872b000 2024-04-23T19:12:12.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f470f786640 1 --2- 172.21.15.63:0/2485543883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13872b420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f470800a470 tx=0x7f470801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1705045fb8b0bd04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f470e784640 1 -- 172.21.15.63:0/2485543883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4708057070 con 0x55a13872b000 2024-04-23T19:12:12.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f470e784640 1 -- 172.21.15.63:0/2485543883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4708052570 con 0x55a13872b000 2024-04-23T19:12:12.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f470e784640 1 -- 172.21.15.63:0/2485543883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4708052870 con 0x55a13872b000 2024-04-23T19:12:12.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f4711c87800 1 -- 172.21.15.63:0/2485543883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 msgr2=0x55a13872b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f4711c87800 1 --2- 172.21.15.63:0/2485543883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13872b420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f470800a470 tx=0x7f470801ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:12.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f4711c87800 1 -- 172.21.15.63:0/2485543883 shutdown_connections 2024-04-23T19:12:12.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f4711c87800 1 --2- 172.21.15.63:0/2485543883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13872b420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f4711c87800 1 -- 172.21.15.63:0/2485543883 >> 172.21.15.63:0/2485543883 conn(0x55a138726600 msgr2=0x55a138728a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f4711c87800 1 -- 172.21.15.63:0/2485543883 shutdown_connections 2024-04-23T19:12:12.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.503+0000 7f4711c87800 1 -- 172.21.15.63:0/2485543883 wait complete. 2024-04-23T19:12:12.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.516+0000 7f4711c87800 1 Processor -- start 2024-04-23T19:12:12.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.516+0000 7f4711c87800 1 -- start start 2024-04-23T19:12:12.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.516+0000 7f4711c87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13881b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.516+0000 7f4711c87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a1386776f0 con 0x55a13872b000 2024-04-23T19:12:12.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.517+0000 7f470f786640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13881b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.517+0000 7f470f786640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13881b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40006/0 (socket says 172.21.15.63:40006) 2024-04-23T19:12:12.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.517+0000 7f470f786640 1 -- 172.21.15.63:0/499183504 learned_addr learned my addr 172.21.15.63:0/499183504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.517+0000 7f470f786640 1 -- 172.21.15.63:0/499183504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13881bbc0 con 0x55a13872b000 2024-04-23T19:12:12.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.517+0000 7f470f786640 1 --2- 172.21.15.63:0/499183504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13881b5f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4708055040 tx=0x7f470804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef2ecec61763552 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f45fd60a640 1 -- 172.21.15.63:0/499183504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4708057040 con 0x55a13872b000 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f45fd60a640 1 -- 172.21.15.63:0/499183504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4708052990 con 0x55a13872b000 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f45fd60a640 1 -- 172.21.15.63:0/499183504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4708052c70 con 0x55a13872b000 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f4711c87800 1 -- 172.21.15.63:0/499183504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 msgr2=0x55a13881b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f4711c87800 1 --2- 172.21.15.63:0/499183504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13881b5f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4708055040 tx=0x7f470804c000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f4711c87800 1 -- 172.21.15.63:0/499183504 shutdown_connections 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f4711c87800 1 --2- 172.21.15.63:0/499183504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a13881b5f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f4711c87800 1 -- 172.21.15.63:0/499183504 >> 172.21.15.63:0/499183504 conn(0x55a138726600 msgr2=0x55a138816f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f4711c87800 1 -- 172.21.15.63:0/499183504 shutdown_connections 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.518+0000 7f4711c87800 1 -- 172.21.15.63:0/499183504 wait complete. 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.519+0000 7f4711c87800 1 Processor -- start 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.519+0000 7f4711c87800 1 -- start start 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.519+0000 7f4711c87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a138823040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.519+0000 7f4711c87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a13869f070 con 0x55a13872b000 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.519+0000 7f470f786640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a138823040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.519+0000 7f470f786640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a138823040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40016/0 (socket says 172.21.15.63:40016) 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.519+0000 7f470f786640 1 -- 172.21.15.63:0/1709163225 learned_addr learned my addr 172.21.15.63:0/1709163225 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.520+0000 7f470f786640 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a138823580 con 0x55a13872b000 2024-04-23T19:12:12.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.520+0000 7f470f786640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a138823040 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f470800a470 tx=0x7f4708003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.520+0000 7f45fbe07640 1 -- 172.21.15.63:0/1709163225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4708073070 con 0x55a13872b000 2024-04-23T19:12:12.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.520+0000 7f45fbe07640 1 -- 172.21.15.63:0/1709163225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470805d570 con 0x55a13872b000 2024-04-23T19:12:12.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.520+0000 7f45fbe07640 1 -- 172.21.15.63:0/1709163225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4708057070 con 0x55a13872b000 2024-04-23T19:12:12.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.520+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a138823810 con 0x55a13872b000 2024-04-23T19:12:12.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.520+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1388264c0 con 0x55a13872b000 2024-04-23T19:12:12.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.521+0000 7f45fbe07640 1 -- 172.21.15.63:0/1709163225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4708076070 con 0x55a13872b000 2024-04-23T19:12:12.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.521+0000 7f45fbe07640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45f4036eb0 0x7f45f4039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.521+0000 7f45fbe07640 1 -- 172.21.15.63:0/1709163225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f470808e660 con 0x55a13872b000 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.521+0000 7f470ef85640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45f4036eb0 0x7f45f4039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.521+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13872c260 0x55a13882a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.521+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a13882a800 con 0x55a13872c260 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.521+0000 7f470ff87640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13872c260 0x55a13882a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.522+0000 7f470ef85640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45f4036eb0 0x7f45f4039370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46f8000f60 tx=0x7f46f80012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.522+0000 7f470ff87640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13872c260 0x55a13882a280 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.522+0000 7f470ff87640 1 -- 172.21.15.63:0/1709163225 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f470000a4a0 con 0x55a13872c260 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.522+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138827850 con 0x55a13872c260 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.522+0000 7f470ff87640 1 -- 172.21.15.63:0/1709163225 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f470000a4a0 con 0x55a13872c260 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.522+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a13882c600 con 0x55a13872c260 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.523+0000 7f470ff87640 1 -- 172.21.15.63:0/1709163225 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f470000a4a0 con 0x55a13872c260 2024-04-23T19:12:12.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.523+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a13882b940 con 0x55a13872c260 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.523+0000 7f470ff87640 1 -- 172.21.15.63:0/1709163225 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f470000a4a0 con 0x55a13872c260 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.523+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a13882d9b0 0x55a13882fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.523+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138830320 con 0x55a13882d9b0 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.523+0000 7f470f786640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a13882d9b0 0x55a13882fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.524+0000 7f470f786640 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a13882d9b0 0x55a13882fda0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.524+0000 7f470f786640 1 -- 172.21.15.63:0/1709163225 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4708075030 con 0x55a13882d9b0 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.524+0000 7f4711c87800 1 Processor -- start 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.524+0000 7f4711c87800 1 -- start start 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.524+0000 7f4711c87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13883e720 0x55a13883eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f4711c87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a13861de30 con 0x55a13883e720 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f470ff87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13883e720 0x55a13883eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f470ff87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13883e720 0x55a13883eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40018/0 (socket says 172.21.15.63:40018) 2024-04-23T19:12:12.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f470ff87640 1 -- 172.21.15.63:0/1297486700 learned_addr learned my addr 172.21.15.63:0/1297486700 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f470ff87640 1 -- 172.21.15.63:0/1297486700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13883f080 con 0x55a13883e720 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f470ff87640 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13883e720 0x55a13883eb40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x55a138824440 tx=0x7f47000178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f45f9602640 1 -- 172.21.15.63:0/1297486700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4700013440 con 0x55a13883e720 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f45f9602640 1 -- 172.21.15.63:0/1297486700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47000135e0 con 0x55a13883e720 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a13883faf0 con 0x55a13883e720 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f45f9602640 1 -- 172.21.15.63:0/1297486700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47000138c0 con 0x55a13883e720 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.525+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a13883fff0 con 0x55a13883e720 2024-04-23T19:12:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.526+0000 7f45f9602640 1 -- 172.21.15.63:0/1297486700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4700011040 con 0x55a13883e720 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.526+0000 7f45f9602640 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45e8036e60 0x7f45e8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.526+0000 7f45f9602640 1 -- 172.21.15.63:0/1297486700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f470001d070 con 0x55a13883e720 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.526+0000 7f470f786640 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45e8036e60 0x7f45e8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f4711c87800 1 Processor -- start 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f4711c87800 1 -- start start 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f470f786640 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45e8036e60 0x7f45e8039320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4708052bf0 tx=0x7f4708006f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f4711c87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a13884bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f4711c87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a13881be80 con 0x55a13884bad0 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f470ff87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a13884bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f470ff87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a13884bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40024/0 (socket says 172.21.15.63:40024) 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f470ff87640 1 -- 172.21.15.63:0/3310192636 learned_addr learned my addr 172.21.15.63:0/3310192636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.527+0000 7f470ff87640 1 -- 172.21.15.63:0/3310192636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13884c4c0 con 0x55a13884bad0 2024-04-23T19:12:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f470ff87640 1 --2- 172.21.15.63:0/3310192636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a13884bef0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f470004ccb0 tx=0x7f4700012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc5d817a4c7decd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f45f2ffd640 1 -- 172.21.15.63:0/3310192636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4700012df0 con 0x55a13884bad0 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f45f2ffd640 1 -- 172.21.15.63:0/3310192636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470003f030 con 0x55a13884bad0 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f45f2ffd640 1 -- 172.21.15.63:0/3310192636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4700001480 con 0x55a13884bad0 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f4711c87800 1 -- 172.21.15.63:0/3310192636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 msgr2=0x55a13884bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f4711c87800 1 --2- 172.21.15.63:0/3310192636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a13884bef0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f470004ccb0 tx=0x7f4700012910 comp rx=0 tx=0).stop 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f4711c87800 1 -- 172.21.15.63:0/3310192636 shutdown_connections 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f4711c87800 1 --2- 172.21.15.63:0/3310192636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a13884bef0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f4711c87800 1 -- 172.21.15.63:0/3310192636 >> 172.21.15.63:0/3310192636 conn(0x55a138847da0 msgr2=0x55a138845e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f4711c87800 1 -- 172.21.15.63:0/3310192636 shutdown_connections 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.528+0000 7f4711c87800 1 -- 172.21.15.63:0/3310192636 wait complete. 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f4711c87800 1 Processor -- start 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f4711c87800 1 -- start start 2024-04-23T19:12:12.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f4711c87800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a138852ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f4711c87800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a13884cc50 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f470ff87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a138852ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f470ff87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a138852ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40038/0 (socket says 172.21.15.63:40038) 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 learned_addr learned my addr 172.21.15.63:0/349441107 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.529+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a138853530 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.530+0000 7f470ff87640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a138852ff0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4700015ac0 tx=0x7f470004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.530+0000 7f45f17fa640 1 -- 172.21.15.63:0/349441107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4700021a50 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.530+0000 7f45f17fa640 1 -- 172.21.15.63:0/349441107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4700021bf0 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.530+0000 7f45f17fa640 1 -- 172.21.15.63:0/349441107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470002a350 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.530+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a1388537c0 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.530+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a138856470 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.530+0000 7f45f17fa640 1 -- 172.21.15.63:0/349441107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f4700040040 con 0x55a13884bad0 2024-04-23T19:12:12.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f45f17fa640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45d8036e60 0x7f45d8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f45f17fa640 1 -- 172.21.15.63:0/349441107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f470000e040 con 0x55a13884bad0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f470f786640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45d8036e60 0x7f45d8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f470f786640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45d8036e60 0x7f45d8039320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f470805d9e0 tx=0x7f4708089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13885ed00 0x55a1388611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138861910 con 0x55a13885ed00 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.531+0000 7f470ef85640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13885ed00 0x55a1388611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f470ef85640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13885ed00 0x55a1388611c0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f8009050 con 0x55a13885ed00 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 20 realm 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a13885e150 con 0x55a13885ed00 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f8009050 con 0x55a13885ed00 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138862400 con 0x55a13885ed00 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.532+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f8009050 con 0x55a13885ed00 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138862cd0 con 0x55a13885ed00 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f46f8009050 con 0x55a13885ed00 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138863070 con 0x55a13885ed00 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f46f8009050 con 0x55a13885ed00 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 20 searching for the correct realm 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a138864f50 0x55a138867340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.533+0000 7f470ff87640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a138864f50 0x55a138867340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.534+0000 7f470ff87640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a138864f50 0x55a138867340 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.534+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.534+0000 7f470ff87640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47000123a0 0x7f4700046660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.534+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f47000123a0 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.534+0000 7f470f786640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47000123a0 0x7f4700046660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.534+0000 7f470f786640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47000123a0 0x7f4700046660 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.535+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.535+0000 7f470f786640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f470809c040 0x7f470808e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.535+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.535+0000 7f470ef85640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f470809c040 0x7f470808e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.535+0000 7f470ef85640 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f470809c040 0x7f470808e050 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.536+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f8009050 con 0x7f470809c040 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.536+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.536+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f8009050 con 0x55a13885ed00 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.536+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.536+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.536+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f47000123a0 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.537+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f47000123a0 2024-04-23T19:12:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.538+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.539+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.540+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.540+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.540+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.540+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.540+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.540+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f47000123a0 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.541+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f47000123a0 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.542+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.542+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.542+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.542+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.542+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.542+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f47000123a0 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a138864f50 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f470809c040 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x55a13885ed00 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.543+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55a1388679f0 con 0x7f47000123a0 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138863a20 con 0x55a13885ed00 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388642c0 con 0x55a13885ed00 2024-04-23T19:12:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.544+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138867e70 con 0x7f470809c040 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 zone default found 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 4 Realm: () 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 zonegroup default 2024-04-23T19:12:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138868320 con 0x7f47000123a0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a13886a0a0 con 0x55a138864f50 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a13886a560 con 0x55a138864f50 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 started sync module instance, tier type = 2024-04-23T19:12:12.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.545+0000 7f4711c87800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.546+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138869540 con 0x7f47000123a0 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.546+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.547+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94150926440752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886ae70 con 0x7f47000123a0 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.547+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886b6c0 con 0x55a13885ed00 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.548+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94150926440752] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.548+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.548+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94150926444992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886b780 con 0x55a13885ed00 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.548+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886c2a0 con 0x55a138864f50 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.549+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94150926444992] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.549+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.549+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94150926450032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886c640 con 0x55a138864f50 2024-04-23T19:12:12.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.549+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886d0c0 con 0x7f47000123a0 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.550+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94150926450032] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.550+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.550+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94150926453680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886d4e0 con 0x7f47000123a0 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.550+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886df00 con 0x55a13885ed00 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.551+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94150926453680] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.551+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.551+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94150926457328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886e2f0 con 0x55a13885ed00 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.551+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886ecf0 con 0x7f470809c040 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.552+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.552+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94150926457328] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.552+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94150926460928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886ec60 con 0x7f470809c040 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.552+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886b250 con 0x7f470809c040 2024-04-23T19:12:12.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.553+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94150926460928] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.554+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.554+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94150926447184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886f760 con 0x7f470809c040 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.554+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138870120 con 0x7f470809c040 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94150926447184] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94150926463168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138870120 con 0x7f470809c040 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 20 add_watcher() i=0 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 20 add_watcher() i=1 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 20 add_watcher() i=2 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 20 add_watcher() i=3 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 20 add_watcher() i=4 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 20 add_watcher() i=5 2024-04-23T19:12:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.555+0000 7f4711c87800 20 add_watcher() i=6 2024-04-23T19:12:12.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.556+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94150926463168] v32'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.556+0000 7f4711c87800 20 add_watcher() i=7 2024-04-23T19:12:12.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.556+0000 7f4711c87800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:12.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.556+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138873540 con 0x55a13885ed00 2024-04-23T19:12:12.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.556+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.556+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94150926466624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388709d0 con 0x55a13885ed00 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.557+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94150926466624] v32'232 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.557+0000 7f45c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.557+0000 7f4711c87800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.557+0000 7f4711c87800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.557+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887be30 con 0x55a13885ed00 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.559+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'35 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.559+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887c500 con 0x7f470809c040 2024-04-23T19:12:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.560+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.560+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887c940 con 0x55a138864f50 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.561+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.561+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887c8d0 con 0x55a13885ed00 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.562+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'233 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.562+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887cec0 con 0x7f47000123a0 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.563+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'74 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.563+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887d370 con 0x55a138864f50 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.564+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'49 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.564+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887d370 con 0x55a13885ed00 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.565+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.565+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887d370 con 0x7f470809c040 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.567+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'49 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.567+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887d370 con 0x7f47000123a0 2024-04-23T19:12:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.568+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.568+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879c80 con 0x55a138864f50 2024-04-23T19:12:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.569+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.569+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879c80 con 0x7f47000123a0 2024-04-23T19:12:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.569+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'75 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.570+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f470809c040 2024-04-23T19:12:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.571+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'34 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.571+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f47000123a0 2024-04-23T19:12:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.572+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.572+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f470809c040 2024-04-23T19:12:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.573+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'56 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.573+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a13885ed00 2024-04-23T19:12:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.574+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'234 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.574+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a138864f50 2024-04-23T19:12:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.575+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'69 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.575+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879600 con 0x7f47000123a0 2024-04-23T19:12:12.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.576+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'76 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.576+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879600 con 0x55a138864f50 2024-04-23T19:12:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.577+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.577+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879600 con 0x55a13885ed00 2024-04-23T19:12:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.579+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'36 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.579+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879600 con 0x55a13885ed00 2024-04-23T19:12:12.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.580+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'235 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.580+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879600 con 0x55a138864f50 2024-04-23T19:12:12.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.581+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'70 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.581+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f47000123a0 2024-04-23T19:12:12.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.582+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'38 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.582+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a138864f50 2024-04-23T19:12:12.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.583+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.583+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a138864f50 2024-04-23T19:12:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.584+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.584+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a138864f50 2024-04-23T19:12:12.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.585+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'24 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.585+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a138864f50 2024-04-23T19:12:12.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.586+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'50 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.586+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f470809c040 2024-04-23T19:12:12.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.587+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.587+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a138864f50 2024-04-23T19:12:12.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.588+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.588+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f470809c040 2024-04-23T19:12:12.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.589+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'50 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.589+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x55a138864f50 2024-04-23T19:12:12.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.590+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'71 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.590+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f470809c040 2024-04-23T19:12:12.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.591+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'57 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.591+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138879d10 con 0x7f47000123a0 2024-04-23T19:12:12.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.592+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f45c4fe1640 20 reqs_thread_entry: start 2024-04-23T19:12:12.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:12.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 20 Filter name: none 2024-04-23T19:12:12.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f453e7fc640 20 reqs_thread_entry: start 2024-04-23T19:12:12.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388a8990 con 0x55a13885ed00 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 20 rados_obj.operate() r=0 bl.length=391 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.593+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aade0 con 0x7f47000123a0 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv43 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 : miss 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:96 8.12 8:4d9c725e:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aad50 con 0x55a138864f50 2024-04-23T19:12:12.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(96 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 20 rados_obj.operate() r=0 bl.length=379 2024-04-23T19:12:12.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 info.flags=0x17 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 to cache LRU end 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.594+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 4.38 4:1e631ee4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1389284e0 con 0x55a13885ed00 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.595+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(97 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.595+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:98 4.d 4:b27ae7e9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1389284e0 con 0x55a138864f50 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.595+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(98 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.595+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:99 4.21 4:846cbf38:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x7f470809c040 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(99 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:100 4.27 4:e59415c9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x55a13885ed00 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(100 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:101 4.31 4:8c10c9f5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x55a13885ed00 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(101 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4 [call out=425b] v0'0 uv5 ondisk = 0) ==== 194+0+425 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f4711c87800 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:102 4.15 4:a9d60cf5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x55a138864f50 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.596+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(102 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.597+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:103 4.2d 4:b7e2bfec:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x7f470809c040 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.597+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(103 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.597+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:104 4.4 4:23c5a37c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1389284e0 con 0x7f470809c040 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.597+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(104 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.597+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:105 4.10 4:097008e7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x55a138864f50 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.598+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(105 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.598+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:106 4.3c 4:3eb82e1b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x55a13885ed00 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.598+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(106 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.598+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 4.b 4:d31a5e11:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a138928520 con 0x7f47000123a0 2024-04-23T19:12:12.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.598+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(107 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.598+0000 7f4711c87800 20 get_obj_state: octx=0x55a13887a240 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55a138926568 s->prefetch_data=0 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.598+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:108 3.19s0 3:999f8241:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1389273f0 con 0x55a13885ed00 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.599+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(108 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+893 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.599+0000 7f4711c87800 10 manifest: total_size = 6 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.599+0000 7f4711c87800 20 get_obj_state: setting s->obj_tag to bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.5764214616888296337 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.599+0000 7f4711c87800 20 get_obj_state: octx=0x55a13887a240 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55a138926568 s->prefetch_data=0 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.599+0000 7f4711c87800 20 prepare_atomic_modification: state is not atomic. state=0x55a138926568 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.599+0000 7f4711c87800 20 bucket index object: :.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.599+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:109 4.31 4:8c10c9f5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138927de0 con 0x55a13885ed00 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.600+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(109 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4 [stat,call,call] v32'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.600+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:110 3.19s0 3:999f8241:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138927de0 con 0x55a13885ed00 2024-04-23T19:12:12.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.601+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(110 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7_15f6f8dc036519d7fe15b39338f6e5db [call] v32'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.601+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:111 4.31 4:8c10c9f5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388af4c0 con 0x55a13885ed00 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.601+0000 7f4711c87800 20 list_objects_ordered: starting attempt 1 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.601+0000 7f4711c87800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:112 4.38 4:1e631ee4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388af4c0 con 0x55a13885ed00 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:113 4.d 4:b27ae7e9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b37a0 con 0x55a138864f50 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:114 4.21 4:846cbf38:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b60a0 con 0x7f470809c040 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:115 4.27 4:e59415c9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b8970 con 0x55a13885ed00 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:116 4.31 4:8c10c9f5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bb480 con 0x55a13885ed00 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:117 4.15 4:a9d60cf5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388be2a0 con 0x55a138864f50 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:118 4.2d 4:b7e2bfec:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388be2a0 con 0x7f470809c040 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:119 4.4 4:23c5a37c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b89b0 con 0x7f470809c040 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:120 4.10 4:097008e7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388c5550 con 0x55a138864f50 2024-04-23T19:12:12.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:121 4.3c 4:3eb82e1b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b89b0 con 0x55a13885ed00 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:122 4.b 4:d31a5e11:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388ca6b0 con 0x7f47000123a0 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(115 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.602+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(112 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(113 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(117 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(121 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(119 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(114 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(118 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(122 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(120 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(111 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4 [stat,call,call] v32'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f45f2ffd640 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== osd_op_reply(116 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7])"; uploads.size()=0; is_truncated=0 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:123 4.38 4:1e631ee4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bdc50 con 0x55a13885ed00 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:124 4.d 4:b27ae7e9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b4be0 con 0x55a138864f50 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:125 4.21 4:846cbf38:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b5fd0 con 0x7f470809c040 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:126 4.27 4:e59415c9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bdc50 con 0x55a13885ed00 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:127 4.31 4:8c10c9f5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b5fd0 con 0x55a13885ed00 2024-04-23T19:12:12.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.603+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:128 4.15 4:a9d60cf5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b4be0 con 0x55a138864f50 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:129 4.2d 4:b7e2bfec:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bdc50 con 0x7f470809c040 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(123 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:130 4.4 4:23c5a37c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b5fd0 con 0x7f470809c040 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(124 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:131 4.10 4:097008e7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bdc50 con 0x55a138864f50 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(126 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:132 4.3c 4:3eb82e1b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388b4be0 con 0x55a13885ed00 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== osd_op_reply(127 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:133 4.b 4:d31a5e11:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bdc50 con 0x7f47000123a0 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(125 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(129 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(130 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(132 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== osd_op_reply(128 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== osd_op_reply(131 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(133 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.604+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:134 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bb4e0 con 0x7f47000123a0 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.605+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(134 foo.buckets [call] v32'46 uv46 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f4708075030 con 0x7f47000123a0 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.605+0000 7f4711c87800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.605+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.605+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:135 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388ca6b0 con 0x7f47000123a0 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.606+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(135 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv40 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.606+0000 7f4711c87800 20 rados_obj.operate() r=0 bl.length=165 2024-04-23T19:12:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.606+0000 7f4711c87800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.606+0000 7f4711c87800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.606+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:136 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1389276f0 con 0x7f47000123a0 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(136 myfoo [call,call,delete] v32'47 uv47 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f4711c87800 10 removing default.rgw.meta+root+myfoo from cache 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f4711c87800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:137 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94150927218416 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388ca050 con 0x7f470809c040 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== watch-notify(notify (1) cookie 94150926447184 notify 137438953513 ret 0) ==== 191+0+0 (crc 0 0 0) 0x55a13887c500 con 0x7f470809c040 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f45f2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953513 cookie 94150926447184 notifier 4779 bl.length()=149 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f45f2ffd640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:138 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f45d4003d60 con 0x7f470809c040 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(137 notify.6 [notify cookie 94150927218416 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== osd_op_reply(138 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.607+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== watch-notify(notify_complete (2) cookie 94150927218416 notify 137438953513 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a138856470 con 0x7f470809c040 2024-04-23T19:12:12.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.608+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:139 8.12 8:4d9c725e:root::.bucket.meta.myfoo%3abc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bdb10 con 0x55a138864f50 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 36 ==== osd_op_reply(139 .bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 [call,call,delete] v32'6 uv6 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 from cache 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7, ofs0, ns] 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:140 7.3 7:c609908c:::notify.5:head [notify cookie 94150927218416 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa620 con 0x7f470809c040 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== watch-notify(notify (1) cookie 94150926460928 notify 137438953514 ret 0) ==== 248+0+0 (crc 0 0 0) 0x55a138856470 con 0x7f470809c040 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f45f27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953514 cookie 94150926460928 notifier 4779 bl.length()=206 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f45f27fc640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:141 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f4544003320 con 0x7f470809c040 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(140 notify.5 [notify cookie 94150927218416 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(141 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== watch-notify(notify_complete (2) cookie 94150927218416 notify 137438953514 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a13887bfc0 con 0x7f470809c040 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:142 4.38 4:1e631ee4:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bdb10 con 0x55a13885ed00 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:143 4.d 4:b27ae7e9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388c9f90 con 0x55a138864f50 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:144 4.21 4:846cbf38:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388a9630 con 0x7f470809c040 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:145 4.27 4:e59415c9:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138925c80 con 0x55a13885ed00 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:146 4.31 4:8c10c9f5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138926060 con 0x55a13885ed00 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:147 4.15 4:a9d60cf5:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388c9f90 con 0x55a138864f50 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:148 4.2d 4:b7e2bfec:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388a9630 con 0x7f470809c040 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:149 4.4 4:23c5a37c:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138925c80 con 0x7f470809c040 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.609+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:150 4.10 4:097008e7:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138926060 con 0x55a138864f50 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.610+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:151 4.3c 4:3eb82e1b:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388a9630 con 0x55a13885ed00 2024-04-23T19:12:12.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.610+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:152 4.b 4:d31a5e11:::.dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a138925c80 con 0x7f47000123a0 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.611+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(149 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.7 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.611+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(152 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.10 [delete] v32'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.611+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== osd_op_reply(144 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.2 [delete] v32'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.611+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(142 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.0 [delete] v32'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.613+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 37 ==== osd_op_reply(147 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.5 [delete] v32'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.613+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(151 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.9 [delete] v32'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.613+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(146 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.4 [delete] v32'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.613+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 42 ==== osd_op_reply(148 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.6 [delete] v32'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.613+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(145 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.3 [delete] v32'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.613+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 38 ==== osd_op_reply(143 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.1 [delete] v32'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 39 ==== osd_op_reply(150 .dir.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7.8 [delete] v32'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f4711c87800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 : miss 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f4711c87800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:153 6.0 6:027b05e1:::pubsub..bucket.myfoo%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa5d0 con 0x7f470809c040 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 43 ==== osd_op_reply(153 pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f4711c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f4711c87800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 info.flags=0x0 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f4711c87800 10 adding default.rgw.log++pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 to cache LRU end 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.614+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:154 6.0 6:027b05e1:::pubsub..bucket.myfoo%2fbc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa5d0 con 0x7f470809c040 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.615+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 44 ==== osd_op_reply(154 pubsub..bucket.myfoo/bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.7 [delete] v32'58 uv42 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.615+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:155 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa5d0 con 0x7f47000123a0 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.616+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(155 foo.buckets [call] v32'48 uv48 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.616+0000 7f4711c87800 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-04-23T19:12:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.616+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:156 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bda00 con 0x7f470809c040 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.617+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 45 ==== osd_op_reply(156 9te6NH5mcdcq0Tc5i8i1 [delete] v32'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.617+0000 7f4711c87800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.617+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:157 7.e 7:7759931f:::notify.3:head [notify cookie 94150927218416 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa550 con 0x7f47000123a0 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.617+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== watch-notify(notify (1) cookie 94150926453680 notify 137438953501 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55a13887a360 con 0x7f47000123a0 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.617+0000 7f45f2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953501 cookie 94150926453680 notifier 4779 bl.length()=170 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.617+0000 7f45f2ffd640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:158 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f45d4003d60 con 0x7f47000123a0 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.617+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(157 notify.3 [notify cookie 94150927218416 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.618+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(158 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.618+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== watch-notify(notify_complete (2) cookie 94150927218416 notify 137438953501 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f47080a6060 con 0x7f47000123a0 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.618+0000 7f4711c87800 10 removing email index: foo@foo.com 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.618+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:159 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bda00 con 0x55a13885ed00 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 49 ==== osd_op_reply(159 foo@foo.com [delete] v32'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f4711c87800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:160 7.e 7:7759931f:::notify.3:head [notify cookie 94150927218416 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa550 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== watch-notify(notify (1) cookie 94150926453680 notify 137438953502 ret 0) ==== 204+0+0 (crc 0 0 0) 0x7f47080a6060 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f45f27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953502 cookie 94150926453680 notifier 4779 bl.length()=162 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f45f27fc640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:161 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f4544003320 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 34 ==== osd_op_reply(160 notify.3 [notify cookie 94150927218416 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 35 ==== osd_op_reply(161 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 36 ==== watch-notify(notify_complete (2) cookie 94150927218416 notify 137438953502 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a1388264c0 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f4711c87800 10 removing user buckets index 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.619+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:162 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bda00 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.620+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 37 ==== osd_op_reply(162 foo.buckets [delete] v32'49 uv49 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.620+0000 7f4711c87800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-04-23T19:12:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.620+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94150927218416 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa550 con 0x7f470809c040 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 46 ==== watch-notify(notify (1) cookie 94150926460928 notify 137438953515 ret 0) ==== 202+0+0 (crc 0 0 0) 0x7f46f8063040 con 0x7f470809c040 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f45f2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953515 cookie 94150926460928 notifier 4779 bl.length()=160 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f45f2ffd640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f45d4003d60 con 0x7f470809c040 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 47 ==== osd_op_reply(163 notify.5 [notify cookie 94150927218416 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 48 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 49 ==== watch-notify(notify_complete (2) cookie 94150927218416 notify 137438953515 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a13887bfc0 con 0x7f470809c040 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f4711c87800 10 removing user index: foo 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.621+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:165 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a1388bda00 con 0x55a13885ed00 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 50 ==== osd_op_reply(165 foo [call,call,delete] v32'19 uv19 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f4711c87800 10 removing default.rgw.meta+users.uid+foo from cache 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f4711c87800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:166 7.d 7:b4812045:::notify.4:head [notify cookie 94150927218416 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a1388aa550 con 0x55a13885ed00 2024-04-23T19:12:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 51 ==== watch-notify(notify (1) cookie 94150926457328 notify 137438953494 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55a13887bfc0 con 0x55a13885ed00 2024-04-23T19:12:12.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f45f27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953494 cookie 94150926457328 notifier 4779 bl.length()=152 2024-04-23T19:12:12.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f45f27fc640 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:167 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f45440032e0 con 0x55a13885ed00 2024-04-23T19:12:12.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.622+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 52 ==== osd_op_reply(166 notify.4 [notify cookie 94150927218416 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.623+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 53 ==== osd_op_reply(167 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.623+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 54 ==== watch-notify(notify_complete (2) cookie 94150927218416 notify 137438953494 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a13887bdf0 con 0x55a13885ed00 2024-04-23T19:12:12.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.624+0000 7f45c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.624+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:168 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94150926466624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887ac90 con 0x55a13885ed00 2024-04-23T19:12:12.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.625+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 55 ==== osd_op_reply(168 data_loggenerations_metadata [watch unwatch cookie 94150926466624] v32'236 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.625+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:169 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94150926440752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887ac90 con 0x7f47000123a0 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.625+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 38 ==== osd_op_reply(169 notify.0 [watch unwatch cookie 94150926440752] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.626+0000 7f4711c87800 20 remove_watcher() i=0 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.626+0000 7f4711c87800 2 removed watcher, disabling cache 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.626+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:170 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94150926444992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887ac90 con 0x55a13885ed00 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.627+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 56 ==== osd_op_reply(170 notify.1 [watch unwatch cookie 94150926444992] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.627+0000 7f4711c87800 20 remove_watcher() i=1 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.627+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:171 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94150926450032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886a9c0 con 0x55a138864f50 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.628+0000 7f470ff87640 1 -- 172.21.15.63:0/349441107 <== osd.3 v2:172.21.15.63:6816/1289558434 40 ==== osd_op_reply(171 notify.2 [watch unwatch cookie 94150926450032] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4700029070 con 0x55a138864f50 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.628+0000 7f4711c87800 20 remove_watcher() i=2 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.628+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:172 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94150926453680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887ac90 con 0x7f47000123a0 2024-04-23T19:12:12.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.629+0000 7f470f786640 1 -- 172.21.15.63:0/349441107 <== osd.2 v2:172.21.15.63:6808/753864785 39 ==== osd_op_reply(172 notify.3 [watch unwatch cookie 94150926453680] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f47080b1040 con 0x7f47000123a0 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.629+0000 7f4711c87800 20 remove_watcher() i=3 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.629+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:173 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94150926457328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886bd70 con 0x55a13885ed00 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.630+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.1 v2:172.21.15.63:6800/3216879695 57 ==== osd_op_reply(173 notify.4 [watch unwatch cookie 94150926457328] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x55a13885ed00 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.630+0000 7f4711c87800 20 remove_watcher() i=4 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.630+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:174 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94150926460928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886cbb0 con 0x7f470809c040 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.631+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 50 ==== osd_op_reply(174 notify.5 [watch unwatch cookie 94150926460928] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.631+0000 7f4711c87800 20 remove_watcher() i=5 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.631+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:175 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94150926447184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13887ac90 con 0x7f470809c040 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.632+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 51 ==== osd_op_reply(175 notify.6 [watch unwatch cookie 94150926447184] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.632+0000 7f4711c87800 20 remove_watcher() i=6 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.632+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:176 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94150926463168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55a13886bd70 con 0x7f470809c040 2024-04-23T19:12:12.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 <== osd.0 v2:172.21.15.16:6800/2262294208 52 ==== osd_op_reply(176 notify.7 [watch unwatch cookie 94150926463168] v32'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f802a040 con 0x7f470809c040 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 20 remove_watcher() i=7 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f470809c040 msgr2=0x7f470808e050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f470809c040 0x7f470808e050 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13885ed00 msgr2=0x55a1388611c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13885ed00 0x55a1388611c0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47000123a0 msgr2=0x7f4700046660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f47000123a0 0x7f4700046660 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a138864f50 msgr2=0x55a138867340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55a138864f50 0x55a138867340 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.633+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45d8036e60 msgr2=0x7f45d8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45d8036e60 0x7f45d8039320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f470805d9e0 tx=0x7f4708089000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f470ef85640 1 -- 172.21.15.63:0/349441107 reap_dead start 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 msgr2=0x55a138852ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a138852ff0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4700015ac0 tx=0x7f470004cc40 comp rx=0 tx=0).stop 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 shutdown_connections 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/349441107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13884bad0 0x55a138852ff0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 >> 172.21.15.63:0/349441107 conn(0x55a138847da0 msgr2=0x55a138845400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 shutdown_connections 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/349441107 wait complete. 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45e8036e60 msgr2=0x7f45e8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45e8036e60 0x7f45e8039320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4708052bf0 tx=0x7f4708006f50 comp rx=0 tx=0).stop 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13883e720 msgr2=0x55a13883eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13883e720 0x55a13883eb40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x55a138824440 tx=0x7f47000178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 shutdown_connections 2024-04-23T19:12:12.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45e8036e60 0x7f45e8039320 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/1297486700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13883e720 0x55a13883eb40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 >> 172.21.15.63:0/1297486700 conn(0x55a1388327c0 msgr2=0x55a138832bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 shutdown_connections 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1297486700 wait complete. 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a13882d9b0 msgr2=0x55a13882fda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a13882d9b0 0x55a13882fda0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13872c260 msgr2=0x55a13882a280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.634+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13872c260 0x55a13882a280 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45f4036eb0 msgr2=0x7f45f4039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45f4036eb0 0x7f45f4039370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46f8000f60 tx=0x7f46f80012b0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 msgr2=0x55a138823040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a138823040 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f470800a470 tx=0x7f4708003020 comp rx=0 tx=0).stop 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 shutdown_connections 2024-04-23T19:12:12.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55a13882d9b0 0x55a13882fda0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55a13872c260 0x55a13882a280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f45f4036eb0 0x7f45f4039370 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 --2- 172.21.15.63:0/1709163225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a13872b000 0x55a138823040 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 >> 172.21.15.63:0/1709163225 conn(0x55a138726600 msgr2=0x55a138815710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 shutdown_connections 2024-04-23T19:12:12.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.635+0000 7f4711c87800 1 -- 172.21.15.63:0/1709163225 wait complete. 2024-04-23T19:12:12.643 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-23T19:12:12.644 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-23T19:12:12.644 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-23T19:12:12.688 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:12.688 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.688+0000 7fe880b2f800 1 Processor -- start 2024-04-23T19:12:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.688+0000 7fe880b2f800 1 -- start start 2024-04-23T19:12:12.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.688+0000 7fe880b2f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5635076dd400 0x5635076dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.688+0000 7fe880b2f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5635075917f0 con 0x5635076e03f0 2024-04-23T19:12:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.688+0000 7fe880b2f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x563507580e20 con 0x5635076dd400 2024-04-23T19:12:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.689+0000 7fe87c7d1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5635076dd400 0x5635076dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.689+0000 7fe87bfd0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5635076e03f0 0x5635076e08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56634/0 (socket says 172.21.15.63:56634) 2024-04-23T19:12:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.689+0000 7fe87bfd0640 1 -- 172.21.15.63:0/3917917222 learned_addr learned my addr 172.21.15.63:0/3917917222 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.689+0000 7fe87c7d1640 1 -- 172.21.15.63:0/3917917222 >> v1:172.21.15.16:6789/0 conn(0x5635076e03f0 legacy=0x5635076e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.689+0000 7fe87c7d1640 1 -- 172.21.15.63:0/3917917222 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350759fb50 con 0x5635076dd400 2024-04-23T19:12:12.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.691+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/3917917222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635076dd820 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00a470 tx=0x7fe86c01ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935114c113112571 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.691+0000 7fe87b7cf640 1 -- 172.21.15.63:0/3917917222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe86c057070 con 0x5635076dd400 2024-04-23T19:12:12.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.691+0000 7fe87b7cf640 1 -- 172.21.15.63:0/3917917222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c04ad80 con 0x5635076dd400 2024-04-23T19:12:12.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.691+0000 7fe87b7cf640 1 -- 172.21.15.63:0/3917917222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe86c052590 con 0x5635076dd400 2024-04-23T19:12:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.692+0000 7fe880b2f800 1 -- 172.21.15.63:0/3917917222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 msgr2=0x5635076dd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.692+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3917917222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635076dd820 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00a470 tx=0x7fe86c01ce50 comp rx=0 tx=0).stop 2024-04-23T19:12:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.692+0000 7fe880b2f800 1 -- 172.21.15.63:0/3917917222 shutdown_connections 2024-04-23T19:12:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.692+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3917917222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635076dd820 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.692+0000 7fe880b2f800 1 -- 172.21.15.63:0/3917917222 >> 172.21.15.63:0/3917917222 conn(0x5635076d8db0 msgr2=0x5635076db1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.692+0000 7fe880b2f800 1 -- 172.21.15.63:0/3917917222 shutdown_connections 2024-04-23T19:12:12.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.692+0000 7fe880b2f800 1 -- 172.21.15.63:0/3917917222 wait complete. 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.705+0000 7fe880b2f800 1 Processor -- start 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.705+0000 7fe880b2f800 1 -- start start 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.705+0000 7fe880b2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.705+0000 7fe880b2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56350762a6f0 con 0x5635076dd400 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.705+0000 7fe87c7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.705+0000 7fe87c7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40066/0 (socket says 172.21.15.63:40066) 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.705+0000 7fe87c7d1640 1 -- 172.21.15.63:0/1660207521 learned_addr learned my addr 172.21.15.63:0/1660207521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.706+0000 7fe87c7d1640 1 -- 172.21.15.63:0/1660207521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635077d0e90 con 0x5635076dd400 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.706+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/1660207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d08c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe86c055040 tx=0x7fe86c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef617f721685e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.706+0000 7fe76a60c640 1 -- 172.21.15.63:0/1660207521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe86c057040 con 0x5635076dd400 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.706+0000 7fe76a60c640 1 -- 172.21.15.63:0/1660207521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c063070 con 0x5635076dd400 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.706+0000 7fe76a60c640 1 -- 172.21.15.63:0/1660207521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe86c0527e0 con 0x5635076dd400 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 -- 172.21.15.63:0/1660207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 msgr2=0x5635077d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 --2- 172.21.15.63:0/1660207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d08c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe86c055040 tx=0x7fe86c04c000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 -- 172.21.15.63:0/1660207521 shutdown_connections 2024-04-23T19:12:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 --2- 172.21.15.63:0/1660207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d08c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 -- 172.21.15.63:0/1660207521 >> 172.21.15.63:0/1660207521 conn(0x5635076d8db0 msgr2=0x5635077cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 -- 172.21.15.63:0/1660207521 shutdown_connections 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 -- 172.21.15.63:0/1660207521 wait complete. 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 Processor -- start 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 -- start start 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.707+0000 7fe880b2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635076e88f0 con 0x5635076dd400 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe87c7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe87c7d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40078/0 (socket says 172.21.15.63:40078) 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe87c7d1640 1 -- 172.21.15.63:0/3278316629 learned_addr learned my addr 172.21.15.63:0/3278316629 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe87c7d1640 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635077d8920 con 0x5635076dd400 2024-04-23T19:12:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d83e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe86c060040 tx=0x7fe86c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe768e09640 1 -- 172.21.15.63:0/3278316629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe86c057050 con 0x5635076dd400 2024-04-23T19:12:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe768e09640 1 -- 172.21.15.63:0/3278316629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c052ce0 con 0x5635076dd400 2024-04-23T19:12:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5635077d8bb0 con 0x5635076dd400 2024-04-23T19:12:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.708+0000 7fe768e09640 1 -- 172.21.15.63:0/3278316629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe86c05d470 con 0x5635076dd400 2024-04-23T19:12:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.709+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635077db860 con 0x5635076dd400 2024-04-23T19:12:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.709+0000 7fe768e09640 1 -- 172.21.15.63:0/3278316629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe86c05d610 con 0x5635076dd400 2024-04-23T19:12:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.709+0000 7fe768e09640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe764036eb0 0x7fe764039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.709+0000 7fe768e09640 1 -- 172.21.15.63:0/3278316629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe86c08e360 con 0x5635076dd400 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe87bfd0640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe764036eb0 0x7fe764039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5635076e03f0 0x5635077df620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5635077dfba0 con 0x5635076e03f0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5635076e03f0 0x5635077df620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe87bfd0640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe764036eb0 0x7fe764039370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe870000f60 tx=0x7fe8700012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5635076e03f0 0x5635077df620 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe87cfd2640 1 -- 172.21.15.63:0/3278316629 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe87400a4a0 con 0x5635076e03f0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.710+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5635077dcbf0 con 0x5635076e03f0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.711+0000 7fe87cfd2640 1 -- 172.21.15.63:0/3278316629 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe87400a4a0 con 0x5635076e03f0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.711+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5635077e19f0 con 0x5635076e03f0 2024-04-23T19:12:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.711+0000 7fe87cfd2640 1 -- 172.21.15.63:0/3278316629 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe87400a4a0 con 0x5635076e03f0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.711+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5635077e0d30 con 0x5635076e03f0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.711+0000 7fe87cfd2640 1 -- 172.21.15.63:0/3278316629 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe87400a4a0 con 0x5635076e03f0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.711+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5635077e2da0 0x5635077e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.711+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5635077e5710 con 0x5635077e2da0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.712+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5635077e2da0 0x5635077e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.712+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5635077e2da0 0x5635077e5190 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.712+0000 7fe87c7d1640 1 -- 172.21.15.63:0/3278316629 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe86c078070 con 0x5635077e2da0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe880b2f800 1 Processor -- start 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe880b2f800 1 -- start start 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe880b2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635077f3ac0 0x5635077f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe880b2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635075d0e30 con 0x5635077f3ac0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe87cfd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635077f3ac0 0x5635077f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe87cfd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635077f3ac0 0x5635077f3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40084/0 (socket says 172.21.15.63:40084) 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe87cfd2640 1 -- 172.21.15.63:0/4153803914 learned_addr learned my addr 172.21.15.63:0/4153803914 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.713+0000 7fe87cfd2640 1 -- 172.21.15.63:0/4153803914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635077f4420 con 0x5635077f3ac0 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.714+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635077f3ac0 0x5635077f3ee0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x5635077dff60 tx=0x7fe8740178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.714+0000 7fe761ffb640 1 -- 172.21.15.63:0/4153803914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe874013440 con 0x5635077f3ac0 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.714+0000 7fe761ffb640 1 -- 172.21.15.63:0/4153803914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8740135e0 con 0x5635077f3ac0 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.714+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5635077f4e90 con 0x5635077f3ac0 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.714+0000 7fe761ffb640 1 -- 172.21.15.63:0/4153803914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8740138c0 con 0x5635077f3ac0 2024-04-23T19:12:12.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.714+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635077f5390 con 0x5635077f3ac0 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe761ffb640 1 -- 172.21.15.63:0/4153803914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe874011040 con 0x5635077f3ac0 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe761ffb640 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe754036eb0 0x7fe754039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe761ffb640 1 -- 172.21.15.63:0/4153803914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe87401d070 con 0x5635077f3ac0 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe754036eb0 0x7fe754039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe880b2f800 1 Processor -- start 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe880b2f800 1 -- start start 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe754036eb0 0x7fe754039370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe86c05efd0 tx=0x7fe86c05cf70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe880b2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507801230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.715+0000 7fe880b2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635077d1150 con 0x563507800e10 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe87cfd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507801230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe87cfd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507801230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40088/0 (socket says 172.21.15.63:40088) 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe87cfd2640 1 -- 172.21.15.63:0/1467321764 learned_addr learned my addr 172.21.15.63:0/1467321764 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe87cfd2640 1 -- 172.21.15.63:0/1467321764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563507801800 con 0x563507800e10 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/1467321764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507801230 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe87404cc30 tx=0x7fe8740129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a65029df18b510d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe75b7fe640 1 -- 172.21.15.63:0/1467321764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8740123a0 con 0x563507800e10 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe75b7fe640 1 -- 172.21.15.63:0/1467321764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87404a020 con 0x563507800e10 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.716+0000 7fe75b7fe640 1 -- 172.21.15.63:0/1467321764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe874001310 con 0x563507800e10 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 -- 172.21.15.63:0/1467321764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 msgr2=0x563507801230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 --2- 172.21.15.63:0/1467321764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507801230 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe87404cc30 tx=0x7fe8740129d0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 -- 172.21.15.63:0/1467321764 shutdown_connections 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 --2- 172.21.15.63:0/1467321764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507801230 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 -- 172.21.15.63:0/1467321764 >> 172.21.15.63:0/1467321764 conn(0x5635077fd0e0 msgr2=0x5635077fb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 -- 172.21.15.63:0/1467321764 shutdown_connections 2024-04-23T19:12:12.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 -- 172.21.15.63:0/1467321764 wait complete. 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 Processor -- start 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 -- start start 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507808310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe880b2f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563507801f20 con 0x563507800e10 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe87cfd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507808310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe87cfd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507808310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40100/0 (socket says 172.21.15.63:40100) 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.717+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 learned_addr learned my addr 172.21.15.63:0/2453048264 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.718+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563507808850 con 0x563507800e10 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.718+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507808310 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe874012a10 tx=0x7fe87404cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.718+0000 7fe759ffb640 1 -- 172.21.15.63:0/2453048264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe874001610 con 0x563507800e10 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.718+0000 7fe759ffb640 1 -- 172.21.15.63:0/2453048264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87402ac70 con 0x563507800e10 2024-04-23T19:12:12.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.718+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563507808ae0 con 0x563507800e10 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.718+0000 7fe759ffb640 1 -- 172.21.15.63:0/2453048264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe87402a350 con 0x563507800e10 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.718+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350780b790 con 0x563507800e10 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe759ffb640 1 -- 172.21.15.63:0/2453048264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe874045040 con 0x563507800e10 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe759ffb640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe74c036eb0 0x7fe74c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe759ffb640 1 -- 172.21.15.63:0/2453048264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe874043020 con 0x563507800e10 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe74c036eb0 0x7fe74c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563507813f80 0x563507816440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563507816b90 con 0x563507813f80 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe87bfd0640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563507813f80 0x563507816440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.719+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe74c036eb0 0x7fe74c039370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe86c08c760 tx=0x7fe86c084000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe87bfd0640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563507813f80 0x563507816440 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe870009050 con 0x563507813f80 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 20 realm 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563507813420 con 0x563507813f80 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe870009050 con 0x563507813f80 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563507817670 con 0x563507813f80 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe870009050 con 0x563507813f80 2024-04-23T19:12:12.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.720+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563507817f40 con 0x563507813f80 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe870009050 con 0x563507813f80 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe880b2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5635078182e0 con 0x563507813f80 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe870009050 con 0x563507813f80 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe880b2f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe880b2f800 20 searching for the correct realm 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56350781a1c0 0x56350781c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56350781a1c0 0x56350781c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.721+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56350781a1c0 0x56350781c5b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87cfd2640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe87402a4f0 0x7fe874047430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe87402a4f0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe87402a4f0 0x7fe874047430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe87402a4f0 0x7fe874047430 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87c7d1640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe86c09b040 0x7fe86c05b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.722+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.723+0000 7fe87bfd0640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe86c09b040 0x7fe86c05b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.723+0000 7fe87bfd0640 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe86c09b040 0x7fe86c05b370 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.723+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe870009050 con 0x7fe86c09b040 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.723+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.723+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe870009050 con 0x563507813f80 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.723+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe87402a4f0 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.724+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.725+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe87402a4f0 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.726+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.726+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.726+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.726+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.726+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.726+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.727+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.728+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.728+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.728+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.728+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.728+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe87402a4f0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe87402a4f0 2024-04-23T19:12:12.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.729+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.730+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.730+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe87402a4f0 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.730+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.730+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x56350781a1c0 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.730+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.730+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe86c09b040 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.731+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.731+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x563507813f80 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.731+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.731+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x56350781cc60 con 0x7fe87402a4f0 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563507818c90 con 0x563507813f80 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563507819530 con 0x563507813f80 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.732+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5635078188d0 con 0x7fe86c09b040 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 zone default found 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 4 Realm: () 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 zonegroup default 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56350781d0c0 con 0x7fe87402a4f0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56350781f2f0 con 0x56350781a1c0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56350781f690 con 0x56350781a1c0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 started sync module instance, tier type = 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.733+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350781e730 con 0x7fe87402a4f0 2024-04-23T19:12:12.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.735+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.735+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94785759210272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350781ef40 con 0x7fe87402a4f0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.735+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507820500 con 0x563507813f80 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.736+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94785759210272] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.736+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.736+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94785759215696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507820920 con 0x563507813f80 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.736+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5635078212e0 con 0x56350781a1c0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.737+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94785759215696] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.737+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.737+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94785759216752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507820890 con 0x56350781a1c0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.737+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507822150 con 0x7fe87402a4f0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.738+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94785759216752] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.738+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.738+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94785759222944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507822540 con 0x7fe87402a4f0 2024-04-23T19:12:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.738+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507822f60 con 0x563507813f80 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.739+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.739+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94785759226448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507823800 con 0x563507813f80 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.739+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5635078242e0 con 0x7fe86c09b040 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.740+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94785759222944] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.741+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94785759226448] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.741+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.741+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94785759231472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507823280 con 0x7fe86c09b040 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.741+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507824b80 con 0x7fe86c09b040 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.742+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(56 notify.6 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.742+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94785759231472] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.742+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94785759233744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507823280 con 0x7fe86c09b040 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.742+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507825500 con 0x7fe86c09b040 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94785759233744] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94785759236176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507823280 con 0x7fe86c09b040 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 20 add_watcher() i=0 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 20 add_watcher() i=1 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 20 add_watcher() i=2 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 20 add_watcher() i=3 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 20 add_watcher() i=4 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 20 add_watcher() i=5 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.743+0000 7fe880b2f800 20 add_watcher() i=6 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.744+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94785759236176] v32'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.744+0000 7fe880b2f800 20 add_watcher() i=7 2024-04-23T19:12:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.744+0000 7fe880b2f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.744+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563507828610 con 0x563507813f80 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.744+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.744+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94785759234800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563507825a90 con 0x563507813f80 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.745+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94785759234800] v32'237 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.745+0000 7fe7327e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.745+0000 7fe880b2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.745+0000 7fe880b2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.745+0000 7fe880b2f800 5 note: GC not initialized 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe7317e2640 20 reqs_thread_entry: start 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe880b2f800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe880b2f800 20 Filter name: none 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe6a2ffd640 20 reqs_thread_entry: start 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe880b2f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5f0 con 0x563507813f80 2024-04-23T19:12:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.746+0000 7fe880b2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:could not fetch user info: no user info saved 2024-04-23T19:12:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.747+0000 7fe7327e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.747+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94785759234800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x563507813f80 2024-04-23T19:12:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.748+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94785759234800] v32'238 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.748+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94785759210272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x7fe87402a4f0 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.749+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94785759210272] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.749+0000 7fe880b2f800 20 remove_watcher() i=0 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.749+0000 7fe880b2f800 2 removed watcher, disabling cache 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.749+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94785759215696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x563507813f80 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.750+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94785759215696] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.750+0000 7fe880b2f800 20 remove_watcher() i=1 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.750+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94785759216752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x56350781a1c0 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.751+0000 7fe87cfd2640 1 -- 172.21.15.63:0/2453048264 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94785759216752] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87404cd70 con 0x56350781a1c0 2024-04-23T19:12:12.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.751+0000 7fe880b2f800 20 remove_watcher() i=2 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.751+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94785759222944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x7fe87402a4f0 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.752+0000 7fe87c7d1640 1 -- 172.21.15.63:0/2453048264 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94785759222944] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe86c078070 con 0x7fe87402a4f0 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.752+0000 7fe880b2f800 20 remove_watcher() i=3 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.752+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94785759226448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x563507813f80 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.753+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94785759226448] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x563507813f80 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.753+0000 7fe880b2f800 20 remove_watcher() i=4 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.753+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94785759231472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x7fe86c09b040 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.754+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94785759231472] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.755+0000 7fe880b2f800 20 remove_watcher() i=5 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.755+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94785759233744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x7fe86c09b040 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.756+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94785759233744] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.756+0000 7fe880b2f800 20 remove_watcher() i=6 2024-04-23T19:12:12.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.756+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94785759236176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56350785b5b0 con 0x7fe86c09b040 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94785759236176] v32'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe87002a040 con 0x7fe86c09b040 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 20 remove_watcher() i=7 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe86c09b040 msgr2=0x7fe86c05b370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe86c09b040 0x7fe86c05b370 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563507813f80 msgr2=0x563507816440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563507813f80 0x563507816440 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe87402a4f0 msgr2=0x7fe874047430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe87402a4f0 0x7fe874047430 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56350781a1c0 msgr2=0x56350781c5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x56350781a1c0 0x56350781c5b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe74c036eb0 msgr2=0x7fe74c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe74c036eb0 0x7fe74c039370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe86c08c760 tx=0x7fe86c084000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 msgr2=0x563507808310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507808310 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe874012a10 tx=0x7fe87404cbc0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe87bfd0640 1 -- 172.21.15.63:0/2453048264 reap_dead start 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 shutdown_connections 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/2453048264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563507800e10 0x563507808310 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 >> 172.21.15.63:0/2453048264 conn(0x5635077fd0e0 msgr2=0x5635077fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 shutdown_connections 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/2453048264 wait complete. 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe754036eb0 msgr2=0x7fe754039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe754036eb0 0x7fe754039370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe86c05efd0 tx=0x7fe86c05cf70 comp rx=0 tx=0).stop 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635077f3ac0 msgr2=0x5635077f3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635077f3ac0 0x5635077f3ee0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x5635077dff60 tx=0x7fe8740178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 shutdown_connections 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe754036eb0 0x7fe754039370 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 --2- 172.21.15.63:0/4153803914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635077f3ac0 0x5635077f3ee0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 >> 172.21.15.63:0/4153803914 conn(0x5635077e7b60 msgr2=0x5635077e7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 shutdown_connections 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.757+0000 7fe880b2f800 1 -- 172.21.15.63:0/4153803914 wait complete. 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5635077e2da0 msgr2=0x5635077e5190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5635077e2da0 0x5635077e5190 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5635076e03f0 msgr2=0x5635077df620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5635076e03f0 0x5635077df620 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe764036eb0 msgr2=0x7fe764039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe764036eb0 0x7fe764039370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe870000f60 tx=0x7fe8700012b0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 msgr2=0x5635077d83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d83e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe86c060040 tx=0x7fe86c002fb0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 shutdown_connections 2024-04-23T19:12:12.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5635077e2da0 0x5635077e5190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5635076e03f0 0x5635077df620 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe764036eb0 0x7fe764039370 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 --2- 172.21.15.63:0/3278316629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635076dd400 0x5635077d83e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 >> 172.21.15.63:0/3278316629 conn(0x5635076d8db0 msgr2=0x5635077ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 shutdown_connections 2024-04-23T19:12:12.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.758+0000 7fe880b2f800 1 -- 172.21.15.63:0/3278316629 wait complete. 2024-04-23T19:12:12.767 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T19:12:12.768 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-04-23T19:12:12.768 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-04-23T19:12:12.768 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-04-23T19:12:12.811 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:12.811 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.811+0000 7f1b42e40800 1 Processor -- start 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.811+0000 7f1b42e40800 1 -- start start 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b42e40800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5650e0d10570 0x5650e0d12960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b42e40800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5650e0bbf7f0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b42e40800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5650e0baee20 con 0x5650e0d10570 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b3d2db640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5650e0d10570 0x5650e0d12960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b3dadc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5650e0d0c5c0 0x5650e0d0c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56650/0 (socket says 172.21.15.63:56650) 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b3dadc640 1 -- 172.21.15.63:0/2893108373 learned_addr learned my addr 172.21.15.63:0/2893108373 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:12.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b3cada640 1 -- 172.21.15.63:0/2893108373 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659951632 0 0) 0x5650e0bbf7f0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b3cada640 1 -- 172.21.15.63:0/2893108373 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b2c001e10 con 0x5650e0d0c5c0 2024-04-23T19:12:12.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b3d2db640 1 -- 172.21.15.63:0/2893108373 >> v1:172.21.15.16:6789/0 conn(0x5650e0d0c5c0 legacy=0x5650e0d0c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.812+0000 7f1b3d2db640 1 -- 172.21.15.63:0/2893108373 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0bcdb50 con 0x5650e0d10570 2024-04-23T19:12:12.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.814+0000 7f1b3d2db640 1 --2- 172.21.15.63:0/2893108373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d10570 0x5650e0d12960 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800a470 tx=0x7f1b2801cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3a15810dcf0d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.814+0000 7f1b3cada640 1 -- 172.21.15.63:0/2893108373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28057070 con 0x5650e0d10570 2024-04-23T19:12:12.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.814+0000 7f1b3cada640 1 -- 172.21.15.63:0/2893108373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b28052570 con 0x5650e0d10570 2024-04-23T19:12:12.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.814+0000 7f1b3cada640 1 -- 172.21.15.63:0/2893108373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28052870 con 0x5650e0d10570 2024-04-23T19:12:12.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.815+0000 7f1b42e40800 1 -- 172.21.15.63:0/2893108373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d10570 msgr2=0x5650e0d12960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.815+0000 7f1b42e40800 1 --2- 172.21.15.63:0/2893108373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d10570 0x5650e0d12960 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800a470 tx=0x7f1b2801cdb0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.815+0000 7f1b42e40800 1 -- 172.21.15.63:0/2893108373 shutdown_connections 2024-04-23T19:12:12.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.815+0000 7f1b42e40800 1 --2- 172.21.15.63:0/2893108373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d10570 0x5650e0d12960 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.815+0000 7f1b42e40800 1 -- 172.21.15.63:0/2893108373 >> 172.21.15.63:0/2893108373 conn(0x5650e0d06e60 msgr2=0x5650e0d09280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.815+0000 7f1b42e40800 1 -- 172.21.15.63:0/2893108373 shutdown_connections 2024-04-23T19:12:12.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.815+0000 7f1b42e40800 1 -- 172.21.15.63:0/2893108373 wait complete. 2024-04-23T19:12:12.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.828+0000 7f1b42e40800 1 Processor -- start 2024-04-23T19:12:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.828+0000 7f1b42e40800 1 -- start start 2024-04-23T19:12:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.828+0000 7f1b42e40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0d0bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.828+0000 7f1b42e40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e0c586f0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.828+0000 7f1b3dadc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0d0bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.828+0000 7f1b3dadc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0d0bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40128/0 (socket says 172.21.15.63:40128) 2024-04-23T19:12:12.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.828+0000 7f1b3dadc640 1 -- 172.21.15.63:0/4277215205 learned_addr learned my addr 172.21.15.63:0/4277215205 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.829+0000 7f1b3dadc640 1 -- 172.21.15.63:0/4277215205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0e00fc0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.829+0000 7f1b3dadc640 1 --2- 172.21.15.63:0/4277215205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0d0bcc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1b30001650 tx=0x7f1b30001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b018b3558d1377 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.829+0000 7f1a2be07640 1 -- 172.21.15.63:0/4277215205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b30004520 con 0x5650e0d0c5c0 2024-04-23T19:12:12.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.829+0000 7f1a2be07640 1 -- 172.21.15.63:0/4277215205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b300046c0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.829+0000 7f1a2be07640 1 -- 172.21.15.63:0/4277215205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b300049a0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 -- 172.21.15.63:0/4277215205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 msgr2=0x5650e0d0bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 --2- 172.21.15.63:0/4277215205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0d0bcc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1b30001650 tx=0x7f1b30001b10 comp rx=0 tx=0).stop 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 -- 172.21.15.63:0/4277215205 shutdown_connections 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 --2- 172.21.15.63:0/4277215205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0d0bcc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 -- 172.21.15.63:0/4277215205 >> 172.21.15.63:0/4277215205 conn(0x5650e0d06e60 msgr2=0x5650e0df8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 -- 172.21.15.63:0/4277215205 shutdown_connections 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 -- 172.21.15.63:0/4277215205 wait complete. 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 Processor -- start 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 -- start start 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0e05d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b42e40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e0d13230 con 0x5650e0d0c5c0 2024-04-23T19:12:12.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b3dadc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0e05d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b3dadc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0e05d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40142/0 (socket says 172.21.15.63:40142) 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.830+0000 7f1b3dadc640 1 -- 172.21.15.63:0/57289546 learned_addr learned my addr 172.21.15.63:0/57289546 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.831+0000 7f1b3dadc640 1 -- 172.21.15.63:0/57289546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0e062c0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.831+0000 7f1b3dadc640 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0e05d80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b30000f80 tx=0x7f1b30001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.831+0000 7f1a2a604640 1 -- 172.21.15.63:0/57289546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b30022020 con 0x5650e0d0c5c0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.831+0000 7f1a2a604640 1 -- 172.21.15.63:0/57289546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b300148b0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.831+0000 7f1a2a604640 1 -- 172.21.15.63:0/57289546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b30014bd0 con 0x5650e0d0c5c0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.831+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5650e0e06550 con 0x5650e0d0c5c0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.831+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e0e09200 con 0x5650e0d0c5c0 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b42e40800 1 Processor -- start 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b42e40800 1 -- start start 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b42e40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e13a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b42e40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e0bfee30 con 0x5650e0e13650 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1a2a604640 1 -- 172.21.15.63:0/57289546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1b30025020 con 0x5650e0d0c5c0 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b3e2dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e13a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1a2a604640 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a24036eb0 0x7f1a24039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b3e2dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e13a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40154/0 (socket says 172.21.15.63:40154) 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b3e2dd640 1 -- 172.21.15.63:0/612596917 learned_addr learned my addr 172.21.15.63:0/612596917 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1a2a604640 1 -- 172.21.15.63:0/57289546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f1b30018030 con 0x5650e0d0c5c0 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b3dadc640 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a24036eb0 0x7f1a24039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.832+0000 7f1b3e2dd640 1 -- 172.21.15.63:0/612596917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0e13fb0 con 0x5650e0e13650 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b3e2dd640 1 --2- 172.21.15.63:0/612596917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e13a70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800a400 tx=0x7f1b3800a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257aa30516df5830 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b3dadc640 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a24036eb0 0x7f1a24039370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1b30024ff0 tx=0x7f1b30011f40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1a23fff640 1 -- 172.21.15.63:0/612596917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b380087e0 con 0x5650e0e13650 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1a23fff640 1 -- 172.21.15.63:0/612596917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b380089a0 con 0x5650e0e13650 2024-04-23T19:12:12.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1a23fff640 1 -- 172.21.15.63:0/612596917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38008d00 con 0x5650e0e13650 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b42e40800 1 -- 172.21.15.63:0/612596917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 msgr2=0x5650e0e13a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b42e40800 1 --2- 172.21.15.63:0/612596917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e13a70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800a400 tx=0x7f1b3800a8c0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b42e40800 1 -- 172.21.15.63:0/612596917 shutdown_connections 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b42e40800 1 --2- 172.21.15.63:0/612596917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e13a70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b42e40800 1 -- 172.21.15.63:0/612596917 >> 172.21.15.63:0/612596917 conn(0x5650e0d10570 msgr2=0x5650e0e100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b42e40800 1 -- 172.21.15.63:0/612596917 shutdown_connections 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.833+0000 7f1b42e40800 1 -- 172.21.15.63:0/612596917 wait complete. 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b42e40800 1 Processor -- start 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b42e40800 1 -- start start 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b42e40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b42e40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e0e14940 con 0x5650e0e13650 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b3e2dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b3e2dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e1ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40168/0 (socket says 172.21.15.63:40168) 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b3e2dd640 1 -- 172.21.15.63:0/669665067 learned_addr learned my addr 172.21.15.63:0/669665067 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.834+0000 7f1b3e2dd640 1 -- 172.21.15.63:0/669665067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0e1b080 con 0x5650e0e13650 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.835+0000 7f1b3e2dd640 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e1ab40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5650e0e14400 tx=0x7f1b38002ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.835+0000 7f1a227fc640 1 -- 172.21.15.63:0/669665067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38022020 con 0x5650e0e13650 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.835+0000 7f1a227fc640 1 -- 172.21.15.63:0/669665067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38019040 con 0x5650e0e13650 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.835+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5650e0e1b310 con 0x5650e0e13650 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.835+0000 7f1a227fc640 1 -- 172.21.15.63:0/669665067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38013950 con 0x5650e0e13650 2024-04-23T19:12:12.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.835+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e0e1dfc0 con 0x5650e0e13650 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1a227fc640 1 -- 172.21.15.63:0/669665067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1b38025020 con 0x5650e0e13650 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1a227fc640 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a18036e60 0x7f1a18039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1a227fc640 1 -- 172.21.15.63:0/669665067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f1b3804c9e0 con 0x5650e0e13650 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1b42e40800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1b42e40800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1b3d2db640 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a18036e60 0x7f1a18039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1b42e40800 1 Processor -- start 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1b3d2db640 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a18036e60 0x7f1a18039320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1b28054000 tx=0x7f1b2804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.836+0000 7f1b42e40800 1 -- start start 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b42e40800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0dfeb70 0x5650e0dfcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b42e40800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e0e01310 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b3dadc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0dfeb70 0x5650e0dfcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b3dadc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0dfeb70 0x5650e0dfcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40178/0 (socket says 172.21.15.63:40178) 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b3dadc640 1 -- 172.21.15.63:0/385820314 learned_addr learned my addr 172.21.15.63:0/385820314 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b3dadc640 1 -- 172.21.15.63:0/385820314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0dfd520 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b3dadc640 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0dfeb70 0x5650e0dfcfe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1b300044c0 tx=0x7f1b3004b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1a027e4640 1 -- 172.21.15.63:0/385820314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3000d280 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1a027e4640 1 -- 172.21.15.63:0/385820314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b30041070 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5650e0dfd7b0 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1a027e4640 1 -- 172.21.15.63:0/385820314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3000da90 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.837+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e0dfdcb0 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.838+0000 7f1a027e4640 1 -- 172.21.15.63:0/385820314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1b30010070 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.838+0000 7f1a027e4640 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1980036eb0 0x7f1980039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.838+0000 7f1a027e4640 1 -- 172.21.15.63:0/385820314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f1b30047d80 con 0x5650e0dfeb70 2024-04-23T19:12:12.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.838+0000 7f1b3d2db640 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1980036eb0 0x7f1980039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1973fff640 20 reqs_thread_entry: start 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1b3d2db640 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1980036eb0 0x7f1980039370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b2801cdf0 tx=0x7f1b28005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1b42e40800 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5650e0e44e40 0x5650e0e472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5650e0e479a0 con 0x5650e0e44e40 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1b3e2dd640 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5650e0e44e40 0x5650e0e472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1b3e2dd640 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5650e0e44e40 0x5650e0e472d0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1b3e2dd640 1 -- 172.21.15.63:0/57289546 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b3801e990 con 0x5650e0e44e40 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.839+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5650e0e492b0 con 0x5650e0e44e40 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.840+0000 7f1b3e2dd640 1 -- 172.21.15.63:0/57289546 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f1b3801e990 con 0x5650e0e44e40 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.840+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a18036e60 msgr2=0x7f1a18039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.840+0000 7f1b42e40800 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a18036e60 0x7f1a18039320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1b28054000 tx=0x7f1b2804e000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.840+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 msgr2=0x5650e0e1ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.840+0000 7f1b42e40800 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e1ab40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5650e0e14400 tx=0x7f1b38002ff0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 shutdown_connections 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a18036e60 0x7f1a18039320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/669665067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0e13650 0x5650e0e1ab40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 >> 172.21.15.63:0/669665067 conn(0x5650e0d10570 msgr2=0x5650e0e0d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 shutdown_connections 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/669665067 wait complete. 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1980036eb0 msgr2=0x7f1980039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1980036eb0 0x7f1980039370 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b2801cdf0 tx=0x7f1b28005a00 comp rx=0 tx=0).stop 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0dfeb70 msgr2=0x5650e0dfcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0dfeb70 0x5650e0dfcfe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1b300044c0 tx=0x7f1b3004b340 comp rx=0 tx=0).stop 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 shutdown_connections 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1980036eb0 0x7f1980039370 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/385820314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0dfeb70 0x5650e0dfcfe0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 >> 172.21.15.63:0/385820314 conn(0x5650e0e29cf0 msgr2=0x5650e0e2a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 shutdown_connections 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/385820314 wait complete. 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5650e0e44e40 msgr2=0x5650e0e472d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5650e0e44e40 0x5650e0e472d0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a24036eb0 msgr2=0x7f1a24039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a24036eb0 0x7f1a24039370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1b30024ff0 tx=0x7f1b30011f40 comp rx=0 tx=0).stop 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 msgr2=0x5650e0e05d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0e05d80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b30000f80 tx=0x7f1b30001250 comp rx=0 tx=0).stop 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 shutdown_connections 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5650e0e44e40 0x5650e0e472d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1a24036eb0 0x7f1a24039370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 --2- 172.21.15.63:0/57289546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e0d0c5c0 0x5650e0e05d80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 >> 172.21.15.63:0/57289546 conn(0x5650e0d06e60 msgr2=0x5650e0df7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 shutdown_connections 2024-04-23T19:12:12.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.841+0000 7f1b42e40800 1 -- 172.21.15.63:0/57289546 wait complete. 2024-04-23T19:12:12.852 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-23T19:12:12.852 DEBUG:tasks.util.rgw: json result: {'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-23T19:12:12.852 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-04-23T19:12:12.852 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-04-23T19:12:12.852 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-04-23T19:12:12.934 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:12.935 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc44db2d800 1 Processor -- start 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc44db2d800 1 -- start start 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc44db2d800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55cabbe28710 0x55cabbe2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc44db2d800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55cabbcd67f0 con 0x55cabbe24c00 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc44db2d800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55cabbcc5e20 con 0x55cabbe28710 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc448f7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55cabbe28710 0x55cabbe2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc44977f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55cabbe24c00 0x55cabbe26ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56664/0 (socket says 172.21.15.63:56664) 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc44977f640 1 -- 172.21.15.63:0/2207857329 learned_addr learned my addr 172.21.15.63:0/2207857329 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc43bfff640 1 -- 172.21.15.63:0/2207857329 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1968488737 0 0) 0x55cabbcd67f0 con 0x55cabbe24c00 2024-04-23T19:12:12.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.935+0000 7fc43bfff640 1 -- 172.21.15.63:0/2207857329 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc434001e10 con 0x55cabbe24c00 2024-04-23T19:12:12.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.936+0000 7fc448f7e640 1 -- 172.21.15.63:0/2207857329 >> v1:172.21.15.16:6789/0 conn(0x55cabbe24c00 legacy=0x55cabbe26ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.936+0000 7fc448f7e640 1 -- 172.21.15.63:0/2207857329 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabbce4b50 con 0x55cabbe28710 2024-04-23T19:12:12.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc448f7e640 1 --2- 172.21.15.63:0/2207857329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe28710 0x55cabbe2ab00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc44000a470 tx=0x7fc44001cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a4c4b02a49ae48 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc43bfff640 1 -- 172.21.15.63:0/2207857329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc440057070 con 0x55cabbe28710 2024-04-23T19:12:12.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc43bfff640 1 -- 172.21.15.63:0/2207857329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440052570 con 0x55cabbe28710 2024-04-23T19:12:12.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc43bfff640 1 -- 172.21.15.63:0/2207857329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc440052870 con 0x55cabbe28710 2024-04-23T19:12:12.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc44db2d800 1 -- 172.21.15.63:0/2207857329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe28710 msgr2=0x55cabbe2ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc44db2d800 1 --2- 172.21.15.63:0/2207857329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe28710 0x55cabbe2ab00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc44000a470 tx=0x7fc44001cdb0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc44db2d800 1 -- 172.21.15.63:0/2207857329 shutdown_connections 2024-04-23T19:12:12.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc44db2d800 1 --2- 172.21.15.63:0/2207857329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe28710 0x55cabbe2ab00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc44db2d800 1 -- 172.21.15.63:0/2207857329 >> 172.21.15.63:0/2207857329 conn(0x55cabbe1e5f0 msgr2=0x55cabbe20a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc44db2d800 1 -- 172.21.15.63:0/2207857329 shutdown_connections 2024-04-23T19:12:12.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.938+0000 7fc44db2d800 1 -- 172.21.15.63:0/2207857329 wait complete. 2024-04-23T19:12:12.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.951+0000 7fc44db2d800 1 Processor -- start 2024-04-23T19:12:12.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.951+0000 7fc44db2d800 1 -- start start 2024-04-23T19:12:12.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.951+0000 7fc44db2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbe24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.951+0000 7fc44db2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cabbd6f6e0 con 0x55cabbe24c00 2024-04-23T19:12:12.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc44977f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbe24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc44977f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbe24540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40210/0 (socket says 172.21.15.63:40210) 2024-04-23T19:12:12.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc44977f640 1 -- 172.21.15.63:0/1849490167 learned_addr learned my addr 172.21.15.63:0/1849490167 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc44977f640 1 -- 172.21.15.63:0/1849490167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabbe22b00 con 0x55cabbe24c00 2024-04-23T19:12:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc44977f640 1 --2- 172.21.15.63:0/1849490167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbe24540 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc43c001de0 tx=0x7fc43c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f442f5aedf4f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc337606640 1 -- 172.21.15.63:0/1849490167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c012020 con 0x55cabbe24c00 2024-04-23T19:12:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc337606640 1 -- 172.21.15.63:0/1849490167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c00e5a0 con 0x55cabbe24c00 2024-04-23T19:12:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.952+0000 7fc337606640 1 -- 172.21.15.63:0/1849490167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c00e900 con 0x55cabbe24c00 2024-04-23T19:12:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 -- 172.21.15.63:0/1849490167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 msgr2=0x55cabbe24540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 --2- 172.21.15.63:0/1849490167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbe24540 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc43c001de0 tx=0x7fc43c00b6c0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 -- 172.21.15.63:0/1849490167 shutdown_connections 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 --2- 172.21.15.63:0/1849490167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbe24540 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 -- 172.21.15.63:0/1849490167 >> 172.21.15.63:0/1849490167 conn(0x55cabbe1e5f0 msgr2=0x55cabbf10a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 -- 172.21.15.63:0/1849490167 shutdown_connections 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 -- 172.21.15.63:0/1849490167 wait complete. 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.953+0000 7fc44db2d800 1 Processor -- start 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44db2d800 1 -- start start 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44db2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbf1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44db2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cabbe2d350 con 0x55cabbe24c00 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44977f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbf1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44977f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbf1eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40214/0 (socket says 172.21.15.63:40214) 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 learned_addr learned my addr 172.21.15.63:0/474426955 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabbf1f3f0 con 0x55cabbe24c00 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc44977f640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbf1eeb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c010 tx=0x7fc43c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc335e03640 1 -- 172.21.15.63:0/474426955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c012030 con 0x55cabbe24c00 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.954+0000 7fc335e03640 1 -- 172.21.15.63:0/474426955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c01a7a0 con 0x55cabbe24c00 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.955+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cabbf1f680 con 0x55cabbe24c00 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.955+0000 7fc335e03640 1 -- 172.21.15.63:0/474426955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c01a950 con 0x55cabbe24c00 2024-04-23T19:12:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.955+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabbf22330 con 0x55cabbe24c00 2024-04-23T19:12:12.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.955+0000 7fc44db2d800 1 Processor -- start 2024-04-23T19:12:12.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.955+0000 7fc335e03640 1 -- 172.21.15.63:0/474426955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc43c015020 con 0x55cabbe24c00 2024-04-23T19:12:12.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.955+0000 7fc44db2d800 1 -- start start 2024-04-23T19:12:12.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc44db2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc44db2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cabbd15e30 con 0x55cabbf2c760 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc335e03640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc330036eb0 0x7fc330039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc335e03640 1 -- 172.21.15.63:0/474426955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fc43c020030 con 0x55cabbe24c00 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc44977f640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc330036eb0 0x7fc330039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc449f80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc449f80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf2cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40230/0 (socket says 172.21.15.63:40230) 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc449f80640 1 -- 172.21.15.63:0/3606855313 learned_addr learned my addr 172.21.15.63:0/3606855313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc44977f640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc330036eb0 0x7fc330039370 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02dfd0 tx=0x7fc43c00dee0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc449f80640 1 -- 172.21.15.63:0/3606855313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabbf2d0c0 con 0x55cabbf2c760 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.956+0000 7fc449f80640 1 --2- 172.21.15.63:0/3606855313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf2cb80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc44400a430 tx=0x7fc44400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4debf6dfc58b11a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc32f7fe640 1 -- 172.21.15.63:0/3606855313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc444015070 con 0x55cabbf2c760 2024-04-23T19:12:12.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc32f7fe640 1 -- 172.21.15.63:0/3606855313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc444001670 con 0x55cabbf2c760 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc32f7fe640 1 -- 172.21.15.63:0/3606855313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc444001990 con 0x55cabbf2c760 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc44db2d800 1 -- 172.21.15.63:0/3606855313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 msgr2=0x55cabbf2cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc44db2d800 1 --2- 172.21.15.63:0/3606855313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf2cb80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc44400a430 tx=0x7fc44400a8f0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc44db2d800 1 -- 172.21.15.63:0/3606855313 shutdown_connections 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc44db2d800 1 --2- 172.21.15.63:0/3606855313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf2cb80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc44db2d800 1 -- 172.21.15.63:0/3606855313 >> 172.21.15.63:0/3606855313 conn(0x55cabbe28710 msgr2=0x55cabbf29200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc44db2d800 1 -- 172.21.15.63:0/3606855313 shutdown_connections 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.957+0000 7fc44db2d800 1 -- 172.21.15.63:0/3606855313 wait complete. 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc44db2d800 1 Processor -- start 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc44db2d800 1 -- start start 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc44db2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf33c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc44db2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cabbf2da50 con 0x55cabbf2c760 2024-04-23T19:12:12.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc449f80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf33c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc449f80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf33c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40240/0 (socket says 172.21.15.63:40240) 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc449f80640 1 -- 172.21.15.63:0/3438110937 learned_addr learned my addr 172.21.15.63:0/3438110937 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.958+0000 7fc449f80640 1 -- 172.21.15.63:0/3438110937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabbf34160 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.959+0000 7fc449f80640 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf33c20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc444013040 tx=0x7fc444003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.959+0000 7fc32dffb640 1 -- 172.21.15.63:0/3438110937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc444020020 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.959+0000 7fc32dffb640 1 -- 172.21.15.63:0/3438110937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4440108e0 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.959+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cabbf343f0 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.959+0000 7fc32dffb640 1 -- 172.21.15.63:0/3438110937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc444015040 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.959+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabbf370a0 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc32dffb640 1 -- 172.21.15.63:0/3438110937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc444023020 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc32dffb640 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc320036e60 0x7fc320039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc32dffb640 1 -- 172.21.15.63:0/3438110937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fc44404c7c0 con 0x55cabbf2c760 2024-04-23T19:12:12.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc448f7e640 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc320036e60 0x7fc320039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc44db2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc44db2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc44db2d800 1 Processor -- start 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc44db2d800 1 -- start start 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc448f7e640 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc320036e60 0x7fc320039320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc440054000 tx=0x7fc44004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.960+0000 7fc44db2d800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf4e2a0 0x55cabbf4e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc44db2d800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cabbe22e80 con 0x55cabbf4e2a0 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc44977f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf4e2a0 0x55cabbf4e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc44977f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf4e2a0 0x55cabbf4e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40254/0 (socket says 172.21.15.63:40254) 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc44977f640 1 -- 172.21.15.63:0/10555605 learned_addr learned my addr 172.21.15.63:0/10555605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc44977f640 1 -- 172.21.15.63:0/10555605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabbf4ec00 con 0x55cabbf4e2a0 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc44977f640 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf4e2a0 0x55cabbf4e6c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00bc10 tx=0x7fc43c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc30efdd640 1 -- 172.21.15.63:0/10555605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c049070 con 0x55cabbf4e2a0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc30efdd640 1 -- 172.21.15.63:0/10555605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c016530 con 0x55cabbf4e2a0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cabbf4f670 con 0x55cabbf4e2a0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.961+0000 7fc30efdd640 1 -- 172.21.15.63:0/10555605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c017da0 con 0x55cabbf4e2a0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.962+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabbf4fb70 con 0x55cabbf4e2a0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.962+0000 7fc30efdd640 1 -- 172.21.15.63:0/10555605 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc43c00d7a0 con 0x55cabbf4e2a0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.962+0000 7fc30efdd640 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc280036eb0 0x7fc280039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.962+0000 7fc30efdd640 1 -- 172.21.15.63:0/10555605 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fc43c051020 con 0x55cabbf4e2a0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.962+0000 7fc448f7e640 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc280036eb0 0x7fc280039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.963+0000 7fc30cfd9640 20 reqs_thread_entry: start 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.963+0000 7fc448f7e640 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc280036eb0 0x7fc280039370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc44001cdf0 tx=0x7fc440005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.963+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cabbf5bca0 0x55cabbf5e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.963+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf5e7a0 con 0x55cabbf5bca0 2024-04-23T19:12:12.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.963+0000 7fc449f80640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cabbf5bca0 0x55cabbf5e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.963+0000 7fc449f80640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cabbf5bca0 0x55cabbf5e0f0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.963+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.964+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf5f080 con 0x55cabbf5bca0 2024-04-23T19:12:12.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.964+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.964+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf60430 con 0x55cabbf5bca0 2024-04-23T19:12:12.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.964+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.964+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf6d360 con 0x55cabbf5bca0 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:NOTICE: set zone's realm_id= 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cabbf64760 0x55cabbf67690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x55cabbf64760 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc44977f640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cabbf64760 0x55cabbf67690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc44977f640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cabbf64760 0x55cabbf67690 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(5 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc44977f640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc43c017050 0x7fc43c04c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x7fc43c017050 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.965+0000 7fc448f7e640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc43c017050 0x7fc43c04c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.966+0000 7fc448f7e640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc43c017050 0x7fc43c04c490 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.966+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc440066030 con 0x7fc43c017050 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.966+0000 7fc448f7e640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc44006a040 0x7fc44006b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.966+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x7fc44006a040 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.966+0000 7fc449f80640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc44006a040 0x7fc44006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.967+0000 7fc449f80640 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc44006a040 0x7fc44006b940 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.967+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.967+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x55cabbf5bca0 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.967+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.967+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x55cabbf5bca0 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.967+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.967+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x7fc44006a040 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(10 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x7fc44006a040 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(11 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x7fc43c017050 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc440066030 con 0x7fc43c017050 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x55cabbf64760 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.968+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x55cabbf5bca0 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x7fc44006a040 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x55cabbf5bca0 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf67210 con 0x7fc43c017050 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(17 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc440066030 con 0x7fc43c017050 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.969+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf64760 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf64760 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc44006a040 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf64760 2024-04-23T19:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.970+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf64760 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf5bca0 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf5bca0 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=74b] v23'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc44006a040 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(25 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.971+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf64760 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc43c017050 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc440066030 con 0x7fc43c017050 2024-04-23T19:12:12.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf5bca0 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(28 [pgnls start_epoch 32 out=107b] v30'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc43c017050 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(29 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc440066030 con 0x7fc43c017050 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.972+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf64760 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.973+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.973+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc44006a040 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.973+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.973+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc43c017050 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.973+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc440066030 con 0x7fc43c017050 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.973+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf64760 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.973+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc43c042070 con 0x55cabbf64760 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.974+0000 7fc44977f640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc44006a040 2024-04-23T19:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.974+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc44400e070 con 0x7fc44006a040 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.974+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.974+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(35 [pgnls start_epoch 32 out=115b] v23'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.974+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fc440078060 con 0x7fc43c017050 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.974+0000 7fc448f7e640 1 -- 172.21.15.63:0/474426955 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(36 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc440066030 con 0x7fc43c017050 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.974+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf6b270 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.975+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.975+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf607d0 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.975+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.975+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [create,call version.set in=58b,writefull 0~1155 in=1155b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf66aa0 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.976+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [create,call,writefull 0~1155] v32'3 uv3 ondisk = 0) ==== 274+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.976+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf66aa0 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.977+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v32'3 uv3 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.977+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55cabbf66aa0 con 0x55cabbf5bca0 2024-04-23T19:12:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.978+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v32'2 uv1 ondisk = -17 ((17) File exists)) ==== 199+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x55cabbf5bca0 2024-04-23T19:12:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc320036e60 msgr2=0x7fc320039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc320036e60 0x7fc320039320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc440054000 tx=0x7fc44004e000 comp rx=0 tx=0).stop 2024-04-23T19:12:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 msgr2=0x55cabbf33c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf33c20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc444013040 tx=0x7fc444003370 comp rx=0 tx=0).stop 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 shutdown_connections 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc320036e60 0x7fc320039320 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 --2- 172.21.15.63:0/3438110937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf2c760 0x55cabbf33c20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 >> 172.21.15.63:0/3438110937 conn(0x55cabbe28710 msgr2=0x55cabbf26840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 shutdown_connections 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.979+0000 7fc44db2d800 1 -- 172.21.15.63:0/3438110937 wait complete. 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc280036eb0 msgr2=0x7fc280039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc280036eb0 0x7fc280039370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc44001cdf0 tx=0x7fc440005a00 comp rx=0 tx=0).stop 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf4e2a0 msgr2=0x55cabbf4e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf4e2a0 0x55cabbf4e6c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00bc10 tx=0x7fc43c00c910 comp rx=0 tx=0).stop 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 shutdown_connections 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc280036eb0 0x7fc280039370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/10555605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbf4e2a0 0x55cabbf4e6c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 >> 172.21.15.63:0/10555605 conn(0x55cabbf42d50 msgr2=0x55cabbf43770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 shutdown_connections 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/10555605 wait complete. 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc44006a040 msgr2=0x7fc44006b940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fc44006a040 0x7fc44006b940 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cabbf5bca0 msgr2=0x55cabbf5e0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55cabbf5bca0 0x55cabbf5e0f0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc43c017050 msgr2=0x7fc43c04c490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fc43c017050 0x7fc43c04c490 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cabbf64760 msgr2=0x55cabbf67690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55cabbf64760 0x55cabbf67690 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc330036eb0 msgr2=0x7fc330039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc330036eb0 0x7fc330039370 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02dfd0 tx=0x7fc43c00dee0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 msgr2=0x55cabbf1eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbf1eeb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c010 tx=0x7fc43c000ed0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc449f80640 1 -- 172.21.15.63:0/474426955 reap_dead start 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 shutdown_connections 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 --2- 172.21.15.63:0/474426955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cabbe24c00 0x55cabbf1eeb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 >> 172.21.15.63:0/474426955 conn(0x55cabbe1e5f0 msgr2=0x55cabbe233e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 shutdown_connections 2024-04-23T19:12:12.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:12.980+0000 7fc44db2d800 1 -- 172.21.15.63:0/474426955 wait complete. 2024-04-23T19:12:12.992 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-23T19:12:12.992 DEBUG:tasks.util.rgw: json result: {'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-23T19:12:12.992 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-04-23T19:12:12.993 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-04-23T19:12:12.993 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-04-23T19:12:13.074 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:13.074 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:13.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.074+0000 7f8cf9f2a800 1 Processor -- start 2024-04-23T19:12:13.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.074+0000 7f8cf9f2a800 1 -- start start 2024-04-23T19:12:13.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf9f2a800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5566e3e53ff0 0x5566e3e54410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf9f2a800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5566e3d077f0 con 0x5566e3e55250 2024-04-23T19:12:13.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf9f2a800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5566e3cf6e20 con 0x5566e3e53ff0 2024-04-23T19:12:13.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf5bf2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5566e3e53ff0 0x5566e3e54410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf53f1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5566e3e55250 0x5566e3e556d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56668/0 (socket says 172.21.15.63:56668) 2024-04-23T19:12:13.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf53f1640 1 -- 172.21.15.63:0/1348607615 learned_addr learned my addr 172.21.15.63:0/1348607615 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:13.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/1348607615 >> v1:172.21.15.16:6789/0 conn(0x5566e3e55250 legacy=0x5566e3e556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/1348607615 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e3d15b50 con 0x5566e3e53ff0 2024-04-23T19:12:13.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.075+0000 7f8cf4bf0640 1 -- 172.21.15.63:0/1348607615 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 135551549 0 0) 0x5566e3d077f0 con 0x5566e3e55250 2024-04-23T19:12:13.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.077+0000 7f8cf5bf2640 1 --2- 172.21.15.63:0/1348607615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3e54410 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800a600 tx=0x7f8ce801ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54999ed110e933f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.077+0000 7f8cf4bf0640 1 -- 172.21.15.63:0/1348607615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce800a200 con 0x5566e3e53ff0 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.077+0000 7f8cf4bf0640 1 -- 172.21.15.63:0/1348607615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce800a3a0 con 0x5566e3e53ff0 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.077+0000 7f8cf4bf0640 1 -- 172.21.15.63:0/1348607615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce80086e0 con 0x5566e3e53ff0 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.078+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/1348607615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 msgr2=0x5566e3e54410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.078+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/1348607615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3e54410 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800a600 tx=0x7f8ce801ed70 comp rx=0 tx=0).stop 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.078+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/1348607615 shutdown_connections 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.078+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/1348607615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3e54410 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.078+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/1348607615 >> 172.21.15.63:0/1348607615 conn(0x5566e3e4f5f0 msgr2=0x5566e3e51a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.078+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/1348607615 shutdown_connections 2024-04-23T19:12:13.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.078+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/1348607615 wait complete. 2024-04-23T19:12:13.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.091+0000 7f8cf9f2a800 1 Processor -- start 2024-04-23T19:12:13.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.091+0000 7f8cf9f2a800 1 -- start start 2024-04-23T19:12:13.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.091+0000 7f8cf9f2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f48840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.091+0000 7f8cf9f2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566e3da06e0 con 0x5566e3e53ff0 2024-04-23T19:12:13.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.091+0000 7f8cf5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f48840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.091+0000 7f8cf5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f48840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40280/0 (socket says 172.21.15.63:40280) 2024-04-23T19:12:13.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.091+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/4012091775 learned_addr learned my addr 172.21.15.63:0/4012091775 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.092+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/4012091775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e3f48e10 con 0x5566e3e53ff0 2024-04-23T19:12:13.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.092+0000 7f8cf5bf2640 1 --2- 172.21.15.63:0/4012091775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f48840 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8000c00 tx=0x7f8ce804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fd02d1f395f790 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.092+0000 7f8be3e07640 1 -- 172.21.15.63:0/4012091775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce8007520 con 0x5566e3e53ff0 2024-04-23T19:12:13.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.092+0000 7f8be3e07640 1 -- 172.21.15.63:0/4012091775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce8063070 con 0x5566e3e53ff0 2024-04-23T19:12:13.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.092+0000 7f8be3e07640 1 -- 172.21.15.63:0/4012091775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce8007830 con 0x5566e3e53ff0 2024-04-23T19:12:13.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4012091775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 msgr2=0x5566e3f48840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4012091775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f48840 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8000c00 tx=0x7f8ce804e000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4012091775 shutdown_connections 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4012091775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f48840 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4012091775 >> 172.21.15.63:0/4012091775 conn(0x5566e3e4f5f0 msgr2=0x5566e3f3fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4012091775 shutdown_connections 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4012091775 wait complete. 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 Processor -- start 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 -- start start 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f50290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.093+0000 7f8cf9f2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566e3dc8060 con 0x5566e3e53ff0 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8cf5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f50290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8cf5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f50290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40294/0 (socket says 172.21.15.63:40294) 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/471002993 learned_addr learned my addr 172.21.15.63:0/471002993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/471002993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e3f507d0 con 0x5566e3e53ff0 2024-04-23T19:12:13.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8cf5bf2640 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f50290 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8060040 tx=0x7f8ce8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8be2604640 1 -- 172.21.15.63:0/471002993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce806c070 con 0x5566e3e53ff0 2024-04-23T19:12:13.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8be2604640 1 -- 172.21.15.63:0/471002993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce8008ba0 con 0x5566e3e53ff0 2024-04-23T19:12:13.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5566e3f50a60 con 0x5566e3e53ff0 2024-04-23T19:12:13.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8be2604640 1 -- 172.21.15.63:0/471002993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce8071460 con 0x5566e3e53ff0 2024-04-23T19:12:13.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.094+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e3f53710 con 0x5566e3e53ff0 2024-04-23T19:12:13.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf9f2a800 1 Processor -- start 2024-04-23T19:12:13.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf9f2a800 1 -- start start 2024-04-23T19:12:13.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8be2604640 1 -- 172.21.15.63:0/471002993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8ce8071600 con 0x5566e3e53ff0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf9f2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f5def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf9f2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566e3d46e30 con 0x5566e3f5dad0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8be2604640 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bdc036eb0 0x7f8bdc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8be2604640 1 -- 172.21.15.63:0/471002993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f8ce804ce00 con 0x5566e3e53ff0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f5def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f5def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40304/0 (socket says 172.21.15.63:40304) 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf63f3640 1 -- 172.21.15.63:0/4154805239 learned_addr learned my addr 172.21.15.63:0/4154805239 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.095+0000 7f8cf5bf2640 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bdc036eb0 0x7f8bdc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.096+0000 7f8cf63f3640 1 -- 172.21.15.63:0/4154805239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e3f5e430 con 0x5566e3f5dad0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.096+0000 7f8cf5bf2640 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bdc036eb0 0x7f8bdc039370 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ce805efd0 tx=0x7f8ce807d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.096+0000 7f8cf63f3640 1 --2- 172.21.15.63:0/4154805239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f5def0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000a430 tx=0x7f8cf000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27b73aa23e220ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.096+0000 7f8bdbfff640 1 -- 172.21.15.63:0/4154805239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf0015070 con 0x5566e3f5dad0 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.096+0000 7f8bdbfff640 1 -- 172.21.15.63:0/4154805239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf0001670 con 0x5566e3f5dad0 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.096+0000 7f8bdbfff640 1 -- 172.21.15.63:0/4154805239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf0001990 con 0x5566e3f5dad0 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4154805239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 msgr2=0x5566e3f5def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4154805239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f5def0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000a430 tx=0x7f8cf000a8f0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4154805239 shutdown_connections 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4154805239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f5def0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4154805239 >> 172.21.15.63:0/4154805239 conn(0x5566e3e55250 msgr2=0x5566e3f5a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4154805239 shutdown_connections 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4154805239 wait complete. 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 Processor -- start 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 -- start start 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f64f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf9f2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566e3f5edc0 con 0x5566e3f5dad0 2024-04-23T19:12:13.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f64f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.097+0000 7f8cf63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f64f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40320/0 (socket says 172.21.15.63:40320) 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8cf63f3640 1 -- 172.21.15.63:0/2110911536 learned_addr learned my addr 172.21.15.63:0/2110911536 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8cf63f3640 1 -- 172.21.15.63:0/2110911536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e3f654b0 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8cf63f3640 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f64f70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0013040 tx=0x7f8cf0003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8bda7fc640 1 -- 172.21.15.63:0/2110911536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf0020020 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8bda7fc640 1 -- 172.21.15.63:0/2110911536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf00108e0 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5566e3f65740 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8bda7fc640 1 -- 172.21.15.63:0/2110911536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf0015040 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.098+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e3f683f0 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.099+0000 7f8bda7fc640 1 -- 172.21.15.63:0/2110911536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8cf0023020 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.099+0000 7f8bda7fc640 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bd0036e60 0x7f8bd0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.099+0000 7f8bda7fc640 1 -- 172.21.15.63:0/2110911536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f8cf004c880 con 0x5566e3f5dad0 2024-04-23T19:12:13.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.099+0000 7f8cf53f1640 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bd0036e60 0x7f8bd0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.099+0000 7f8cf9f2a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.099+0000 7f8cf9f2a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf53f1640 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bd0036e60 0x7f8bd0039320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec001330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf9f2a800 1 Processor -- start 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf9f2a800 1 -- start start 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf9f2a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f7f5b0 0x5566e3f7f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf9f2a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566e3f490d0 con 0x5566e3f7f5b0 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f7f5b0 0x5566e3f7f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f7f5b0 0x5566e3f7f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40334/0 (socket says 172.21.15.63:40334) 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.100+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/4188479279 learned_addr learned my addr 172.21.15.63:0/4188479279 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.101+0000 7f8cf5bf2640 1 -- 172.21.15.63:0/4188479279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e3f7ff10 con 0x5566e3f7f5b0 2024-04-23T19:12:13.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.101+0000 7f8cf5bf2640 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f7f5b0 0x5566e3f7f9d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8053560 tx=0x7f8ce805dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.101+0000 7f8bba7e4640 1 -- 172.21.15.63:0/4188479279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce8093020 con 0x5566e3f7f5b0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.101+0000 7f8bba7e4640 1 -- 172.21.15.63:0/4188479279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce806e310 con 0x5566e3f7f5b0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.101+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5566e3f80980 con 0x5566e3f7f5b0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.101+0000 7f8bba7e4640 1 -- 172.21.15.63:0/4188479279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce805a2a0 con 0x5566e3f7f5b0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.101+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e3f80e80 con 0x5566e3f7f5b0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8bba7e4640 1 -- 172.21.15.63:0/4188479279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8ce8096020 con 0x5566e3f7f5b0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8bba7e4640 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b38036e10 0x7f8b380392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8bba7e4640 1 -- 172.21.15.63:0/4188479279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f8ce808c030 con 0x5566e3f7f5b0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8cf53f1640 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b38036e10 0x7f8b380392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8b2bfff640 20 reqs_thread_entry: start 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8cf53f1640 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b38036e10 0x7f8b380392d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000f60 tx=0x7f8cec006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5566e3f8cfb0 0x5566e3f8f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5566e3f8fa50 con 0x5566e3f8cfb0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.102+0000 7f8cf63f3640 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5566e3f8cfb0 0x5566e3f8f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.103+0000 7f8cf63f3640 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5566e3f8cfb0 0x5566e3f8f3a0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.103+0000 7f8cf63f3640 1 -- 172.21.15.63:0/471002993 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8cf000e070 con 0x5566e3f8cfb0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.103+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5566e3f90380 con 0x5566e3f8cfb0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.103+0000 7f8cf63f3640 1 -- 172.21.15.63:0/471002993 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8cf000e070 con 0x5566e3f8cfb0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.103+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5566e3f91730 con 0x5566e3f8cfb0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf63f3640 1 -- 172.21.15.63:0/471002993 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~1155 out=1155b] v0'0 uv3 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7f8cf000e070 con 0x5566e3f8cfb0 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bd0036e60 msgr2=0x7f8bd0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bd0036e60 0x7f8bd0039320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec001330 comp rx=0 tx=0).stop 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 msgr2=0x5566e3f64f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f64f70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0013040 tx=0x7f8cf0003370 comp rx=0 tx=0).stop 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 shutdown_connections 2024-04-23T19:12:13.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bd0036e60 0x7f8bd0039320 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/2110911536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f5dad0 0x5566e3f64f70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 >> 172.21.15.63:0/2110911536 conn(0x5566e3e55250 msgr2=0x5566e3f57bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 shutdown_connections 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.104+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/2110911536 wait complete. 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b38036e10 msgr2=0x7f8b380392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b38036e10 0x7f8b380392d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000f60 tx=0x7f8cec006000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f7f5b0 msgr2=0x5566e3f7f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f7f5b0 0x5566e3f7f9d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8053560 tx=0x7f8ce805dfc0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 shutdown_connections 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b38036e10 0x7f8b380392d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/4188479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3f7f5b0 0x5566e3f7f9d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 >> 172.21.15.63:0/4188479279 conn(0x5566e3f74080 msgr2=0x5566e3f74aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 shutdown_connections 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/4188479279 wait complete. 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5566e3f8cfb0 msgr2=0x5566e3f8f3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5566e3f8cfb0 0x5566e3f8f3a0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bdc036eb0 msgr2=0x7f8bdc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bdc036eb0 0x7f8bdc039370 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ce805efd0 tx=0x7f8ce807d040 comp rx=0 tx=0).stop 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 msgr2=0x5566e3f50290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f50290 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8060040 tx=0x7f8ce8002fb0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 shutdown_connections 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5566e3f8cfb0 0x5566e3f8f3a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8bdc036eb0 0x7f8bdc039370 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 --2- 172.21.15.63:0/471002993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566e3e53ff0 0x5566e3f50290 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 >> 172.21.15.63:0/471002993 conn(0x5566e3e4f5f0 msgr2=0x5566e3f3e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 shutdown_connections 2024-04-23T19:12:13.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.105+0000 7f8cf9f2a800 1 -- 172.21.15.63:0/471002993 wait complete. 2024-04-23T19:12:13.116 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-23T19:12:13.116 DEBUG:tasks.util.rgw: json result: {'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-23T19:12:13.116 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-23T19:12:13.116 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-23T19:12:13.116 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-23T19:12:13.204 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:13.204 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.204+0000 7fd9aab19800 1 Processor -- start 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.204+0000 7fd9aab19800 1 -- start start 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.204+0000 7fd9aab19800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5567ca16a7f0 0x5567ca16cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.204+0000 7fd9aab19800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5567ca0187f0 con 0x5567ca166ce0 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.204+0000 7fd9aab19800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5567ca007e20 con 0x5567ca16a7f0 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.205+0000 7fd9a7e1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5567ca16a7f0 0x5567ca16cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.205+0000 7fd9a861f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5567ca166ce0 0x5567ca1690d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56680/0 (socket says 172.21.15.63:56680) 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.205+0000 7fd9a861f640 1 -- 172.21.15.63:0/4034772981 learned_addr learned my addr 172.21.15.63:0/4034772981 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.205+0000 7fd9a7e1e640 1 -- 172.21.15.63:0/4034772981 >> v1:172.21.15.16:6789/0 conn(0x5567ca166ce0 legacy=0x5567ca1690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.205+0000 7fd9a7e1e640 1 -- 172.21.15.63:0/4034772981 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ca026b50 con 0x5567ca16a7f0 2024-04-23T19:12:13.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.207+0000 7fd9a7e1e640 1 --2- 172.21.15.63:0/4034772981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca16a7f0 0x5567ca16cbe0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00a470 tx=0x7fd99c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5990f0b30f3b13b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.207+0000 7fd9a761d640 1 -- 172.21.15.63:0/4034772981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99c057070 con 0x5567ca16a7f0 2024-04-23T19:12:13.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.207+0000 7fd9a761d640 1 -- 172.21.15.63:0/4034772981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c052570 con 0x5567ca16a7f0 2024-04-23T19:12:13.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.207+0000 7fd9a761d640 1 -- 172.21.15.63:0/4034772981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99c052870 con 0x5567ca16a7f0 2024-04-23T19:12:13.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.208+0000 7fd9aab19800 1 -- 172.21.15.63:0/4034772981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca16a7f0 msgr2=0x5567ca16cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.208+0000 7fd9aab19800 1 --2- 172.21.15.63:0/4034772981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca16a7f0 0x5567ca16cbe0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00a470 tx=0x7fd99c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:13.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.208+0000 7fd9aab19800 1 -- 172.21.15.63:0/4034772981 shutdown_connections 2024-04-23T19:12:13.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.208+0000 7fd9aab19800 1 --2- 172.21.15.63:0/4034772981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca16a7f0 0x5567ca16cbe0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.208+0000 7fd9aab19800 1 -- 172.21.15.63:0/4034772981 >> 172.21.15.63:0/4034772981 conn(0x5567ca160690 msgr2=0x5567ca162ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.208+0000 7fd9aab19800 1 -- 172.21.15.63:0/4034772981 shutdown_connections 2024-04-23T19:12:13.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.208+0000 7fd9aab19800 1 -- 172.21.15.63:0/4034772981 wait complete. 2024-04-23T19:12:13.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.220+0000 7fd9aab19800 1 Processor -- start 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.220+0000 7fd9aab19800 1 -- start start 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.221+0000 7fd9aab19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.221+0000 7fd9aab19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567ca0b1710 con 0x5567ca166ce0 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.221+0000 7fd9a861f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.221+0000 7fd9a861f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca166620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40350/0 (socket says 172.21.15.63:40350) 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.221+0000 7fd9a861f640 1 -- 172.21.15.63:0/2630787328 learned_addr learned my addr 172.21.15.63:0/2630787328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.221+0000 7fd9a861f640 1 -- 172.21.15.63:0/2630787328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ca164be0 con 0x5567ca166ce0 2024-04-23T19:12:13.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.221+0000 7fd9a861f640 1 --2- 172.21.15.63:0/2630787328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca166620 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd998001de0 tx=0x7fd99800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3abfc26d1fbc39 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd89660c640 1 -- 172.21.15.63:0/2630787328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd998012020 con 0x5567ca166ce0 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd89660c640 1 -- 172.21.15.63:0/2630787328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99800e5a0 con 0x5567ca166ce0 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd89660c640 1 -- 172.21.15.63:0/2630787328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99800e8c0 con 0x5567ca166ce0 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd9aab19800 1 -- 172.21.15.63:0/2630787328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 msgr2=0x5567ca166620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd9aab19800 1 --2- 172.21.15.63:0/2630787328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca166620 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd998001de0 tx=0x7fd99800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd9aab19800 1 -- 172.21.15.63:0/2630787328 shutdown_connections 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd9aab19800 1 --2- 172.21.15.63:0/2630787328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca166620 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd9aab19800 1 -- 172.21.15.63:0/2630787328 >> 172.21.15.63:0/2630787328 conn(0x5567ca160690 msgr2=0x5567ca252df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd9aab19800 1 -- 172.21.15.63:0/2630787328 shutdown_connections 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.222+0000 7fd9aab19800 1 -- 172.21.15.63:0/2630787328 wait complete. 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.223+0000 7fd9aab19800 1 Processor -- start 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.223+0000 7fd9aab19800 1 -- start start 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.223+0000 7fd9aab19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca261210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.223+0000 7fd9aab19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567ca16f430 con 0x5567ca166ce0 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.223+0000 7fd9a861f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca261210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.223+0000 7fd9a861f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca261210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40352/0 (socket says 172.21.15.63:40352) 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.223+0000 7fd9a861f640 1 -- 172.21.15.63:0/667466491 learned_addr learned my addr 172.21.15.63:0/667466491 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd9a861f640 1 -- 172.21.15.63:0/667466491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ca261750 con 0x5567ca166ce0 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd9a861f640 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca261210 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c010 tx=0x7fd998000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd894e09640 1 -- 172.21.15.63:0/667466491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd998012070 con 0x5567ca166ce0 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd894e09640 1 -- 172.21.15.63:0/667466491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99801a7a0 con 0x5567ca166ce0 2024-04-23T19:12:13.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd894e09640 1 -- 172.21.15.63:0/667466491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99801aa80 con 0x5567ca166ce0 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567ca2619e0 con 0x5567ca166ce0 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567ca264690 con 0x5567ca166ce0 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.224+0000 7fd9aab19800 1 Processor -- start 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9aab19800 1 -- start start 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9aab19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca26eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9aab19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567ca057e30 con 0x5567ca26eac0 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd894e09640 1 -- 172.21.15.63:0/667466491 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd998015020 con 0x5567ca166ce0 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9a8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca26eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9a8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca26eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40366/0 (socket says 172.21.15.63:40366) 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9a8e20640 1 -- 172.21.15.63:0/447571329 learned_addr learned my addr 172.21.15.63:0/447571329 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd894e09640 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd890036eb0 0x7fd890039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd894e09640 1 -- 172.21.15.63:0/667466491 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fd998020070 con 0x5567ca166ce0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9a861f640 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd890036eb0 0x7fd890039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.225+0000 7fd9a8e20640 1 -- 172.21.15.63:0/447571329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ca26f420 con 0x5567ca26eac0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9a8e20640 1 --2- 172.21.15.63:0/447571329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca26eee0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000a430 tx=0x7fd9a000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec0ddfd155944fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9a861f640 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd890036eb0 0x7fd890039370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd99801a6f0 tx=0x7fd998032040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd88e7fc640 1 -- 172.21.15.63:0/447571329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0015070 con 0x5567ca26eac0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd88e7fc640 1 -- 172.21.15.63:0/447571329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a0001670 con 0x5567ca26eac0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd88e7fc640 1 -- 172.21.15.63:0/447571329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0001990 con 0x5567ca26eac0 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9aab19800 1 -- 172.21.15.63:0/447571329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 msgr2=0x5567ca26eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9aab19800 1 --2- 172.21.15.63:0/447571329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca26eee0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000a430 tx=0x7fd9a000a8f0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9aab19800 1 -- 172.21.15.63:0/447571329 shutdown_connections 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9aab19800 1 --2- 172.21.15.63:0/447571329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca26eee0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9aab19800 1 -- 172.21.15.63:0/447571329 >> 172.21.15.63:0/447571329 conn(0x5567ca16a7f0 msgr2=0x5567ca26b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9aab19800 1 -- 172.21.15.63:0/447571329 shutdown_connections 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.226+0000 7fd9aab19800 1 -- 172.21.15.63:0/447571329 wait complete. 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9aab19800 1 Processor -- start 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9aab19800 1 -- start start 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9aab19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca275f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9aab19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567ca26fdb0 con 0x5567ca26eac0 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9a8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca275f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9a8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca275f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40380/0 (socket says 172.21.15.63:40380) 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9a8e20640 1 -- 172.21.15.63:0/3598816908 learned_addr learned my addr 172.21.15.63:0/3598816908 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9a8e20640 1 -- 172.21.15.63:0/3598816908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ca276460 con 0x5567ca26eac0 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.227+0000 7fd9a8e20640 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca275f20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0013040 tx=0x7fd9a0003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.228+0000 7fd88cff9640 1 -- 172.21.15.63:0/3598816908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0020020 con 0x5567ca26eac0 2024-04-23T19:12:13.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.228+0000 7fd88cff9640 1 -- 172.21.15.63:0/3598816908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a00108e0 con 0x5567ca26eac0 2024-04-23T19:12:13.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.228+0000 7fd88cff9640 1 -- 172.21.15.63:0/3598816908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0015040 con 0x5567ca26eac0 2024-04-23T19:12:13.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.228+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567ca2766f0 con 0x5567ca26eac0 2024-04-23T19:12:13.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.228+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567ca2793a0 con 0x5567ca26eac0 2024-04-23T19:12:13.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd88cff9640 1 -- 172.21.15.63:0/3598816908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd9a0023020 con 0x5567ca26eac0 2024-04-23T19:12:13.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd88cff9640 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd880036e60 0x7fd880039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd9aab19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd9aab19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd88cff9640 1 -- 172.21.15.63:0/3598816908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fd9a004c880 con 0x5567ca26eac0 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd9a7e1e640 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd880036e60 0x7fd880039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd9aab19800 1 Processor -- start 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd9aab19800 1 -- start start 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.229+0000 7fd9a7e1e640 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd880036e60 0x7fd880039320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd99c054000 tx=0x7fd99c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.230+0000 7fd9aab19800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca290630 0x5567ca290a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.230+0000 7fd9aab19800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567ca164f60 con 0x5567ca290630 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.230+0000 7fd9a861f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca290630 0x5567ca290a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.230+0000 7fd9a861f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca290630 0x5567ca290a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40382/0 (socket says 172.21.15.63:40382) 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.230+0000 7fd9a861f640 1 -- 172.21.15.63:0/4200893380 learned_addr learned my addr 172.21.15.63:0/4200893380 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.230+0000 7fd9a861f640 1 -- 172.21.15.63:0/4200893380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ca290f90 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.230+0000 7fd9a861f640 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca290630 0x5567ca290a50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd99804b020 tx=0x7fd998001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.231+0000 7fd86cfe1640 1 -- 172.21.15.63:0/4200893380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99800d3d0 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.231+0000 7fd86cfe1640 1 -- 172.21.15.63:0/4200893380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99800d570 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.231+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567ca291a00 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.231+0000 7fd86cfe1640 1 -- 172.21.15.63:0/4200893380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99800d850 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.231+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567ca291f00 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.231+0000 7fd86cfe1640 1 -- 172.21.15.63:0/4200893380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd9980163b0 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd86cfe1640 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd7e0036eb0 0x7fd7e0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd86cfe1640 1 -- 172.21.15.63:0/4200893380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fd998047d10 con 0x5567ca290630 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd9a7e1e640 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd7e0036eb0 0x7fd7e0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd9a7e1e640 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd7e0036eb0 0x7fd7e0039370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd99c01ced0 tx=0x7fd99c005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd7fe7fc640 20 reqs_thread_entry: start 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd9aab19800 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5567ca29e080 0x5567ca2a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5567ca2a0bf0 con 0x5567ca29e080 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd9a8e20640 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5567ca29e080 0x5567ca2a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.232+0000 7fd9a8e20640 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5567ca29e080 0x5567ca2a0520 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.233+0000 7fd9a8e20640 1 -- 172.21.15.63:0/667466491 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd9a000e070 con 0x5567ca29e080 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.233+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5567ca2a24b0 con 0x5567ca29e080 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.233+0000 7fd9a8e20640 1 -- 172.21.15.63:0/667466491 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~1155 out=1155b] v0'0 uv3 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7fd9a000e070 con 0x5567ca29e080 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.233+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5567ca2a1e30 con 0x5567ca29e080 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.235+0000 7fd9a8e20640 1 -- 172.21.15.63:0/667466491 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [stat,call,call,writefull 0~993] v32'4 uv4 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7fd9a000e070 con 0x5567ca29e080 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.235+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd880036e60 msgr2=0x7fd880039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.235+0000 7fd9aab19800 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd880036e60 0x7fd880039320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd99c054000 tx=0x7fd99c04e000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.235+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 msgr2=0x5567ca275f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.235+0000 7fd9aab19800 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca275f20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0013040 tx=0x7fd9a0003370 comp rx=0 tx=0).stop 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 shutdown_connections 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd880036e60 0x7fd880039320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/3598816908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca26eac0 0x5567ca275f20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 >> 172.21.15.63:0/3598816908 conn(0x5567ca16a7f0 msgr2=0x5567ca268ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 shutdown_connections 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/3598816908 wait complete. 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd7e0036eb0 msgr2=0x7fd7e0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd7e0036eb0 0x7fd7e0039370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd99c01ced0 tx=0x7fd99c005a00 comp rx=0 tx=0).stop 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca290630 msgr2=0x5567ca290a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca290630 0x5567ca290a50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd99804b020 tx=0x7fd998001520 comp rx=0 tx=0).stop 2024-04-23T19:12:13.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 shutdown_connections 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd7e0036eb0 0x7fd7e0039370 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/4200893380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca290630 0x5567ca290a50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 >> 172.21.15.63:0/4200893380 conn(0x5567ca285100 msgr2=0x5567ca285b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 shutdown_connections 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/4200893380 wait complete. 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5567ca29e080 msgr2=0x5567ca2a0520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5567ca29e080 0x5567ca2a0520 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd890036eb0 msgr2=0x7fd890039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd890036eb0 0x7fd890039370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd99801a6f0 tx=0x7fd998032040 comp rx=0 tx=0).stop 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 msgr2=0x5567ca261210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca261210 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c010 tx=0x7fd998000ed0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 shutdown_connections 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5567ca29e080 0x5567ca2a0520 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd890036eb0 0x7fd890039370 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 --2- 172.21.15.63:0/667466491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567ca166ce0 0x5567ca261210 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 >> 172.21.15.63:0/667466491 conn(0x5567ca160690 msgr2=0x5567ca1654c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 shutdown_connections 2024-04-23T19:12:13.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.236+0000 7fd9aab19800 1 -- 172.21.15.63:0/667466491 wait complete. 2024-04-23T19:12:13.246 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-23T19:12:13.246 DEBUG:tasks.util.rgw: json result: {'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-23T19:12:13.246 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-04-23T19:12:13.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-04-23T19:12:13.247 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-04-23T19:12:13.288 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:13.288 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.288+0000 7f5f34da2800 1 Processor -- start 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.288+0000 7f5f34da2800 1 -- start start 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f34da2800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5570f9a9d690 0x5570f9a9fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f34da2800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5570f994b7f0 con 0x5570f9a99b80 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f34da2800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5570f993ae20 con 0x5570f9a9d690 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f320a0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5570f9a9d690 0x5570f9a9fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f328a1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5570f9a99b80 0x5570f9a9bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56696/0 (socket says 172.21.15.63:56696) 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f328a1640 1 -- 172.21.15.63:0/2367489519 learned_addr learned my addr 172.21.15.63:0/2367489519 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:13.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f320a0640 1 -- 172.21.15.63:0/2367489519 >> v1:172.21.15.16:6789/0 conn(0x5570f9a99b80 legacy=0x5570f9a9bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.289+0000 7f5f320a0640 1 -- 172.21.15.63:0/2367489519 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f9959b50 con 0x5570f9a9d690 2024-04-23T19:12:13.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.291+0000 7f5f320a0640 1 --2- 172.21.15.63:0/2367489519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a9d690 0x5570f9a9fa80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800a470 tx=0x7f5f2801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d7781f34c339c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.291+0000 7f5f3189f640 1 -- 172.21.15.63:0/2367489519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f28057070 con 0x5570f9a9d690 2024-04-23T19:12:13.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.291+0000 7f5f3189f640 1 -- 172.21.15.63:0/2367489519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f28052570 con 0x5570f9a9d690 2024-04-23T19:12:13.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.291+0000 7f5f3189f640 1 -- 172.21.15.63:0/2367489519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f28052870 con 0x5570f9a9d690 2024-04-23T19:12:13.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.292+0000 7f5f34da2800 1 -- 172.21.15.63:0/2367489519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a9d690 msgr2=0x5570f9a9fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.292+0000 7f5f34da2800 1 --2- 172.21.15.63:0/2367489519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a9d690 0x5570f9a9fa80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800a470 tx=0x7f5f2801ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:13.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.292+0000 7f5f34da2800 1 -- 172.21.15.63:0/2367489519 shutdown_connections 2024-04-23T19:12:13.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.292+0000 7f5f34da2800 1 --2- 172.21.15.63:0/2367489519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a9d690 0x5570f9a9fa80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.292+0000 7f5f34da2800 1 -- 172.21.15.63:0/2367489519 >> 172.21.15.63:0/2367489519 conn(0x5570f9a935f0 msgr2=0x5570f9a95a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.292+0000 7f5f34da2800 1 -- 172.21.15.63:0/2367489519 shutdown_connections 2024-04-23T19:12:13.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.292+0000 7f5f34da2800 1 -- 172.21.15.63:0/2367489519 wait complete. 2024-04-23T19:12:13.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.304+0000 7f5f34da2800 1 Processor -- start 2024-04-23T19:12:13.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.305+0000 7f5f34da2800 1 -- start start 2024-04-23T19:12:13.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.305+0000 7f5f34da2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b8a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.305+0000 7f5f34da2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f99e46e0 con 0x5570f9a99b80 2024-04-23T19:12:13.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.305+0000 7f5f328a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b8a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.305+0000 7f5f328a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b8a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40394/0 (socket says 172.21.15.63:40394) 2024-04-23T19:12:13.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.305+0000 7f5f328a1640 1 -- 172.21.15.63:0/1091183657 learned_addr learned my addr 172.21.15.63:0/1091183657 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.305+0000 7f5f328a1640 1 -- 172.21.15.63:0/1091183657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f9b8a940 con 0x5570f9a99b80 2024-04-23T19:12:13.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.306+0000 7f5f328a1640 1 --2- 172.21.15.63:0/1091183657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b8a370 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001de0 tx=0x7f5f2400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8f405c2d5bc884 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.306+0000 7f5e20e09640 1 -- 172.21.15.63:0/1091183657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f24012020 con 0x5570f9a99b80 2024-04-23T19:12:13.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.306+0000 7f5e20e09640 1 -- 172.21.15.63:0/1091183657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2400e5a0 con 0x5570f9a99b80 2024-04-23T19:12:13.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.306+0000 7f5e20e09640 1 -- 172.21.15.63:0/1091183657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2400e8c0 con 0x5570f9a99b80 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.306+0000 7f5f34da2800 1 -- 172.21.15.63:0/1091183657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 msgr2=0x5570f9b8a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.306+0000 7f5f34da2800 1 --2- 172.21.15.63:0/1091183657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b8a370 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001de0 tx=0x7f5f2400b6c0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 -- 172.21.15.63:0/1091183657 shutdown_connections 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 --2- 172.21.15.63:0/1091183657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b8a370 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 -- 172.21.15.63:0/1091183657 >> 172.21.15.63:0/1091183657 conn(0x5570f9a935f0 msgr2=0x5570f9b85c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 -- 172.21.15.63:0/1091183657 shutdown_connections 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 -- 172.21.15.63:0/1091183657 wait complete. 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 Processor -- start 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 -- start start 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f34da2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f9aa02c0 con 0x5570f9a99b80 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f328a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f328a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b91eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40396/0 (socket says 172.21.15.63:40396) 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.307+0000 7f5f328a1640 1 -- 172.21.15.63:0/403313950 learned_addr learned my addr 172.21.15.63:0/403313950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.308+0000 7f5f328a1640 1 -- 172.21.15.63:0/403313950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f9b923f0 con 0x5570f9a99b80 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.308+0000 7f5f328a1640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b91eb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c010 tx=0x7f5f24000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.308+0000 7f5e1f606640 1 -- 172.21.15.63:0/403313950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f24012070 con 0x5570f9a99b80 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.308+0000 7f5e1f606640 1 -- 172.21.15.63:0/403313950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2401a7a0 con 0x5570f9a99b80 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.308+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570f9b92680 con 0x5570f9a99b80 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.308+0000 7f5e1f606640 1 -- 172.21.15.63:0/403313950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2401aa80 con 0x5570f9a99b80 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.308+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f9b95330 con 0x5570f9a99b80 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f34da2800 1 Processor -- start 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f34da2800 1 -- start start 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f34da2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9b9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f34da2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f998ae30 con 0x5570f9b9f760 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5e1f606640 1 -- 172.21.15.63:0/403313950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5f24015020 con 0x5570f9a99b80 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f330a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9b9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f330a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9b9fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40406/0 (socket says 172.21.15.63:40406) 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f330a2640 1 -- 172.21.15.63:0/767671531 learned_addr learned my addr 172.21.15.63:0/767671531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5e1f606640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036eb0 0x7f5e18039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5e1f606640 1 -- 172.21.15.63:0/403313950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f5f24020070 con 0x5570f9a99b80 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f330a2640 1 -- 172.21.15.63:0/767671531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f9ba00c0 con 0x5570f9b9f760 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.309+0000 7f5f328a1640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036eb0 0x7f5e18039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f330a2640 1 --2- 172.21.15.63:0/767671531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9b9fb80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00a430 tx=0x7f5f2c00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dc89c43a3f719a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f328a1640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036eb0 0x7f5e18039370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2401a6f0 tx=0x7f5f24019c90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5e1d602640 1 -- 172.21.15.63:0/767671531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c015070 con 0x5570f9b9f760 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5e1d602640 1 -- 172.21.15.63:0/767671531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c001670 con 0x5570f9b9f760 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5e1d602640 1 -- 172.21.15.63:0/767671531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c001990 con 0x5570f9b9f760 2024-04-23T19:12:13.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f34da2800 1 -- 172.21.15.63:0/767671531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 msgr2=0x5570f9b9fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f34da2800 1 --2- 172.21.15.63:0/767671531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9b9fb80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00a430 tx=0x7f5f2c00a8f0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f34da2800 1 -- 172.21.15.63:0/767671531 shutdown_connections 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f34da2800 1 --2- 172.21.15.63:0/767671531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9b9fb80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f34da2800 1 -- 172.21.15.63:0/767671531 >> 172.21.15.63:0/767671531 conn(0x5570f9a9d690 msgr2=0x5570f9b9c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f34da2800 1 -- 172.21.15.63:0/767671531 shutdown_connections 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.310+0000 7f5f34da2800 1 -- 172.21.15.63:0/767671531 wait complete. 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.311+0000 7f5f34da2800 1 Processor -- start 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.311+0000 7f5f34da2800 1 -- start start 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.311+0000 7f5f34da2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9a994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.311+0000 7f5f34da2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f9ba0a50 con 0x5570f9b9f760 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.311+0000 7f5f330a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9a994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.311+0000 7f5f330a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9a994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40412/0 (socket says 172.21.15.63:40412) 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.311+0000 7f5f330a2640 1 -- 172.21.15.63:0/498156475 learned_addr learned my addr 172.21.15.63:0/498156475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.312+0000 7f5f330a2640 1 -- 172.21.15.63:0/498156475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f9a97b90 con 0x5570f9b9f760 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.312+0000 7f5f330a2640 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9a994a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c013040 tx=0x7f5f2c003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.312+0000 7f5e177fe640 1 -- 172.21.15.63:0/498156475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c020020 con 0x5570f9b9f760 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.312+0000 7f5e177fe640 1 -- 172.21.15.63:0/498156475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c0108e0 con 0x5570f9b9f760 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.312+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570f9a97e20 con 0x5570f9b9f760 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.312+0000 7f5e177fe640 1 -- 172.21.15.63:0/498156475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2c015040 con 0x5570f9b9f760 2024-04-23T19:12:13.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.312+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f9a982c0 con 0x5570f9b9f760 2024-04-23T19:12:13.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.313+0000 7f5e177fe640 1 -- 172.21.15.63:0/498156475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5f2c023020 con 0x5570f9b9f760 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.313+0000 7f5e177fe640 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036e60 0x7f5e0c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.313+0000 7f5e177fe640 1 -- 172.21.15.63:0/498156475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f5f2c04c7c0 con 0x5570f9b9f760 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.313+0000 7f5f320a0640 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036e60 0x7f5e0c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.313+0000 7f5f34da2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.313+0000 7f5f34da2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f320a0640 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036e60 0x7f5e0c039320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5f28054000 tx=0x7f5f2804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f34da2800 1 Processor -- start 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f34da2800 1 -- start start 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f34da2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9bc34c0 0x5570f9bc38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f34da2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f9b8acc0 con 0x5570f9bc34c0 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f328a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9bc34c0 0x5570f9bc38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f328a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9bc34c0 0x5570f9bc38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40420/0 (socket says 172.21.15.63:40420) 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.314+0000 7f5f328a1640 1 -- 172.21.15.63:0/563252938 learned_addr learned my addr 172.21.15.63:0/563252938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.315+0000 7f5f328a1640 1 -- 172.21.15.63:0/563252938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f9bc3e20 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.315+0000 7f5f328a1640 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9bc34c0 0x5570f9bc38e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000fd0 tx=0x7f5f2400d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.315+0000 7f5df77e6640 1 -- 172.21.15.63:0/563252938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f24016cb0 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.315+0000 7f5df77e6640 1 -- 172.21.15.63:0/563252938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f24016e50 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.315+0000 7f5df77e6640 1 -- 172.21.15.63:0/563252938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f24019600 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.315+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570f9bc4890 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.315+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f9bc4d90 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5df77e6640 1 -- 172.21.15.63:0/563252938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f5f2400d8d0 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5df77e6640 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d74036e60 0x7f5d74039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5df77e6640 1 -- 172.21.15.63:0/563252938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f5f24035070 con 0x5570f9bc34c0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5f320a0640 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d74036e60 0x7f5d74039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5df57e2640 20 reqs_thread_entry: start 2024-04-23T19:12:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5f320a0640 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d74036e60 0x7f5d74039320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f2801ced0 tx=0x7f5f28005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570f9bd0ec0 0x5570f9bd32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.316+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570f9bd39a0 con 0x5570f9bd0ec0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.317+0000 7f5f330a2640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570f9bd0ec0 0x5570f9bd32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.317+0000 7f5f330a2640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570f9bd0ec0 0x5570f9bd32f0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.317+0000 7f5f330a2640 1 -- 172.21.15.63:0/403313950 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c00e070 con 0x5570f9bd0ec0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.317+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570f9bd4280 con 0x5570f9bd0ec0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.317+0000 7f5f330a2640 1 -- 172.21.15.63:0/403313950 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5f2c00e070 con 0x5570f9bd0ec0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.317+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570f9bd5b90 0x5570f9bd8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.317+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:3 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5570f9bd8780 con 0x5570f9bd5b90 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.318+0000 7f5f328a1640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570f9bd5b90 0x5570f9bd8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.318+0000 7f5f328a1640 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570f9bd5b90 0x5570f9bd8020 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.318+0000 7f5f328a1640 1 -- 172.21.15.63:0/403313950 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(3 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5f24043020 con 0x5570f9bd5b90 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036e60 msgr2=0x7f5e0c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036e60 0x7f5e0c039320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5f28054000 tx=0x7f5f2804e000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 msgr2=0x5570f9a994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9a994a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c013040 tx=0x7f5f2c003370 comp rx=0 tx=0).stop 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 shutdown_connections 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e0c036e60 0x7f5e0c039320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 --2- 172.21.15.63:0/498156475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9b9f760 0x5570f9a994a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 >> 172.21.15.63:0/498156475 conn(0x5570f9a9d690 msgr2=0x5570f9b99840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 shutdown_connections 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.319+0000 7f5f34da2800 1 -- 172.21.15.63:0/498156475 wait complete. 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d74036e60 msgr2=0x7f5d74039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d74036e60 0x7f5d74039320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f2801ced0 tx=0x7f5f28005a00 comp rx=0 tx=0).stop 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9bc34c0 msgr2=0x5570f9bc38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9bc34c0 0x5570f9bc38e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000fd0 tx=0x7f5f2400d7a0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 shutdown_connections 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5d74036e60 0x7f5d74039320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/563252938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9bc34c0 0x5570f9bc38e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 >> 172.21.15.63:0/563252938 conn(0x5570f9bb7f70 msgr2=0x5570f9bb8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 shutdown_connections 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/563252938 wait complete. 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570f9bd5b90 msgr2=0x5570f9bd8020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570f9bd5b90 0x5570f9bd8020 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570f9bd0ec0 msgr2=0x5570f9bd32f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570f9bd0ec0 0x5570f9bd32f0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036eb0 msgr2=0x7f5e18039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036eb0 0x7f5e18039370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2401a6f0 tx=0x7f5f24019c90 comp rx=0 tx=0).stop 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 msgr2=0x5570f9b91eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b91eb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c010 tx=0x7f5f24000ed0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 shutdown_connections 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5570f9bd5b90 0x5570f9bd8020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5570f9bd0ec0 0x5570f9bd32f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f5e18036eb0 0x7f5e18039370 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 --2- 172.21.15.63:0/403313950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f9a99b80 0x5570f9b91eb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 >> 172.21.15.63:0/403313950 conn(0x5570f9a935f0 msgr2=0x5570f9b84490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 shutdown_connections 2024-04-23T19:12:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.320+0000 7f5f34da2800 1 -- 172.21.15.63:0/403313950 wait complete. 2024-04-23T19:12:13.330 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"8815d3af-8342-460f-8220-31ec38570cc3","name":"default","api_name":"default","is_master":true,"endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038","zones":[{"id":"bc28bc2a-1a44-4b88-9ab6-dfcb32cea038","name":"default","endpoints":[],"log_meta":false,"log_data":false,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]} 2024-04-23T19:12:13.330 DEBUG:tasks.util.rgw: json result: {'id': '8815d3af-8342-460f-8220-31ec38570cc3', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038', 'zones': [{'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['notification_v2', 'resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['notification_v2', 'resharding']} 2024-04-23T19:12:13.333 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2024-04-23T19:12:13.352 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-23T19:12:13.352 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-23T19:12:13.352 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2024-04-23T19:12:13.424 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:13.425 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:13.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c4383f800 1 Processor -- start 2024-04-23T19:12:13.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c4383f800 1 -- start start 2024-04-23T19:12:13.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c4383f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55de61fb1120 0x55de61fb1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c4383f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55de61e637f0 con 0x55de61fb22e0 2024-04-23T19:12:13.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c4383f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55de61e52e20 con 0x55de61fb1120 2024-04-23T19:12:13.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c4133e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55de61fb1120 0x55de61fb1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c40b3d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55de61fb22e0 0x55de61fb2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56702/0 (socket says 172.21.15.63:56702) 2024-04-23T19:12:13.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.425+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1792815574 learned_addr learned my addr 172.21.15.63:0/1792815574 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:13.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.426+0000 7f0c4133e640 1 -- 172.21.15.63:0/1792815574 >> v1:172.21.15.16:6789/0 conn(0x55de61fb22e0 legacy=0x55de61fb2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.426+0000 7f0c33fff640 1 -- 172.21.15.63:0/1792815574 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906455669 0 0) 0x55de61e637f0 con 0x55de61fb22e0 2024-04-23T19:12:13.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.426+0000 7f0c4133e640 1 -- 172.21.15.63:0/1792815574 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de61e71b50 con 0x55de61fb1120 2024-04-23T19:12:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4133e640 1 --2- 172.21.15.63:0/1792815574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61fb1540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400a470 tx=0x7f0c3401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c6d9db0cf97613 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c33fff640 1 -- 172.21.15.63:0/1792815574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34057070 con 0x55de61fb1120 2024-04-23T19:12:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c33fff640 1 -- 172.21.15.63:0/1792815574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c34052570 con 0x55de61fb1120 2024-04-23T19:12:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c33fff640 1 -- 172.21.15.63:0/1792815574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34052870 con 0x55de61fb1120 2024-04-23T19:12:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4383f800 1 -- 172.21.15.63:0/1792815574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 msgr2=0x55de61fb1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1792815574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61fb1540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400a470 tx=0x7f0c3401ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4383f800 1 -- 172.21.15.63:0/1792815574 shutdown_connections 2024-04-23T19:12:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1792815574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61fb1540 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4383f800 1 -- 172.21.15.63:0/1792815574 >> 172.21.15.63:0/1792815574 conn(0x55de61fac780 msgr2=0x55de61faeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4383f800 1 -- 172.21.15.63:0/1792815574 shutdown_connections 2024-04-23T19:12:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.428+0000 7f0c4383f800 1 -- 172.21.15.63:0/1792815574 wait complete. 2024-04-23T19:12:13.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.441+0000 7f0c4383f800 1 Processor -- start 2024-04-23T19:12:13.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.441+0000 7f0c4383f800 1 -- start start 2024-04-23T19:12:13.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.441+0000 7f0c4383f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61f26a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.441+0000 7f0c4383f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de61efc690 con 0x55de61fb1120 2024-04-23T19:12:13.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.442+0000 7f0c4133e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61f26a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.442+0000 7f0c4133e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61f26a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40436/0 (socket says 172.21.15.63:40436) 2024-04-23T19:12:13.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.442+0000 7f0c4133e640 1 -- 172.21.15.63:0/1969957178 learned_addr learned my addr 172.21.15.63:0/1969957178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.442+0000 7f0c4133e640 1 -- 172.21.15.63:0/1969957178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de61f24ff0 con 0x55de61fb1120 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.442+0000 7f0c4133e640 1 --2- 172.21.15.63:0/1969957178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61f26a30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0c34055040 tx=0x7f0c3404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f1e95b6459b9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0b2f606640 1 -- 172.21.15.63:0/1969957178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34057040 con 0x55de61fb1120 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0b2f606640 1 -- 172.21.15.63:0/1969957178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c34052990 con 0x55de61fb1120 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0b2f606640 1 -- 172.21.15.63:0/1969957178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34052c70 con 0x55de61fb1120 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0c4383f800 1 -- 172.21.15.63:0/1969957178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 msgr2=0x55de61f26a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1969957178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61f26a30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0c34055040 tx=0x7f0c3404c000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0c4383f800 1 -- 172.21.15.63:0/1969957178 shutdown_connections 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1969957178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de61f26a30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0c4383f800 1 -- 172.21.15.63:0/1969957178 >> 172.21.15.63:0/1969957178 conn(0x55de61fac780 msgr2=0x55de6209e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0c4383f800 1 -- 172.21.15.63:0/1969957178 shutdown_connections 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.443+0000 7f0c4383f800 1 -- 172.21.15.63:0/1969957178 wait complete. 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.444+0000 7f0c4383f800 1 Processor -- start 2024-04-23T19:12:13.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.444+0000 7f0c4383f800 1 -- start start 2024-04-23T19:12:13.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.444+0000 7f0c4383f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de620ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.444+0000 7f0c4383f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de61fba780 con 0x55de61fb1120 2024-04-23T19:12:13.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.444+0000 7f0c4133e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de620ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.444+0000 7f0c4133e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de620ac3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40442/0 (socket says 172.21.15.63:40442) 2024-04-23T19:12:13.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.444+0000 7f0c4133e640 1 -- 172.21.15.63:0/3613985832 learned_addr learned my addr 172.21.15.63:0/3613985832 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.445+0000 7f0c4133e640 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de620ac8e0 con 0x55de61fb1120 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.445+0000 7f0c4133e640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de620ac3a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400a470 tx=0x7f0c34003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.445+0000 7f0b2de03640 1 -- 172.21.15.63:0/3613985832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34073070 con 0x55de61fb1120 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.445+0000 7f0b2de03640 1 -- 172.21.15.63:0/3613985832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3405d440 con 0x55de61fb1120 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.445+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de620acb70 con 0x55de61fb1120 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.445+0000 7f0b2de03640 1 -- 172.21.15.63:0/3613985832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34057070 con 0x55de61fb1120 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.445+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de620af820 con 0x55de61fb1120 2024-04-23T19:12:13.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.446+0000 7f0b2de03640 1 -- 172.21.15.63:0/3613985832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0c34076070 con 0x55de61fb1120 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.446+0000 7f0b2de03640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b28036e60 0x7f0b28039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.446+0000 7f0b2de03640 1 -- 172.21.15.63:0/3613985832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f0c3408e570 con 0x55de61fb1120 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.446+0000 7f0c40b3d640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b28036e60 0x7f0b28039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.446+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de61fb22e0 0x55de620b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.446+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620b3bb0 con 0x55de61fb22e0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.446+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de61fb22e0 0x55de620b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.447+0000 7f0c40b3d640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b28036e60 0x7f0b28039320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0c38000c00 tx=0x7f0c380013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.447+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de61fb22e0 0x55de620b35e0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.447+0000 7f0c41b3f640 1 -- 172.21.15.63:0/3613985832 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0c3c00a4a0 con 0x55de61fb22e0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.447+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620b0b70 con 0x55de61fb22e0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.447+0000 7f0c41b3f640 1 -- 172.21.15.63:0/3613985832 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0c3c00a4a0 con 0x55de61fb22e0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.447+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620b59b0 con 0x55de61fb22e0 2024-04-23T19:12:13.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.448+0000 7f0c41b3f640 1 -- 172.21.15.63:0/3613985832 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0c3c00a4a0 con 0x55de61fb22e0 2024-04-23T19:12:13.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.448+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620b4be0 con 0x55de61fb22e0 2024-04-23T19:12:13.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.448+0000 7f0c41b3f640 1 -- 172.21.15.63:0/3613985832 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0c3c00a4a0 con 0x55de61fb22e0 2024-04-23T19:12:13.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.448+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de620b6d60 0x55de620b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.448+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620b9690 con 0x55de620b6d60 2024-04-23T19:12:13.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.448+0000 7f0c4133e640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de620b6d60 0x55de620b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.449+0000 7f0c4133e640 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de620b6d60 0x55de620b9150 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.449+0000 7f0c4133e640 1 -- 172.21.15.63:0/3613985832 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0c34075030 con 0x55de620b6d60 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.449+0000 7f0c4383f800 1 Processor -- start 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.449+0000 7f0c4383f800 1 -- start start 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c4383f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620c7ad0 0x55de620c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c4383f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de61ea2e30 con 0x55de620c7ad0 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c41b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620c7ad0 0x55de620c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c41b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620c7ad0 0x55de620c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40458/0 (socket says 172.21.15.63:40458) 2024-04-23T19:12:13.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c41b3f640 1 -- 172.21.15.63:0/898014769 learned_addr learned my addr 172.21.15.63:0/898014769 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c41b3f640 1 -- 172.21.15.63:0/898014769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de620c8430 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620c7ad0 0x55de620c7ef0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55de620b3f90 tx=0x7f0c3c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0b26ffd640 1 -- 172.21.15.63:0/898014769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c013440 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0b26ffd640 1 -- 172.21.15.63:0/898014769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c0135e0 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0b26ffd640 1 -- 172.21.15.63:0/898014769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c0138c0 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.450+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de620c8ea0 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.451+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de620c93a0 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.451+0000 7f0b26ffd640 1 -- 172.21.15.63:0/898014769 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0c3c011040 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.451+0000 7f0b26ffd640 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b18036e60 0x7f0b18039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.451+0000 7f0b26ffd640 1 -- 172.21.15.63:0/898014769 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f0c3c01d070 con 0x55de620c7ad0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c4133e640 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b18036e60 0x7f0b18039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c4383f800 1 Processor -- start 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c4383f800 1 -- start start 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c4133e640 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b18036e60 0x7f0b18039320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55de620b4f80 tx=0x7f0c340067c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c4383f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c4383f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de61f252b0 con 0x55de620d4dd0 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c41b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c41b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620d51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40470/0 (socket says 172.21.15.63:40470) 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c41b3f640 1 -- 172.21.15.63:0/2384862227 learned_addr learned my addr 172.21.15.63:0/2384862227 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.452+0000 7f0c41b3f640 1 -- 172.21.15.63:0/2384862227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de620d57c0 con 0x55de620d4dd0 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/2384862227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620d51f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c04cd80 tx=0x7f0c3c04ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544b43fae18c749e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0b24ff9640 1 -- 172.21.15.63:0/2384862227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c012cc0 con 0x55de620d4dd0 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0b24ff9640 1 -- 172.21.15.63:0/2384862227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c02ac50 con 0x55de620d4dd0 2024-04-23T19:12:13.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0b24ff9640 1 -- 172.21.15.63:0/2384862227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c0014f0 con 0x55de620d4dd0 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c4383f800 1 -- 172.21.15.63:0/2384862227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 msgr2=0x55de620d51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c4383f800 1 --2- 172.21.15.63:0/2384862227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620d51f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c04cd80 tx=0x7f0c3c04ce70 comp rx=0 tx=0).stop 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c4383f800 1 -- 172.21.15.63:0/2384862227 shutdown_connections 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c4383f800 1 --2- 172.21.15.63:0/2384862227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620d51f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c4383f800 1 -- 172.21.15.63:0/2384862227 >> 172.21.15.63:0/2384862227 conn(0x55de620d1130 msgr2=0x55de620cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c4383f800 1 -- 172.21.15.63:0/2384862227 shutdown_connections 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.453+0000 7f0c4383f800 1 -- 172.21.15.63:0/2384862227 wait complete. 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c4383f800 1 Processor -- start 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c4383f800 1 -- start start 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c4383f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620dc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c4383f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de620d5f30 con 0x55de620d4dd0 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c41b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c41b3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620dc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40474/0 (socket says 172.21.15.63:40474) 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 learned_addr learned my addr 172.21.15.63:0/1958266353 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.454+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de620dc8a0 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.455+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620dc360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c04cd80 tx=0x7f0c3c04cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.455+0000 7f0b16ffd640 1 -- 172.21.15.63:0/1958266353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c013e20 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.455+0000 7f0b16ffd640 1 -- 172.21.15.63:0/1958266353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c02ae00 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.455+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de620dcb30 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.455+0000 7f0b16ffd640 1 -- 172.21.15.63:0/1958266353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c012700 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.455+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de620df7e0 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.455+0000 7f0b16ffd640 1 -- 172.21.15.63:0/1958266353 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0c3c041040 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0b16ffd640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b0c036e60 0x7f0b0c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0b16ffd640 1 -- 172.21.15.63:0/1958266353 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f0c3c00e040 con 0x55de620d4dd0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0c4133e640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b0c036e60 0x7f0b0c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0c4133e640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b0c036e60 0x7f0b0c039320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0c3408cb10 tx=0x7f0c3408a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de620e8050 0x55de620ea510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620eac60 con 0x55de620e8050 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.456+0000 7f0c40b3d640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de620e8050 0x55de620ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c40b3d640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de620e8050 0x55de620ea510 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 20 realm 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620e74a0 con 0x55de620e8050 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620eb750 con 0x55de620e8050 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.457+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620ec020 con 0x55de620e8050 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620ec3c0 con 0x55de620e8050 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 20 searching for the correct realm 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de620ee2f0 0x55de620f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.458+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de620ee2f0 0x55de620f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.459+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de620ee2f0 0x55de620f0700 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.459+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.459+0000 7f0c41b3f640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0c3c04e3e0 0x7f0c3c02c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.459+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.459+0000 7f0c4133e640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0c3c04e3e0 0x7f0c3c02c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.459+0000 7f0c4133e640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0c3c04e3e0 0x7f0c3c02c450 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.460+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.460+0000 7f0c4133e640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f0c3409c040 0x7f0c3408e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.460+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.460+0000 7f0c40b3d640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f0c3409c040 0x7f0c3408e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.460+0000 7f0c40b3d640 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f0c3409c040 0x7f0c3408e890 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.461+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.461+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.461+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.461+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.461+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.461+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.462+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.463+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.463+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.463+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.463+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.463+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.463+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.464+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.465+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.466+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.467+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.467+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.467+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.467+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.467+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.467+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620ee2f0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.467+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3409c040 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x55de620e8050 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55de620f0dd0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.468+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620ecd90 con 0x55de620e8050 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620ed630 con 0x55de620e8050 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620f1250 con 0x7f0c3409c040 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 zone default found 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 4 Realm: () 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 zonegroup default 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.469+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620ed240 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620f35e0 con 0x55de620ee2f0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620f3980 con 0x55de620ee2f0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 20 started sync module instance, tier type = 2024-04-23T19:12:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.470+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f2a70 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.471+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.471+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94413616262752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f4240 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.471+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f4a90 con 0x55de620e8050 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.472+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94413616262752] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.473+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.473+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94413616266640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f4b30 con 0x55de620e8050 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.473+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f5630 con 0x55de620ee2f0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.474+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.474+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94413616271648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f5f00 con 0x55de620ee2f0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.474+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f69d0 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.474+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94413616266640] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.475+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94413616271648] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.475+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.475+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94413616268768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f5f00 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.475+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f72b0 con 0x55de620e8050 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.476+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94413616268768] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.476+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.476+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94413616274000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f76a0 con 0x55de620e8050 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.476+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f80c0 con 0x7f0c3409c040 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.477+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94413616274000] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.478+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.478+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94413616282544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f8080 con 0x7f0c3409c040 2024-04-23T19:12:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.478+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f8ac0 con 0x7f0c3409c040 2024-04-23T19:12:13.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.479+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94413616282544] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.479+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.479+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94413616285072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f8a80 con 0x7f0c3409c040 2024-04-23T19:12:13.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.479+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f94c0 con 0x7f0c3409c040 2024-04-23T19:12:13.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94413616287632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f9930 con 0x7f0c3409c040 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 20 add_watcher() i=0 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 20 add_watcher() i=1 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 20 add_watcher() i=2 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 20 add_watcher() i=3 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 20 add_watcher() i=4 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 20 add_watcher() i=5 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94413616285072] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.480+0000 7f0c4383f800 20 add_watcher() i=6 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.481+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94413616287632] v32'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.481+0000 7f0c4383f800 20 add_watcher() i=7 2024-04-23T19:12:13.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.481+0000 7f0c4383f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.481+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de620f9930 con 0x55de620e8050 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.481+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.481+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94413616288896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f98a0 con 0x55de620e8050 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.482+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94413616288896] v32'239 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0af77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 5 note: GC not initialized 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0af67e4640 20 reqs_thread_entry: start 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 20 Filter name: none 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0a67fff640 20 reqs_thread_entry: start 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:13.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.483+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de6212fed0 con 0x7f0c3409c040 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:63 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de6212e670 con 0x55de620e8050 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(63 MH1WC2XQ1S8UISFDZC8W [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-23T19:12:13.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.484+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de6212ef30 con 0x7f0c3409c040 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(64 ada [delete,create,call,writefull 0~473] v32'3 uv3 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0c4383f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0c4383f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0c4383f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94413616507520 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de62133810 con 0x55de620e8050 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== watch-notify(notify (1) cookie 94413616274000 notify 137438953495 ret 0) ==== 691+0+0 (crc 0 0 0) 0x55de6212f2d0 con 0x55de620e8050 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0b24ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953495 cookie 94413616274000 notifier 4828 bl.length()=649 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0b24ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0b24ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0b24ff9640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f0b04003ea0 con 0x55de620e8050 2024-04-23T19:12:13.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.485+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify cookie 94413616507520 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.486+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.486+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== watch-notify(notify_complete (2) cookie 94413616507520 notify 137438953495 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de6212f920 con 0x55de620e8050 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.486+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:67 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de62131320 con 0x55de620e8050 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(67 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v32'20 uv20 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0c4383f800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0c4383f800 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0c4383f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94413616507408 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55de621337f0 con 0x55de620ee2f0 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== watch-notify(notify (1) cookie 94413616271648 notify 137438953480 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55de620df7e0 con 0x55de620ee2f0 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0b17fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953480 cookie 94413616271648 notifier 4828 bl.length()=177 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0b17fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0b17fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-23T19:12:13.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0b17fff640 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f0a74003400 con 0x55de620ee2f0 2024-04-23T19:12:13.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.487+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(68 notify.2 [notify cookie 94413616507408 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.488+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== watch-notify(notify_complete (2) cookie 94413616507408 notify 137438953480 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de620dcb30 con 0x55de620ee2f0 2024-04-23T19:12:13.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.488+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.499 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:13.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.488+0000 7f0af77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.488+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94413616288896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620e4c50 con 0x55de620e8050 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.489+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94413616288896] v32'240 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.489+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94413616262752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f9480 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.490+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94413616262752] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.491+0000 7f0c4383f800 20 remove_watcher() i=0 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.491+0000 7f0c4383f800 2 removed watcher, disabling cache 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.491+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94413616266640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f9480 con 0x55de620e8050 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.492+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94413616266640] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.492+0000 7f0c4383f800 20 remove_watcher() i=1 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.492+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94413616271648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f3d90 con 0x55de620ee2f0 2024-04-23T19:12:13.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.493+0000 7f0c41b3f640 1 -- 172.21.15.63:0/1958266353 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94413616271648] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3c029070 con 0x55de620ee2f0 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.493+0000 7f0c4383f800 20 remove_watcher() i=2 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.493+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94413616268768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f9480 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.494+0000 7f0c4133e640 1 -- 172.21.15.63:0/1958266353 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94413616268768] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c34075030 con 0x7f0c3c04e3e0 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.494+0000 7f0c4383f800 20 remove_watcher() i=3 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.494+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94413616274000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f45e0 con 0x55de620e8050 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.495+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94413616274000] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x55de620e8050 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.495+0000 7f0c4383f800 20 remove_watcher() i=4 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.495+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94413616282544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f9480 con 0x7f0c3409c040 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.496+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94413616282544] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.496+0000 7f0c4383f800 20 remove_watcher() i=5 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.496+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94413616285072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f5a50 con 0x7f0c3409c040 2024-04-23T19:12:13.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.497+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94413616285072] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.497+0000 7f0c4383f800 20 remove_watcher() i=6 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.497+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94413616287632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55de620f9480 con 0x7f0c3409c040 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.498+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94413616287632] v32'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0c3800f5f0 con 0x7f0c3409c040 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.498+0000 7f0c4383f800 20 remove_watcher() i=7 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f0c3409c040 msgr2=0x7f0c3408e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f0c3409c040 0x7f0c3408e890 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de620e8050 msgr2=0x55de620ea510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de620e8050 0x55de620ea510 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0c3c04e3e0 msgr2=0x7f0c3c02c450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f0c3c04e3e0 0x7f0c3c02c450 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de620ee2f0 msgr2=0x55de620f0700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55de620ee2f0 0x55de620f0700 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b0c036e60 msgr2=0x7f0b0c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b0c036e60 0x7f0b0c039320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0c3408cb10 tx=0x7f0c3408a000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c40b3d640 1 -- 172.21.15.63:0/1958266353 reap_dead start 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 msgr2=0x55de620dc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620dc360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c04cd80 tx=0x7f0c3c04cbd0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 shutdown_connections 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/1958266353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620d4dd0 0x55de620dc360 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 >> 172.21.15.63:0/1958266353 conn(0x55de620d1130 msgr2=0x55de620ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 shutdown_connections 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/1958266353 wait complete. 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b18036e60 msgr2=0x7f0b18039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b18036e60 0x7f0b18039320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55de620b4f80 tx=0x7f0c340067c0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620c7ad0 msgr2=0x55de620c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620c7ad0 0x55de620c7ef0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55de620b3f90 tx=0x7f0c3c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 shutdown_connections 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b18036e60 0x7f0b18039320 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 --2- 172.21.15.63:0/898014769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de620c7ad0 0x55de620c7ef0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 >> 172.21.15.63:0/898014769 conn(0x55de620bbb70 msgr2=0x55de620bbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 shutdown_connections 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.499+0000 7f0c4383f800 1 -- 172.21.15.63:0/898014769 wait complete. 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de620b6d60 msgr2=0x55de620b9150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de620b6d60 0x55de620b9150 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de61fb22e0 msgr2=0x55de620b35e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de61fb22e0 0x55de620b35e0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b28036e60 msgr2=0x7f0b28039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b28036e60 0x7f0b28039320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0c38000c00 tx=0x7f0c380013b0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 msgr2=0x55de620ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de620ac3a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400a470 tx=0x7f0c34003020 comp rx=0 tx=0).stop 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 shutdown_connections 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55de620b6d60 0x55de620b9150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55de61fb22e0 0x55de620b35e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0b28036e60 0x7f0b28039320 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 --2- 172.21.15.63:0/3613985832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de61fb1120 0x55de620ac3a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 >> 172.21.15.63:0/3613985832 conn(0x55de61fac780 msgr2=0x55de61f25810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 shutdown_connections 2024-04-23T19:12:13.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.500+0000 7f0c4383f800 1 -- 172.21.15.63:0/3613985832 wait complete. 2024-04-23T19:12:13.511 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:13.511 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:13.511 ERROR:root:0 2024-04-23T19:12:13.512 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:13.518 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx00000c799ef5777fc5305-006628080d-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:12:13.518 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2024-04-23T19:12:13.528 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:13.528 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2024-04-23T19:12:13.541 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2024-04-23T19:12:13.541 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2024-04-23T19:12:13.557 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-23T19:12:13.557 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-23T19:12:13.565 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2024-04-23T19:12:13.566 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:13.569 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:13.570 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2024-04-23T19:12:13.574 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:13.574 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:13.574 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:13.574 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write' 2024-04-23T19:12:13.621 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:13.621 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.621+0000 7fd1bb546800 1 Processor -- start 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.621+0000 7fd1bb546800 1 -- start start 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.621+0000 7fd1bb546800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55ce2205ec00 0x55ce2205d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.621+0000 7fd1bb546800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ce21f9b7f0 con 0x55ce2205d6a0 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.621+0000 7fd1bb546800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55ce21f8ae20 con 0x55ce2205ec00 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.622+0000 7fd1b904c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55ce2205ec00 0x55ce2205d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.622+0000 7fd1b884b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55ce2205d6a0 0x55ce220f1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56714/0 (socket says 172.21.15.63:56714) 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.622+0000 7fd1b904c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55ce2205ec00 0x55ce2205d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40484/0 (socket says 172.21.15.63:40484) 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.622+0000 7fd1b884b640 1 -- 172.21.15.63:0/2305031711 learned_addr learned my addr 172.21.15.63:0/2305031711 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.622+0000 7fd1b904c640 1 -- 172.21.15.63:0/2305031711 >> v1:172.21.15.16:6789/0 conn(0x55ce2205d6a0 legacy=0x55ce220f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.622+0000 7fd1abfff640 1 -- 172.21.15.63:0/2305031711 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3226720788 0 0) 0x55ce21f9b7f0 con 0x55ce2205d6a0 2024-04-23T19:12:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.622+0000 7fd1b904c640 1 -- 172.21.15.63:0/2305031711 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce21fa9b50 con 0x55ce2205ec00 2024-04-23T19:12:13.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.624+0000 7fd1b904c640 1 --2- 172.21.15.63:0/2305031711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205ec00 0x55ce2205d160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00a470 tx=0x7fd1ac01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79425db78325c22b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.624+0000 7fd1abfff640 1 -- 172.21.15.63:0/2305031711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac057070 con 0x55ce2205ec00 2024-04-23T19:12:13.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.624+0000 7fd1abfff640 1 -- 172.21.15.63:0/2305031711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac04ac00 con 0x55ce2205ec00 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.624+0000 7fd1abfff640 1 -- 172.21.15.63:0/2305031711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac052440 con 0x55ce2205ec00 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.625+0000 7fd1bb546800 1 -- 172.21.15.63:0/2305031711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205ec00 msgr2=0x55ce2205d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.625+0000 7fd1bb546800 1 --2- 172.21.15.63:0/2305031711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205ec00 0x55ce2205d160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00a470 tx=0x7fd1ac01cbd0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.625+0000 7fd1bb546800 1 -- 172.21.15.63:0/2305031711 shutdown_connections 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.625+0000 7fd1bb546800 1 --2- 172.21.15.63:0/2305031711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205ec00 0x55ce2205d160 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.625+0000 7fd1bb546800 1 -- 172.21.15.63:0/2305031711 >> 172.21.15.63:0/2305031711 conn(0x55ce220e47e0 msgr2=0x55ce220e6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.625+0000 7fd1bb546800 1 -- 172.21.15.63:0/2305031711 shutdown_connections 2024-04-23T19:12:13.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.625+0000 7fd1bb546800 1 -- 172.21.15.63:0/2305031711 wait complete. 2024-04-23T19:12:13.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.637+0000 7fd1bb546800 1 Processor -- start 2024-04-23T19:12:13.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.638+0000 7fd1bb546800 1 -- start start 2024-04-23T19:12:13.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.638+0000 7fd1bb546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221dc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.638+0000 7fd1bb546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce220346f0 con 0x55ce2205d6a0 2024-04-23T19:12:13.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.638+0000 7fd1b904c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221dc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.638+0000 7fd1b904c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221dc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40496/0 (socket says 172.21.15.63:40496) 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.638+0000 7fd1b904c640 1 -- 172.21.15.63:0/174910848 learned_addr learned my addr 172.21.15.63:0/174910848 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.638+0000 7fd1b904c640 1 -- 172.21.15.63:0/174910848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce221dce70 con 0x55ce2205d6a0 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd1b904c640 1 --2- 172.21.15.63:0/174910848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221dc8a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac055040 tx=0x7fd1ac04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad819b637a0e3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd0a7606640 1 -- 172.21.15.63:0/174910848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac057040 con 0x55ce2205d6a0 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd0a7606640 1 -- 172.21.15.63:0/174910848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac063070 con 0x55ce2205d6a0 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd0a7606640 1 -- 172.21.15.63:0/174910848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac0528e0 con 0x55ce2205d6a0 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd1bb546800 1 -- 172.21.15.63:0/174910848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 msgr2=0x55ce221dc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd1bb546800 1 --2- 172.21.15.63:0/174910848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221dc8a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac055040 tx=0x7fd1ac04c000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd1bb546800 1 -- 172.21.15.63:0/174910848 shutdown_connections 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd1bb546800 1 --2- 172.21.15.63:0/174910848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221dc8a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.639+0000 7fd1bb546800 1 -- 172.21.15.63:0/174910848 >> 172.21.15.63:0/174910848 conn(0x55ce220e47e0 msgr2=0x55ce221d81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1bb546800 1 -- 172.21.15.63:0/174910848 shutdown_connections 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1bb546800 1 -- 172.21.15.63:0/174910848 wait complete. 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1bb546800 1 Processor -- start 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1bb546800 1 -- start start 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1bb546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1bb546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce220f4a10 con 0x55ce2205d6a0 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1b904c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1b904c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221e42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40512/0 (socket says 172.21.15.63:40512) 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1b904c640 1 -- 172.21.15.63:0/1284165207 learned_addr learned my addr 172.21.15.63:0/1284165207 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.640+0000 7fd1b904c640 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce221e4830 con 0x55ce2205d6a0 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.641+0000 7fd1b904c640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221e42f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac060040 tx=0x7fd1ac002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.641+0000 7fd0a5e03640 1 -- 172.21.15.63:0/1284165207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac057050 con 0x55ce2205d6a0 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.641+0000 7fd0a5e03640 1 -- 172.21.15.63:0/1284165207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac052ac0 con 0x55ce2205d6a0 2024-04-23T19:12:13.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.641+0000 7fd0a5e03640 1 -- 172.21.15.63:0/1284165207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac052dc0 con 0x55ce2205d6a0 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.641+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce221e4ac0 con 0x55ce2205d6a0 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.641+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce221e7770 con 0x55ce2205d6a0 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.642+0000 7fd0a5e03640 1 -- 172.21.15.63:0/1284165207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd1ac05d470 con 0x55ce2205d6a0 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.642+0000 7fd0a5e03640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0a0036eb0 0x7fd0a0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.642+0000 7fd0a5e03640 1 -- 172.21.15.63:0/1284165207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fd1ac08e130 con 0x55ce2205d6a0 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.642+0000 7fd1b884b640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0a0036eb0 0x7fd0a0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.642+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2205ec00 0x55ce221eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.642+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce221ebab0 con 0x55ce2205ec00 2024-04-23T19:12:13.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.642+0000 7fd1b984d640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2205ec00 0x55ce221eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.643+0000 7fd1b884b640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0a0036eb0 0x7fd0a0039370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0000c00 tx=0x7fd1b00013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.643+0000 7fd1b984d640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2205ec00 0x55ce221eb530 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.643+0000 7fd1b984d640 1 -- 172.21.15.63:0/1284165207 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce2205ec00 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.643+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce221e8ac0 con 0x55ce2205ec00 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.643+0000 7fd1b984d640 1 -- 172.21.15.63:0/1284165207 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce2205ec00 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.643+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce221ed900 con 0x55ce2205ec00 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.643+0000 7fd1b984d640 1 -- 172.21.15.63:0/1284165207 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce2205ec00 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.644+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce221ecb30 con 0x55ce2205ec00 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.644+0000 7fd1b984d640 1 -- 172.21.15.63:0/1284165207 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce2205ec00 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.644+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ce221eecb0 0x55ce221f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.644+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce221f15e0 con 0x55ce221eecb0 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.644+0000 7fd1b904c640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ce221eecb0 0x55ce221f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.644+0000 7fd1b904c640 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ce221eecb0 0x55ce221f10a0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.645+0000 7fd1b904c640 1 -- 172.21.15.63:0/1284165207 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd1ac078070 con 0x55ce221eecb0 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.645+0000 7fd1bb546800 1 Processor -- start 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.645+0000 7fd1bb546800 1 -- start start 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.645+0000 7fd1bb546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce221ffa20 0x55ce221ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.645+0000 7fd1bb546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce21fdae30 con 0x55ce221ffa20 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.645+0000 7fd1b984d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce221ffa20 0x55ce221ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd1b984d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce221ffa20 0x55ce221ffe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40528/0 (socket says 172.21.15.63:40528) 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd1b984d640 1 -- 172.21.15.63:0/3693823436 learned_addr learned my addr 172.21.15.63:0/3693823436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd1b984d640 1 -- 172.21.15.63:0/3693823436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce22200380 con 0x55ce221ffa20 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd1b984d640 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce221ffa20 0x55ce221ffe40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55ce221ebe90 tx=0x7fd1b40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd09effd640 1 -- 172.21.15.63:0/3693823436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b4013440 con 0x55ce221ffa20 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd09effd640 1 -- 172.21.15.63:0/3693823436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b40135e0 con 0x55ce221ffa20 2024-04-23T19:12:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce22200df0 con 0x55ce221ffa20 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd09effd640 1 -- 172.21.15.63:0/3693823436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b40138c0 con 0x55ce221ffa20 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.646+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce222012f0 con 0x55ce221ffa20 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.647+0000 7fd09effd640 1 -- 172.21.15.63:0/3693823436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd1b4011040 con 0x55ce221ffa20 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.647+0000 7fd09effd640 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd094036eb0 0x7fd094039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.647+0000 7fd09effd640 1 -- 172.21.15.63:0/3693823436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fd1b401d070 con 0x55ce221ffa20 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.647+0000 7fd1b904c640 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd094036eb0 0x7fd094039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.647+0000 7fd1bb546800 1 Processor -- start 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.647+0000 7fd1b904c640 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd094036eb0 0x7fd094039370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55ce221eced0 tx=0x7fd1ac08e050 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1bb546800 1 -- start start 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1bb546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce2220d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1bb546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce221dd130 con 0x55ce2220cd10 2024-04-23T19:12:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1b984d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce2220d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1b984d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce2220d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40534/0 (socket says 172.21.15.63:40534) 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1b984d640 1 -- 172.21.15.63:0/383725598 learned_addr learned my addr 172.21.15.63:0/383725598 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1b984d640 1 -- 172.21.15.63:0/383725598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce2220d700 con 0x55ce2220cd10 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd1b984d640 1 --2- 172.21.15.63:0/383725598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce2220d130 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd1b404ccf0 tx=0x7fd1b402b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d73d848eb2b989e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd09cff9640 1 -- 172.21.15.63:0/383725598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b402bc90 con 0x55ce2220cd10 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd09cff9640 1 -- 172.21.15.63:0/383725598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b402be30 con 0x55ce2220cd10 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.648+0000 7fd09cff9640 1 -- 172.21.15.63:0/383725598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b402b4f0 con 0x55ce2220cd10 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 -- 172.21.15.63:0/383725598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 msgr2=0x55ce2220d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 --2- 172.21.15.63:0/383725598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce2220d130 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd1b404ccf0 tx=0x7fd1b402b8e0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 -- 172.21.15.63:0/383725598 shutdown_connections 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 --2- 172.21.15.63:0/383725598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce2220d130 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 -- 172.21.15.63:0/383725598 >> 172.21.15.63:0/383725598 conn(0x55ce22208fd0 msgr2=0x55ce22207080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 -- 172.21.15.63:0/383725598 shutdown_connections 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 -- 172.21.15.63:0/383725598 wait complete. 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 Processor -- start 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.649+0000 7fd1bb546800 1 -- start start 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1bb546800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce222142c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1bb546800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce2220de80 con 0x55ce2220cd10 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1b984d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce222142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1b984d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce222142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40550/0 (socket says 172.21.15.63:40550) 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 learned_addr learned my addr 172.21.15.63:0/3387955826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce22214800 con 0x55ce2220cd10 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1b984d640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce222142c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd1b404bc10 tx=0x7fd1b404cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd08affd640 1 -- 172.21.15.63:0/3387955826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b4013b90 con 0x55ce2220cd10 2024-04-23T19:12:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd08affd640 1 -- 172.21.15.63:0/3387955826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b4013d30 con 0x55ce2220cd10 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd08affd640 1 -- 172.21.15.63:0/3387955826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b402a480 con 0x55ce2220cd10 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce22214a90 con 0x55ce2220cd10 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.650+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce22217740 con 0x55ce2220cd10 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.651+0000 7fd08affd640 1 -- 172.21.15.63:0/3387955826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fd1b402a620 con 0x55ce2220cd10 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.651+0000 7fd08affd640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd084036eb0 0x7fd084039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd08affd640 1 -- 172.21.15.63:0/3387955826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fd1b4032070 con 0x55ce2220cd10 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd1b904c640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd084036eb0 0x7fd084039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2221fd50 0x55ce22222210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd1b904c640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd084036eb0 0x7fd084039370 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac05d040 tx=0x7fd1ac086000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22222960 con 0x55ce2221fd50 2024-04-23T19:12:13.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd1b884b640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2221fd50 0x55ce22222210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.652+0000 7fd1b884b640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2221fd50 0x55ce22222210 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 realm 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce2221f1a0 con 0x55ce2221fd50 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22223450 con 0x55ce2221fd50 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22223d20 con 0x55ce2221fd50 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.653+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce222240c0 con 0x55ce2221fd50 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1bb546800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1bb546800 20 searching for the correct realm 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ce22225ff0 0x55ce22228400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1b984d640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ce22225ff0 0x55ce22228400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1b984d640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ce22225ff0 0x55ce22228400 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.654+0000 7fd1b984d640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1b4045780 0x7fd1b4002b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.655+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1b4045780 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.655+0000 7fd1b904c640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1b4045780 0x7fd1b4002b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.655+0000 7fd1b904c640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1b4045780 0x7fd1b4002b10 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.655+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.655+0000 7fd1b904c640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1ac09b040 0x7fd1ac05e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.655+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.656+0000 7fd1b884b640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1ac09b040 0x7fd1ac05e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.656+0000 7fd1b884b640 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1ac09b040 0x7fd1ac05e790 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.656+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.656+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.657+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1b4045780 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.658+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1b4045780 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.659+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.659+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.659+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.659+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.659+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.659+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.660+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.661+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.661+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.661+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.661+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1b4045780 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.661+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.661+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.662+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.662+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1b4045780 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.662+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.662+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.662+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.662+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1b4045780 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce22225ff0 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1ac09b040 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.663+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x55ce2221fd50 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228ad0 con 0x7fd1b4045780 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:13.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22224a90 con 0x55ce2221fd50 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.664+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22225330 con 0x55ce2221fd50 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22228f50 con 0x7fd1ac09b040 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 zone default found 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 4 Realm: () 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 zonegroup default 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22224f90 con 0x7fd1b4045780 2024-04-23T19:12:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222b310 con 0x55ce22225ff0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.665+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1bb546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222b6b0 con 0x55ce22225ff0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1bb546800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1bb546800 20 started sync module instance, tier type = 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1bb546800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:13.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.666+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222a7a0 con 0x7fd1b4045780 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.667+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.667+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94343824321424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222a400 con 0x7fd1b4045780 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.667+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222c400 con 0x55ce2221fd50 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.668+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94343824321424] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.668+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.668+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94343824325760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222c7f0 con 0x55ce2221fd50 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.668+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222d240 con 0x55ce22225ff0 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.669+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.669+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94343824330032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222d240 con 0x55ce22225ff0 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.669+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222e240 con 0x7fd1b4045780 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.669+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94343824325760] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.670+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.670+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94343824334128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222e630 con 0x7fd1b4045780 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.670+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222f090 con 0x55ce2221fd50 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.670+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94343824330032] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.671+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.671+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94343824332336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222f090 con 0x55ce2221fd50 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.672+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94343824334128] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.672+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222f630 con 0x7fd1ac09b040 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.673+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.673+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94343824326944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.673+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94343824332336] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.673+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce22230070 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.674+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94343824326944] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.674+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.674+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94343824341584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce22230070 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.674+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce22230920 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94343824341584] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94343824337776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce222308e0 con 0x7fd1ac09b040 2024-04-23T19:12:13.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 20 add_watcher() i=0 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 20 add_watcher() i=1 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 20 add_watcher() i=2 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 20 add_watcher() i=3 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 20 add_watcher() i=4 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 20 add_watcher() i=5 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.675+0000 7fd1bb546800 20 add_watcher() i=6 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.676+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94343824337776] v32'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.676+0000 7fd1bb546800 20 add_watcher() i=7 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.676+0000 7fd1bb546800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.676+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22233da0 con 0x55ce2221fd50 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.676+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.676+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94343824343536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce22231220 con 0x55ce2221fd50 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.677+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94343824343536] v32'241 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd06f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 5 note: GC not initialized 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd06e7e4640 20 reqs_thread_entry: start 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 20 Filter name: none 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fcfdffff640 20 reqs_thread_entry: start 2024-04-23T19:12:13.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.678+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce22267350 con 0x7fd1ac09b040 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.679+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.679+0000 7fd1bb546800 20 rados_obj.operate() r=0 bl.length=473 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.679+0000 7fd1bb546800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.679+0000 7fd1bb546800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.679+0000 7fd1bb546800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.679+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2226d940 con 0x7fd1ac09b040 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.680+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v32'5 uv5 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.680+0000 7fd1bb546800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.680+0000 7fd1bb546800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.680+0000 7fd1bb546800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.680+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94343824593840 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ce2226c490 con 0x55ce2221fd50 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94343824332336 notify 137438953497 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7fd1b0018040 con 0x55ce2221fd50 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd09cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953497 cookie 94343824332336 notifier 4835 bl.length()=636 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd09cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd09cff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd09cff9640 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fd07c003ea0 con 0x55ce2221fd50 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94343824593840 out=8b] v0'0 uv243 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.681+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 94343824593840 notify 137438953497 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fd1b0004cc0 con 0x55ce2221fd50 2024-04-23T19:12:13.711 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.682+0000 7fd06f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.682+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94343824343536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2226c8a0 con 0x55ce2221fd50 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.683+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94343824343536] v32'242 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.683+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94343824321424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce222301f0 con 0x7fd1b4045780 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.684+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94343824321424] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.684+0000 7fd1bb546800 20 remove_watcher() i=0 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.684+0000 7fd1bb546800 2 removed watcher, disabling cache 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.684+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94343824325760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222ab90 con 0x55ce2221fd50 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.686+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94343824325760] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.686+0000 7fd1bb546800 20 remove_watcher() i=1 2024-04-23T19:12:13.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.686+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94343824330032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222ab90 con 0x55ce22225ff0 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.687+0000 7fd1b984d640 1 -- 172.21.15.63:0/3387955826 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94343824330032] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b400a4a0 con 0x55ce22225ff0 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.687+0000 7fd1bb546800 20 remove_watcher() i=2 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.687+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94343824334128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222ab90 con 0x7fd1b4045780 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.688+0000 7fd1b904c640 1 -- 172.21.15.63:0/3387955826 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94343824334128] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1ac078070 con 0x7fd1b4045780 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.688+0000 7fd1bb546800 20 remove_watcher() i=3 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.688+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94343824332336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222dd30 con 0x55ce2221fd50 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.689+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94343824332336] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x55ce2221fd50 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.689+0000 7fd1bb546800 20 remove_watcher() i=4 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.689+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94343824326944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222ab90 con 0x7fd1ac09b040 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.690+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94343824326944] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.690+0000 7fd1bb546800 20 remove_watcher() i=5 2024-04-23T19:12:13.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.690+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94343824341584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222d630 con 0x7fd1ac09b040 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.691+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94343824341584] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.691+0000 7fd1bb546800 20 remove_watcher() i=6 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.691+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94343824337776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ce2222dd30 con 0x7fd1ac09b040 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94343824337776] v32'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1b000f5f0 con 0x7fd1ac09b040 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 20 remove_watcher() i=7 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1ac09b040 msgr2=0x7fd1ac05e790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fd1ac09b040 0x7fd1ac05e790 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2221fd50 msgr2=0x55ce22222210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2221fd50 0x55ce22222210 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1b4045780 msgr2=0x7fd1b4002b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fd1b4045780 0x7fd1b4002b10 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ce22225ff0 msgr2=0x55ce22228400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ce22225ff0 0x55ce22228400 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd084036eb0 msgr2=0x7fd084039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd084036eb0 0x7fd084039370 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac05d040 tx=0x7fd1ac086000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 msgr2=0x55ce222142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3387955826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2220cd10 0x55ce222142c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd1b404bc10 tx=0x7fd1b404cc80 comp rx=0 tx=0).stop 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 reap_dead start 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1b884b640 1 -- 172.21.15.63:0/3387955826 reap_dead start 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 shutdown_connections 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 >> 172.21.15.63:0/3387955826 conn(0x55ce22208fd0 msgr2=0x55ce22206680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 shutdown_connections 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.692+0000 7fd1bb546800 1 -- 172.21.15.63:0/3387955826 wait complete. 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd094036eb0 msgr2=0x7fd094039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd094036eb0 0x7fd094039370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55ce221eced0 tx=0x7fd1ac08e050 comp rx=0 tx=0).stop 2024-04-23T19:12:13.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce221ffa20 msgr2=0x55ce221ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce221ffa20 0x55ce221ffe40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55ce221ebe90 tx=0x7fd1b40178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 shutdown_connections 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd094036eb0 0x7fd094039370 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/3693823436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce221ffa20 0x55ce221ffe40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 >> 172.21.15.63:0/3693823436 conn(0x55ce221f3ac0 msgr2=0x55ce221f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 shutdown_connections 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/3693823436 wait complete. 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ce221eecb0 msgr2=0x55ce221f10a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ce221eecb0 0x55ce221f10a0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2205ec00 msgr2=0x55ce221eb530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2205ec00 0x55ce221eb530 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0a0036eb0 msgr2=0x7fd0a0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0a0036eb0 0x7fd0a0039370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0000c00 tx=0x7fd1b00013b0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 msgr2=0x55ce221e42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221e42f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac060040 tx=0x7fd1ac002fb0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 shutdown_connections 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ce221eecb0 0x55ce221f10a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ce2205ec00 0x55ce221eb530 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fd0a0036eb0 0x7fd0a0039370 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 --2- 172.21.15.63:0/1284165207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce2205d6a0 0x55ce221e42f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 >> 172.21.15.63:0/1284165207 conn(0x55ce220e47e0 msgr2=0x55ce221d6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 shutdown_connections 2024-04-23T19:12:13.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.693+0000 7fd1bb546800 1 -- 172.21.15.63:0/1284165207 wait complete. 2024-04-23T19:12:13.722 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:13.723 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:13.723 ERROR:root:0 2024-04-23T19:12:13.723 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-04-23T19:12:13.723 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-04-23T19:12:13.723 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid system_user --display-name 'System User' --access-key system_akey --secret system_skey --caps user-info-without-keys=read --system 2024-04-23T19:12:13.780 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:13.780 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:13.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.780+0000 7fe6cb92e800 1 Processor -- start 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.780+0000 7fe6cb92e800 1 -- start start 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6cb92e800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x555617a5f4b0 0x555617a5f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6cb92e800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5556179127f0 con 0x555617a60230 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6cb92e800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x555617901e20 con 0x555617a5f4b0 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6c7667640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x555617a5f4b0 0x555617a5f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6c6e66640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x555617a60230 0x555617a606e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56728/0 (socket says 172.21.15.63:56728) 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3193480275 learned_addr learned my addr 172.21.15.63:0/3193480275 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6c7667640 1 -- 172.21.15.63:0/3193480275 >> v1:172.21.15.16:6789/0 conn(0x555617a60230 legacy=0x555617a606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.781+0000 7fe6c7667640 1 -- 172.21.15.63:0/3193480275 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555617920b50 con 0x555617a5f4b0 2024-04-23T19:12:13.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.783+0000 7fe6c7667640 1 --2- 172.21.15.63:0/3193480275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5f8d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800a470 tx=0x7fe6b801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38432eb4a452dc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.783+0000 7fe6c6665640 1 -- 172.21.15.63:0/3193480275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b8057070 con 0x555617a5f4b0 2024-04-23T19:12:13.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.783+0000 7fe6c6665640 1 -- 172.21.15.63:0/3193480275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b804ad80 con 0x555617a5f4b0 2024-04-23T19:12:13.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.783+0000 7fe6c6665640 1 -- 172.21.15.63:0/3193480275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b8052590 con 0x555617a5f4b0 2024-04-23T19:12:13.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.784+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3193480275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 msgr2=0x555617a5f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.784+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3193480275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5f8d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800a470 tx=0x7fe6b801ce50 comp rx=0 tx=0).stop 2024-04-23T19:12:13.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.784+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3193480275 shutdown_connections 2024-04-23T19:12:13.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.784+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3193480275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5f8d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.784+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3193480275 >> 172.21.15.63:0/3193480275 conn(0x555617a59d50 msgr2=0x555617a5c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.784+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3193480275 shutdown_connections 2024-04-23T19:12:13.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.784+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3193480275 wait complete. 2024-04-23T19:12:13.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.796+0000 7fe6cb92e800 1 Processor -- start 2024-04-23T19:12:13.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6cb92e800 1 -- start start 2024-04-23T19:12:13.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6cb92e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6cb92e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5556179ab690 con 0x555617a5f4b0 2024-04-23T19:12:13.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6c7667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6c7667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40582/0 (socket says 172.21.15.63:40582) 2024-04-23T19:12:13.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6c7667640 1 -- 172.21.15.63:0/1821157079 learned_addr learned my addr 172.21.15.63:0/1821157079 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6c7667640 1 -- 172.21.15.63:0/1821157079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555617b523d0 con 0x555617a5f4b0 2024-04-23T19:12:13.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.797+0000 7fe6c7667640 1 --2- 172.21.15.63:0/1821157079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5ebb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8055040 tx=0x7fe6b804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24c8e8657131c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe5b560a640 1 -- 172.21.15.63:0/1821157079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b8057040 con 0x555617a5f4b0 2024-04-23T19:12:13.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe5b560a640 1 -- 172.21.15.63:0/1821157079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b8063070 con 0x555617a5f4b0 2024-04-23T19:12:13.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe5b560a640 1 -- 172.21.15.63:0/1821157079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b80527e0 con 0x555617a5f4b0 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1821157079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 msgr2=0x555617a5ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1821157079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5ebb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8055040 tx=0x7fe6b804c000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1821157079 shutdown_connections 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1821157079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617a5ebb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1821157079 >> 172.21.15.63:0/1821157079 conn(0x555617a59d50 msgr2=0x555617b4c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1821157079 shutdown_connections 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.798+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1821157079 wait complete. 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6cb92e800 1 Processor -- start 2024-04-23T19:12:13.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6cb92e800 1 -- start start 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6cb92e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617b5b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6cb92e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555617a68730 con 0x555617a5f4b0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6c7667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617b5b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6c7667640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617b5b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40594/0 (socket says 172.21.15.63:40594) 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6c7667640 1 -- 172.21.15.63:0/4105835512 learned_addr learned my addr 172.21.15.63:0/4105835512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.799+0000 7fe6c7667640 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555617b5bb30 con 0x555617a5f4b0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.800+0000 7fe6c7667640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617b5b5f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8060040 tx=0x7fe6b8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.800+0000 7fe5b3e07640 1 -- 172.21.15.63:0/4105835512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b8057050 con 0x555617a5f4b0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.800+0000 7fe5b3e07640 1 -- 172.21.15.63:0/4105835512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b8052ce0 con 0x555617a5f4b0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.800+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555617b5bdc0 con 0x555617a5f4b0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.800+0000 7fe5b3e07640 1 -- 172.21.15.63:0/4105835512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b805d470 con 0x555617a5f4b0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.800+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555617b5ea20 con 0x555617a5f4b0 2024-04-23T19:12:13.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe5b3e07640 1 -- 172.21.15.63:0/4105835512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe6b805d610 con 0x555617a5f4b0 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe5b3e07640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5ac036eb0 0x7fe5ac039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe5b3e07640 1 -- 172.21.15.63:0/4105835512 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe6b808e420 con 0x555617a5f4b0 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe6c6e66640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5ac036eb0 0x7fe5ac039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617a60230 0x555617b627e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b62d60 con 0x555617a60230 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617a60230 0x555617b627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.801+0000 7fe6c6e66640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5ac036eb0 0x7fe5ac039370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000f60 tx=0x7fe6bc0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.802+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617a60230 0x555617b627e0 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.802+0000 7fe6c7e68640 1 -- 172.21.15.63:0/4105835512 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe6c000a4a0 con 0x555617a60230 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.802+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b5fdb0 con 0x555617a60230 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.802+0000 7fe6c7e68640 1 -- 172.21.15.63:0/4105835512 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe6c000a4a0 con 0x555617a60230 2024-04-23T19:12:13.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.802+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b64b60 con 0x555617a60230 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.803+0000 7fe6c7e68640 1 -- 172.21.15.63:0/4105835512 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe6c000a4a0 con 0x555617a60230 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.803+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b63ea0 con 0x555617a60230 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.803+0000 7fe6c7e68640 1 -- 172.21.15.63:0/4105835512 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe6c000a4a0 con 0x555617a60230 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.803+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555617b65f10 0x555617b68300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.803+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b68880 con 0x555617b65f10 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.803+0000 7fe6c7667640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555617b65f10 0x555617b68300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.804+0000 7fe6c7667640 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555617b65f10 0x555617b68300 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.804+0000 7fe6c7667640 1 -- 172.21.15.63:0/4105835512 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe6b8078070 con 0x555617b65f10 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.804+0000 7fe6cb92e800 1 Processor -- start 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.804+0000 7fe6cb92e800 1 -- start start 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6cb92e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b76c30 0x555617b77050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6cb92e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555617951e30 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6c7e68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b76c30 0x555617b77050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6c7e68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b76c30 0x555617b77050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40604/0 (socket says 172.21.15.63:40604) 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6c7e68640 1 -- 172.21.15.63:0/1889198580 learned_addr learned my addr 172.21.15.63:0/1889198580 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6c7e68640 1 -- 172.21.15.63:0/1889198580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555617b77590 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b76c30 0x555617b77050 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x555617b5c9a0 tx=0x7fe6c00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe5b1602640 1 -- 172.21.15.63:0/1889198580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c0013440 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe5b1602640 1 -- 172.21.15.63:0/1889198580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c00135e0 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555617b78000 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe5b1602640 1 -- 172.21.15.63:0/1889198580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c00138c0 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.805+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555617b78500 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.806+0000 7fe5b1602640 1 -- 172.21.15.63:0/1889198580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe6c0011040 con 0x555617b76c30 2024-04-23T19:12:13.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.806+0000 7fe5b1602640 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5a0036eb0 0x7fe5a0039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.806+0000 7fe5b1602640 1 -- 172.21.15.63:0/1889198580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe6c001d070 con 0x555617b76c30 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.806+0000 7fe6c7667640 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5a0036eb0 0x7fe5a0039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6cb92e800 1 Processor -- start 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6cb92e800 1 -- start start 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6c7667640 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5a0036eb0 0x7fe5a0039370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8067ea0 tx=0x7fe6b805cf70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6cb92e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6cb92e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555617b56aa0 con 0x555617b84030 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6c7e68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6c7e68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b84450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40618/0 (socket says 172.21.15.63:40618) 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6c7e68640 1 -- 172.21.15.63:0/1829362676 learned_addr learned my addr 172.21.15.63:0/1829362676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.807+0000 7fe6c7e68640 1 -- 172.21.15.63:0/1829362676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555617b84a20 con 0x555617b84030 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/1829362676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b84450 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe6c004ccb0 tx=0x7fe6c0001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0ae680ad1e29fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe5aaffd640 1 -- 172.21.15.63:0/1829362676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c0012df0 con 0x555617b84030 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe5aaffd640 1 -- 172.21.15.63:0/1829362676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c0001310 con 0x555617b84030 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe5aaffd640 1 -- 172.21.15.63:0/1829362676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c0001630 con 0x555617b84030 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1829362676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 msgr2=0x555617b84450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1829362676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b84450 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe6c004ccb0 tx=0x7fe6c0001f90 comp rx=0 tx=0).stop 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1829362676 shutdown_connections 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1829362676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b84450 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1829362676 >> 172.21.15.63:0/1829362676 conn(0x555617b802b0 msgr2=0x555617b7e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1829362676 shutdown_connections 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1829362676 wait complete. 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.808+0000 7fe6cb92e800 1 Processor -- start 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6cb92e800 1 -- start start 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6cb92e800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b8b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6cb92e800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555617b85150 con 0x555617b84030 2024-04-23T19:12:13.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6c7e68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6c7e68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b8b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40634/0 (socket says 172.21.15.63:40634) 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 learned_addr learned my addr 172.21.15.63:0/3504851523 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555617b8ba90 con 0x555617b84030 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b8b550 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe6c004bb80 tx=0x7fe6c004cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe5a97fa640 1 -- 172.21.15.63:0/3504851523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c0021c60 con 0x555617b84030 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe5a97fa640 1 -- 172.21.15.63:0/3504851523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c0021e00 con 0x555617b84030 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555617b8bd20 con 0x555617b84030 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.809+0000 7fe5a97fa640 1 -- 172.21.15.63:0/3504851523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c002a350 con 0x555617b84030 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.810+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555617b8e9d0 con 0x555617b84030 2024-04-23T19:12:13.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.810+0000 7fe5a97fa640 1 -- 172.21.15.63:0/3504851523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe6c0040040 con 0x555617b84030 2024-04-23T19:12:13.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.810+0000 7fe5a97fa640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe590036e60 0x7fe590039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.810+0000 7fe5a97fa640 1 -- 172.21.15.63:0/3504851523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe6c000e040 con 0x555617b84030 2024-04-23T19:12:13.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.810+0000 7fe6c7667640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe590036e60 0x7fe590039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.811+0000 7fe6c7667640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe590036e60 0x7fe590039320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe6b805db60 tx=0x7fe6b8084000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.811+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.811+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617b97210 0x555617b996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.811+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b99e20 con 0x555617b97210 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.811+0000 7fe6c6e66640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617b97210 0x555617b996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6c6e66640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617b97210 0x555617b996d0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe6bc009050 con 0x555617b97210 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 realm 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b96660 con 0x555617b97210 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe6bc009050 con 0x555617b97210 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b9a910 con 0x555617b97210 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe6bc009050 con 0x555617b97210 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b9b1e0 con 0x555617b97210 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe6bc009050 con 0x555617b97210 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.812+0000 7fe6cb92e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b9b580 con 0x555617b97210 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe6bc009050 con 0x555617b97210 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6cb92e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:13.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6cb92e800 20 searching for the correct realm 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555617b9d460 0x555617b9f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555617b9d460 0x555617b9f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.813+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555617b9d460 0x555617b9f850 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7e68640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6c005b040 0x7fe6c0027060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6c005b040 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7667640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6c005b040 0x7fe6c0027060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7667640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6c005b040 0x7fe6c0027060 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7667640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe6b809b040 0x7fe6b8066380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.814+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.815+0000 7fe6c6e66640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe6b809b040 0x7fe6b8066380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.815+0000 7fe6c6e66640 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe6b809b040 0x7fe6b8066380 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.815+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc009050 con 0x7fe6b809b040 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.815+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc009050 con 0x555617b97210 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.816+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6c005b040 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.817+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6c005b040 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.818+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.819+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.820+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.820+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.820+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.820+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.820+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.820+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6c005b040 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.820+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.821+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.821+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.821+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6c005b040 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.821+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.821+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.821+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.821+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.822+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.822+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6c005b040 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.822+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.822+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b9d460 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.822+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.822+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6b809b040 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x555617b97210 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x555617b9ff00 con 0x7fe6c005b040 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b9bf00 con 0x555617b97210 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.823+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617b9c7a0 con 0x555617b97210 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617ba0380 con 0x7fe6b809b040 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 zone default found 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 4 Realm: () 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:13.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 zonegroup default 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617ba0830 con 0x7fe6c005b040 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617ba25b0 con 0x555617b9d460 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.824+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617ba2a50 con 0x555617b9d460 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6cb92e800 20 started sync module instance, tier type = 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6cb92e800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.825+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba1a50 con 0x7fe6c005b040 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.826+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.826+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93828253621824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba3360 con 0x7fe6c005b040 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.826+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba3bb0 con 0x555617b97210 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.827+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93828253621824] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.827+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.827+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93828253626032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba3c70 con 0x555617b97210 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.827+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba4790 con 0x555617b9d460 2024-04-23T19:12:13.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.829+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93828253626032] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.829+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.829+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93828253631072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba4b80 con 0x555617b9d460 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.829+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba55a0 con 0x7fe6c005b040 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.830+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.830+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93828253634704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba5e70 con 0x7fe6c005b040 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.830+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba6950 con 0x555617b97210 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.830+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93828253631072] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.831+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.831+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93828253639776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba5e70 con 0x555617b97210 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.831+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba73a0 con 0x7fe6b809b040 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.831+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93828253634704] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.832+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93828253639776] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.832+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.832+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93828253628160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba6eb0 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.832+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba7870 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.833+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93828253628160] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.833+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.833+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93828253637056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba7870 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.833+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba8230 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93828253637056] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93828253643712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba8410 con 0x7fe6b809b040 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 20 add_watcher() i=0 2024-04-23T19:12:13.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 20 add_watcher() i=1 2024-04-23T19:12:13.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 20 add_watcher() i=2 2024-04-23T19:12:13.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 20 add_watcher() i=3 2024-04-23T19:12:13.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 20 add_watcher() i=4 2024-04-23T19:12:13.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 20 add_watcher() i=5 2024-04-23T19:12:13.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.834+0000 7fe6cb92e800 20 add_watcher() i=6 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.836+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93828253643712] v32'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.836+0000 7fe6cb92e800 20 add_watcher() i=7 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.836+0000 7fe6cb92e800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.836+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617bab5a0 con 0x555617b97210 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.836+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.836+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93828253658960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba8360 con 0x555617b97210 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.837+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93828253658960] v32'243 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.837+0000 7fe57dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.837+0000 7fe6cb92e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:13.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.837+0000 7fe6cb92e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.837+0000 7fe6cb92e800 5 note: GC not initialized 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.837+0000 7fe57cfe1640 20 reqs_thread_entry: start 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 20 Filter name: none 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe4f67fc640 20 reqs_thread_entry: start 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617bdebb0 con 0x7fe6b809b040 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 system_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 239+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x0 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-23T19:12:13.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 10 cache get: name=default.rgw.meta+users.keys+system_akey : miss 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:63 8.0 8:01223cbf:users.keys::system_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617bdd350 con 0x555617b9d460 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(63 system_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.838+0000 7fe6cb92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.839+0000 7fe6cb92e800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x0 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.839+0000 7fe6cb92e800 10 adding default.rgw.meta+users.keys+system_akey to cache LRU end 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.839+0000 7fe6cb92e800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.839+0000 7fe6cb92e800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.839+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:64 8.1e 8:78ce4db0:users.uid::system_user:head [delete,create,call version.set in=58b,writefull 0~349 in=349b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617bddbf0 con 0x7fe6b809b040 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(64 system_user [delete,create,call,writefull 0~349] v32'3 uv3 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6cb92e800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6cb92e800 10 moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-23T19:12:13.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6cb92e800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [notify cookie 93828253865792 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617be24f0 con 0x7fe6c005b040 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== watch-notify(notify (1) cookie 93828253621824 notify 137438953505 ret 0) ==== 575+0+0 (crc 0 0 0) 0x7fe6b80a2060 con 0x7fe6c005b040 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe5aaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953505 cookie 93828253621824 notifier 4842 bl.length()=533 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe5aaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe5aaffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(65 notify.0 [notify cookie 93828253865792 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe5aaffd640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fe58c003ea0 con 0x7fe6c005b040 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.840+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(66 notify.0 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.841+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== watch-notify(notify_complete (2) cookie 93828253865792 notify 137438953505 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555617b5ea20 con 0x7fe6c005b040 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.841+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 8.0 8:01223cbf:users.keys::system_akey:head [delete,create,writefull 0~15 in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617be0020 con 0x555617b9d460 2024-04-23T19:12:13.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 system_akey [delete,create,writefull 0~15] v32'1 uv1 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6cb92e800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6cb92e800 10 moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6cb92e800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify cookie 93828253865792 in=188b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x555617be24f0 con 0x555617b97210 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 93828253626032 notify 137438953498 ret 0) ==== 218+0+0 (crc 0 0 0) 0x555617bddf90 con 0x555617b97210 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe5aaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953498 cookie 93828253626032 notifier 4842 bl.length()=176 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe5aaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe5aaffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe5aaffd640 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fe58c003ea0 con 0x555617b97210 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(68 notify.1 [notify cookie 93828253865792 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(69 notify.1 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.842+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 93828253865792 notify 137438953498 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555617bdd4a0 con 0x555617b97210 2024-04-23T19:12:13.860 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"system_user","display_name":"System User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"system_user","access_key":"system_akey","secret_key":"system_skey","active":true,"create_date":"2024-04-23T19:12:13.839417Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","system":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.839403Z","tags":[],"group_ids":[]} 2024-04-23T19:12:13.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.843+0000 7fe57dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.843+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93828253658960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617baa6e0 con 0x555617b97210 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.844+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93828253658960] v32'244 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.844+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93828253621824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617baaf50 con 0x7fe6c005b040 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.845+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93828253621824] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.845+0000 7fe6cb92e800 20 remove_watcher() i=0 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.845+0000 7fe6cb92e800 2 removed watcher, disabling cache 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.845+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93828253626032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617baaf50 con 0x555617b97210 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.846+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93828253626032] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.846+0000 7fe6cb92e800 20 remove_watcher() i=1 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.846+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93828253631072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba2eb0 con 0x555617b9d460 2024-04-23T19:12:13.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.848+0000 7fe6c7e68640 1 -- 172.21.15.63:0/3504851523 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93828253631072] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6c0029070 con 0x555617b9d460 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.848+0000 7fe6cb92e800 20 remove_watcher() i=2 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.848+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93828253634704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617baaf50 con 0x7fe6c005b040 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.849+0000 7fe6c7667640 1 -- 172.21.15.63:0/3504851523 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93828253634704] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6b8078070 con 0x7fe6c005b040 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.849+0000 7fe6cb92e800 20 remove_watcher() i=3 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.849+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93828253639776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba2eb0 con 0x555617b97210 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.850+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93828253639776] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x555617b97210 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.850+0000 7fe6cb92e800 20 remove_watcher() i=4 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.850+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93828253628160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617baaf50 con 0x7fe6b809b040 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.851+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93828253628160] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.851+0000 7fe6cb92e800 20 remove_watcher() i=5 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.851+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93828253637056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba2eb0 con 0x7fe6b809b040 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.852+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93828253637056] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.852+0000 7fe6cb92e800 20 remove_watcher() i=6 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.852+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93828253643712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x555617ba3700 con 0x7fe6b809b040 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93828253643712] v32'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe6bc02a040 con 0x7fe6b809b040 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 20 remove_watcher() i=7 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe6b809b040 msgr2=0x7fe6b8066380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe6b809b040 0x7fe6b8066380 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617b97210 msgr2=0x555617b996d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617b97210 0x555617b996d0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6c005b040 msgr2=0x7fe6c0027060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe6c005b040 0x7fe6c0027060 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555617b9d460 msgr2=0x555617b9f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.853+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x555617b9d460 0x555617b9f850 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe590036e60 msgr2=0x7fe590039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe590036e60 0x7fe590039320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe6b805db60 tx=0x7fe6b8084000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 msgr2=0x555617b8b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/3504851523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b84030 0x555617b8b550 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe6c004bb80 tx=0x7fe6c004cc40 comp rx=0 tx=0).stop 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 reap_dead start 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6c6e66640 1 -- 172.21.15.63:0/3504851523 reap_dead start 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 shutdown_connections 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 >> 172.21.15.63:0/3504851523 conn(0x555617b802b0 msgr2=0x555617b7d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 shutdown_connections 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/3504851523 wait complete. 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5a0036eb0 msgr2=0x7fe5a0039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5a0036eb0 0x7fe5a0039370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8067ea0 tx=0x7fe6b805cf70 comp rx=0 tx=0).stop 2024-04-23T19:12:13.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b76c30 msgr2=0x555617b77050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b76c30 0x555617b77050 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x555617b5c9a0 tx=0x7fe6c00178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 shutdown_connections 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5a0036eb0 0x7fe5a0039370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/1889198580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617b76c30 0x555617b77050 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 >> 172.21.15.63:0/1889198580 conn(0x555617b6acd0 msgr2=0x555617b6b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 shutdown_connections 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/1889198580 wait complete. 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555617b65f10 msgr2=0x555617b68300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555617b65f10 0x555617b68300 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617a60230 msgr2=0x555617b627e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617a60230 0x555617b627e0 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5ac036eb0 msgr2=0x7fe5ac039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5ac036eb0 0x7fe5ac039370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000f60 tx=0x7fe6bc0012b0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 msgr2=0x555617b5b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.854+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617b5b5f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8060040 tx=0x7fe6b8002fb0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 shutdown_connections 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x555617b65f10 0x555617b68300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x555617a60230 0x555617b627e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe5ac036eb0 0x7fe5ac039370 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 --2- 172.21.15.63:0/4105835512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555617a5f4b0 0x555617b5b5f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 >> 172.21.15.63:0/4105835512 conn(0x555617a59d50 msgr2=0x555617b4a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 shutdown_connections 2024-04-23T19:12:13.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.855+0000 7fe6cb92e800 1 -- 172.21.15.63:0/4105835512 wait complete. 2024-04-23T19:12:13.873 DEBUG:tasks.util.rgw: json result: {'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-04-23T19:12:13.839417Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.839403Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:13.873 ERROR:root:{'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-04-23T19:12:13.839417Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.839403Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:13.873 ERROR:root:0 2024-04-23T19:12:13.873 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:13.878 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:13.878 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'system_user'] 2024-04-23T19:12:13.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'system_user'] 2024-04-23T19:12:13.879 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid system_user 2024-04-23T19:12:13.923 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:13.923 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:13.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.923+0000 7f7e23f33800 1 Processor -- start 2024-04-23T19:12:13.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.923+0000 7f7e23f33800 1 -- start start 2024-04-23T19:12:13.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.923+0000 7f7e23f33800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558d517eb000 0x558d517eb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.923+0000 7f7e23f33800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558d5169e7f0 con 0x558d517ec260 2024-04-23T19:12:13.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.923+0000 7f7e23f33800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x558d5168de20 con 0x558d517eb000 2024-04-23T19:12:13.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.924+0000 7f7e1fbf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558d517eb000 0x558d517eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.924+0000 7f7e1f3f4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x558d517ec260 0x558d517ec6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56736/0 (socket says 172.21.15.63:56736) 2024-04-23T19:12:13.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.924+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/237372674 learned_addr learned my addr 172.21.15.63:0/237372674 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:13.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.924+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/237372674 >> v1:172.21.15.16:6789/0 conn(0x558d517ec260 legacy=0x558d517ec6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.924+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/237372674 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d516acb50 con 0x558d517eb000 2024-04-23T19:12:13.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.926+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/237372674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d517eb420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000a470 tx=0x7f7e1001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf0d17518cf1f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.926+0000 7f7e1ebf3640 1 -- 172.21.15.63:0/237372674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e10057070 con 0x558d517eb000 2024-04-23T19:12:13.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.926+0000 7f7e1ebf3640 1 -- 172.21.15.63:0/237372674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e10052570 con 0x558d517eb000 2024-04-23T19:12:13.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.926+0000 7f7e1ebf3640 1 -- 172.21.15.63:0/237372674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e10052870 con 0x558d517eb000 2024-04-23T19:12:13.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.927+0000 7f7e23f33800 1 -- 172.21.15.63:0/237372674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 msgr2=0x558d517eb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.927+0000 7f7e23f33800 1 --2- 172.21.15.63:0/237372674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d517eb420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000a470 tx=0x7f7e1001ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:13.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.927+0000 7f7e23f33800 1 -- 172.21.15.63:0/237372674 shutdown_connections 2024-04-23T19:12:13.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.927+0000 7f7e23f33800 1 --2- 172.21.15.63:0/237372674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d517eb420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.927+0000 7f7e23f33800 1 -- 172.21.15.63:0/237372674 >> 172.21.15.63:0/237372674 conn(0x558d517e6600 msgr2=0x558d517e8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.927+0000 7f7e23f33800 1 -- 172.21.15.63:0/237372674 shutdown_connections 2024-04-23T19:12:13.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.927+0000 7f7e23f33800 1 -- 172.21.15.63:0/237372674 wait complete. 2024-04-23T19:12:13.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.939+0000 7f7e23f33800 1 Processor -- start 2024-04-23T19:12:13.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.940+0000 7f7e23f33800 1 -- start start 2024-04-23T19:12:13.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.940+0000 7f7e23f33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.940+0000 7f7e23f33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d517376f0 con 0x558d517eb000 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.940+0000 7f7e1fbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.940+0000 7f7e1fbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518dc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40644/0 (socket says 172.21.15.63:40644) 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.940+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/4284653797 learned_addr learned my addr 172.21.15.63:0/4284653797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.940+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/4284653797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d518dccb0 con 0x558d517eb000 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.941+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/4284653797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518dc6e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7e10055040 tx=0x7f7e1004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc57b6cc9423a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.941+0000 7f7d0de0b640 1 -- 172.21.15.63:0/4284653797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e10057040 con 0x558d517eb000 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.941+0000 7f7d0de0b640 1 -- 172.21.15.63:0/4284653797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e10052990 con 0x558d517eb000 2024-04-23T19:12:13.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.941+0000 7f7d0de0b640 1 -- 172.21.15.63:0/4284653797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e10052c70 con 0x558d517eb000 2024-04-23T19:12:13.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 -- 172.21.15.63:0/4284653797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 msgr2=0x558d518dc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 --2- 172.21.15.63:0/4284653797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518dc6e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7e10055040 tx=0x7f7e1004c000 comp rx=0 tx=0).stop 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 -- 172.21.15.63:0/4284653797 shutdown_connections 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 --2- 172.21.15.63:0/4284653797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518dc6e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 -- 172.21.15.63:0/4284653797 >> 172.21.15.63:0/4284653797 conn(0x558d517e6600 msgr2=0x558d518d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 -- 172.21.15.63:0/4284653797 shutdown_connections 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 -- 172.21.15.63:0/4284653797 wait complete. 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 Processor -- start 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 -- start start 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e23f33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d517f46d0 con 0x558d517eb000 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.942+0000 7f7e1fbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7e1fbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518e16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40654/0 (socket says 172.21.15.63:40654) 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/3670250618 learned_addr learned my addr 172.21.15.63:0/3670250618 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d518dfdb0 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518e16c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000a470 tx=0x7f7e10003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7d07fff640 1 -- 172.21.15.63:0/3670250618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e10073070 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7d07fff640 1 -- 172.21.15.63:0/3670250618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1005d440 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7d07fff640 1 -- 172.21.15.63:0/3670250618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e10057070 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d518e0040 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.943+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d518e0560 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.944+0000 7f7d07fff640 1 -- 172.21.15.63:0/3670250618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7e10076070 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.944+0000 7f7d07fff640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7d08036eb0 0x7f7d08039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.944+0000 7f7d07fff640 1 -- 172.21.15.63:0/3670250618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f7e1008e530 con 0x558d517eb000 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e1f3f4640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7d08036eb0 0x7f7d08039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d517ec260 0x558d518ee680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d518eec10 con 0x558d517ec260 2024-04-23T19:12:13.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e203f6640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d517ec260 0x558d518ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e1f3f4640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7d08036eb0 0x7f7d08039370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7e14000f60 tx=0x7f7e140012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e203f6640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d517ec260 0x558d518ee680 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e203f6640 1 -- 172.21.15.63:0/3670250618 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7e1800a4a0 con 0x558d517ec260 2024-04-23T19:12:13.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.945+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d518ef4f0 con 0x558d517ec260 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.946+0000 7f7e203f6640 1 -- 172.21.15.63:0/3670250618 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7e1800a4a0 con 0x558d517ec260 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.946+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d518f08e0 con 0x558d517ec260 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.946+0000 7f7e203f6640 1 -- 172.21.15.63:0/3670250618 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7e1800a4a0 con 0x558d517ec260 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.946+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d518efd40 con 0x558d517ec260 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.946+0000 7f7e203f6640 1 -- 172.21.15.63:0/3670250618 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7e1800a4a0 con 0x558d517ec260 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.946+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d518f1d90 0x558d518f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.946+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d518f48f0 con 0x558d518f1d90 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.947+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d518f1d90 0x558d518f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.947+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d518f1d90 0x558d518f4180 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.947+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/3670250618 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7e10075030 con 0x558d518f1d90 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e23f33800 1 Processor -- start 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e23f33800 1 -- start start 2024-04-23T19:12:13.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e23f33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51902d70 0x558d51903190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e23f33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d518e1c00 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e203f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51902d70 0x558d51903190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e203f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51902d70 0x558d51903190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40666/0 (socket says 172.21.15.63:40666) 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e203f6640 1 -- 172.21.15.63:0/3137844200 learned_addr learned my addr 172.21.15.63:0/3137844200 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e203f6640 1 -- 172.21.15.63:0/3137844200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d519036d0 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.948+0000 7f7e203f6640 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51902d70 0x558d51903190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x558d518e1400 tx=0x7f7e180178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.949+0000 7f7d057fa640 1 -- 172.21.15.63:0/3137844200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e18013440 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.949+0000 7f7d057fa640 1 -- 172.21.15.63:0/3137844200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e180135e0 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.949+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d51904140 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.949+0000 7f7d057fa640 1 -- 172.21.15.63:0/3137844200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e180138c0 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.949+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d51904640 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7d057fa640 1 -- 172.21.15.63:0/3137844200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7e18011040 con 0x558d51902d70 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7d057fa640 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7cf8036e60 0x7f7cf8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7d057fa640 1 -- 172.21.15.63:0/3137844200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f7e1801d070 con 0x558d51902d70 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7cf8036e60 0x7f7cf8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7e23f33800 1 Processor -- start 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7e23f33800 1 -- start start 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7cf8036e60 0x7f7cf8039320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x558d518e0ba0 tx=0x7f7e100063e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7e23f33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d51910470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.950+0000 7f7e23f33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d516dde30 con 0x558d51910050 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7e203f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d51910470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7e203f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d51910470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40670/0 (socket says 172.21.15.63:40670) 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7e203f6640 1 -- 172.21.15.63:0/3016648561 learned_addr learned my addr 172.21.15.63:0/3016648561 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7e203f6640 1 -- 172.21.15.63:0/3016648561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d51910a40 con 0x558d51910050 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7e203f6640 1 --2- 172.21.15.63:0/3016648561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d51910470 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7e1804cca0 tx=0x7f7e180127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cc29d19e6a9ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7cfeffd640 1 -- 172.21.15.63:0/3016648561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e18042020 con 0x558d51910050 2024-04-23T19:12:13.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7cfeffd640 1 -- 172.21.15.63:0/3016648561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e18012cc0 con 0x558d51910050 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.951+0000 7f7cfeffd640 1 -- 172.21.15.63:0/3016648561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e18027040 con 0x558d51910050 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 -- 172.21.15.63:0/3016648561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 msgr2=0x558d51910470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3016648561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d51910470 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7e1804cca0 tx=0x7f7e180127a0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 -- 172.21.15.63:0/3016648561 shutdown_connections 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3016648561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d51910470 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 -- 172.21.15.63:0/3016648561 >> 172.21.15.63:0/3016648561 conn(0x558d5190c3b0 msgr2=0x558d5190a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 -- 172.21.15.63:0/3016648561 shutdown_connections 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 -- 172.21.15.63:0/3016648561 wait complete. 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 Processor -- start 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 -- start start 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d519174d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.952+0000 7f7e23f33800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d51911030 con 0x558d51910050 2024-04-23T19:12:13.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7e203f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d519174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7e203f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d519174d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40684/0 (socket says 172.21.15.63:40684) 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 learned_addr learned my addr 172.21.15.63:0/1228209910 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d51917a10 con 0x558d51910050 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7e203f6640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d519174d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7e180127e0 tx=0x7f7e1804cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7cfd7fa640 1 -- 172.21.15.63:0/1228209910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e18027040 con 0x558d51910050 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7cfd7fa640 1 -- 172.21.15.63:0/1228209910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e18021c60 con 0x558d51910050 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d51917ca0 con 0x558d51910050 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7cfd7fa640 1 -- 172.21.15.63:0/1228209910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e1802a350 con 0x558d51910050 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.953+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5191a950 con 0x558d51910050 2024-04-23T19:12:13.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.954+0000 7f7cfd7fa640 1 -- 172.21.15.63:0/1228209910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7e18045020 con 0x558d51910050 2024-04-23T19:12:13.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.954+0000 7f7cfd7fa640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7ce8036eb0 0x7f7ce8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.954+0000 7f7cfd7fa640 1 -- 172.21.15.63:0/1228209910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f7e18033030 con 0x558d51910050 2024-04-23T19:12:13.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.954+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7ce8036eb0 0x7f7ce8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.955+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7ce8036eb0 0x7f7ce8039370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e100064f0 tx=0x7f7e1008a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.955+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.955+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d51922f10 0x558d519253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.955+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d51925b20 con 0x558d51922f10 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.955+0000 7f7e1f3f4640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d51922f10 0x558d519253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.955+0000 7f7e1f3f4640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d51922f10 0x558d519253d0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7e14009050 con 0x558d51922f10 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 realm 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d51922360 con 0x558d51922f10 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7e14009050 con 0x558d51922f10 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d51926610 con 0x558d51922f10 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7e14009050 con 0x558d51922f10 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d51926ee0 con 0x558d51922f10 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7e14009050 con 0x558d51922f10 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.956+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d51927280 con 0x558d51922f10 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.957+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7e14009050 con 0x558d51922f10 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.957+0000 7f7e23f33800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:13.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.957+0000 7f7e23f33800 20 searching for the correct realm 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.957+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d51929160 0x558d5192b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.957+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.957+0000 7f7e203f6640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d51929160 0x558d5192b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.957+0000 7f7e203f6640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d51929160 0x558d5192b550 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e203f6640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7e180123a0 0x7f7e18022640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e180123a0 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7e180123a0 0x7f7e18022640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7e180123a0 0x7f7e18022640 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e1fbf5640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7e1009c040 0x7f7e10079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.958+0000 7f7e1f3f4640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7e1009c040 0x7f7e10079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.959+0000 7f7e1f3f4640 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7e1009c040 0x7f7e10079990 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.959+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e14009050 con 0x7f7e1009c040 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.959+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.959+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e14009050 con 0x558d51922f10 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.959+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e180123a0 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.960+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.961+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.961+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.961+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.961+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.961+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.961+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e180123a0 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.962+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.963+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.963+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.963+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.963+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.963+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.963+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.963+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.964+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.964+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.964+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.964+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.964+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.964+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.964+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e180123a0 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e180123a0 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.965+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.966+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.966+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e180123a0 2024-04-23T19:12:13.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.966+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.966+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51929160 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.966+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.966+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e1009c040 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x558d51922f10 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d5192bc00 con 0x7f7e180123a0 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:13.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d51927c00 con 0x558d51922f10 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.967+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d519284a0 con 0x558d51922f10 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5192c080 con 0x7f7e1009c040 2024-04-23T19:12:13.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 zone default found 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 4 Realm: () 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 zonegroup default 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5192c530 con 0x7f7e180123a0 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.968+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5192e2b0 con 0x558d51929160 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e23f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5192e750 con 0x558d51929160 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e23f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e23f33800 20 started sync module instance, tier type = 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e23f33800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.969+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5192d750 con 0x7f7e180123a0 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.970+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.970+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94065447328576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5192d3b0 con 0x7f7e180123a0 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.970+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5192f680 con 0x558d51922f10 2024-04-23T19:12:13.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.971+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94065447328576] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.972+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.972+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94065447334352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5192faa0 con 0x558d51922f10 2024-04-23T19:12:13.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.972+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51930460 con 0x558d51929160 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.973+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94065447334352] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.973+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.973+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94065447335408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5192fa10 con 0x558d51929160 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.973+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51931320 con 0x7f7e180123a0 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.974+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.974+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94065447341680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51931bc0 con 0x7f7e180123a0 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.974+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519326b0 con 0x558d51922f10 2024-04-23T19:12:13.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.974+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94065447335408] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.975+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94065447341680] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.975+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.975+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94065447343888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51930850 con 0x558d51922f10 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.975+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519330c0 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.976+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.976+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94065447343888] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.976+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94065447349264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519334e0 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.976+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51932180 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.977+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94065447349264] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.978+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.978+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94065447350320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51933450 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.978+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51934390 con 0x7f7e1009c040 2024-04-23T19:12:13.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94065447350320] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94065447354016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519342c0 con 0x7f7e1009c040 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 20 add_watcher() i=0 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 20 add_watcher() i=1 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 20 add_watcher() i=2 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 20 add_watcher() i=3 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 20 add_watcher() i=4 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 20 add_watcher() i=5 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.979+0000 7f7e23f33800 20 add_watcher() i=6 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.980+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94065447354016] v32'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.980+0000 7f7e23f33800 20 add_watcher() i=7 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.980+0000 7f7e23f33800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:13.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.980+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d519377b0 con 0x558d51922f10 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.980+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.980+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94065447352816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51934c70 con 0x558d51922f10 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.981+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94065447352816] v32'245 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.981+0000 7f7cd5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.981+0000 7f7e23f33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.981+0000 7f7e23f33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.982+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51937430 con 0x558d51922f10 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.983+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'36 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.983+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519407e0 con 0x7f7e1009c040 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.984+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.984+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519407e0 con 0x558d51929160 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.985+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.985+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51940c90 con 0x558d51922f10 2024-04-23T19:12:13.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.986+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'246 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.986+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51941320 con 0x7f7e180123a0 2024-04-23T19:12:13.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.987+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'77 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.987+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519417d0 con 0x558d51929160 2024-04-23T19:12:13.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.988+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'51 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.988+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519417d0 con 0x558d51922f10 2024-04-23T19:12:13.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.989+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.989+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519417d0 con 0x7f7e1009c040 2024-04-23T19:12:13.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.990+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'51 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.990+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519417d0 con 0x7f7e180123a0 2024-04-23T19:12:13.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.992+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.992+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519419f0 con 0x558d51929160 2024-04-23T19:12:13.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.993+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:13.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.993+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51941de0 con 0x7f7e180123a0 2024-04-23T19:12:13.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.994+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'78 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.994+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51941e70 con 0x7f7e1009c040 2024-04-23T19:12:13.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.995+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'35 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.995+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51941e70 con 0x7f7e180123a0 2024-04-23T19:12:13.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.996+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:13.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.996+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51941f20 con 0x7f7e1009c040 2024-04-23T19:12:13.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.997+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'59 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:13.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.997+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51941f20 con 0x558d51922f10 2024-04-23T19:12:13.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.998+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'247 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:13.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:13.998+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51941f20 con 0x558d51929160 2024-04-23T19:12:14.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.000+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'72 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.000+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x7f7e180123a0 2024-04-23T19:12:14.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.001+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'79 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:14.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.001+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.001+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.001+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51922f10 2024-04-23T19:12:14.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.002+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'37 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:14.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.003+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51922f10 2024-04-23T19:12:14.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.003+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'248 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:14.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.003+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.005+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'73 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.005+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x7f7e180123a0 2024-04-23T19:12:14.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.005+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'39 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:14.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.005+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.006+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.006+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.007+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.007+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.008+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'25 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.008+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.009+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'52 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.009+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x7f7e1009c040 2024-04-23T19:12:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.011+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.011+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.012+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.012+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x7f7e1009c040 2024-04-23T19:12:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.013+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'52 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.013+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x558d51929160 2024-04-23T19:12:14.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.014+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'74 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.014+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x7f7e1009c040 2024-04-23T19:12:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.015+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'60 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.015+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51942420 con 0x7f7e180123a0 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7cd4fe1640 20 reqs_thread_entry: start 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7e23f33800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7e23f33800 20 Filter name: none 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7c4e7fc640 20 reqs_thread_entry: start 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7e23f33800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7e23f33800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.017+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5196d110 con 0x7f7e1009c040 2024-04-23T19:12:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(94 system_user [call out=48b,read 0~349 out=349b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 239+0+472 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e23f33800 20 rados_obj.operate() r=0 bl.length=349 2024-04-23T19:12:14.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e23f33800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x13 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e23f33800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e23f33800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:95 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5196ccd0 con 0x7f7e1009c040 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(95 system_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e23f33800 10 removing key index: system_akey 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.018+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:96 8.0 8:01223cbf:users.keys::system_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5196ccd0 con 0x558d51929160 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.019+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(96 system_akey [delete] v32'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.019+0000 7f7e23f33800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.019+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [notify cookie 94065447593344 in=173b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5196ff80 con 0x558d51922f10 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.019+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== watch-notify(notify (1) cookie 94065447334352 notify 137438953499 ret 0) ==== 203+0+0 (crc 0 0 0) 0x558d51936eb0 con 0x558d51922f10 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.019+0000 7f7cfeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953499 cookie 94065447334352 notifier 4849 bl.length()=161 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.019+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(97 notify.1 [notify cookie 94065447593344 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.019+0000 7f7cfeffd640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f7ce4003d60 con 0x558d51922f10 2024-04-23T19:12:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.020+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(98 notify.1 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.020+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== watch-notify(notify_complete (2) cookie 94065447593344 notify 137438953499 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558d51937080 con 0x558d51922f10 2024-04-23T19:12:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.020+0000 7f7e23f33800 10 removing email index: 2024-04-23T19:12:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.020+0000 7f7e23f33800 10 removing user buckets index 2024-04-23T19:12:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.020+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:99 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5196d9d0 con 0x7f7e1009c040 2024-04-23T19:12:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.022+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(99 system_user.buckets [delete] v32'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.022+0000 7f7e23f33800 10 removing user index: system_user 2024-04-23T19:12:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.022+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:100 8.1e 8:78ce4db0:users.uid::system_user:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5196f540 con 0x7f7e1009c040 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(100 system_user [call,call,delete] v32'5 uv5 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e23f33800 10 removing default.rgw.meta+users.uid+system_user from cache 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e23f33800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94065447589152 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d5196ee00 con 0x7f7e180123a0 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== watch-notify(notify (1) cookie 94065447328576 notify 137438953506 ret 0) ==== 202+0+0 (crc 0 0 0) 0x558d519408b0 con 0x7f7e180123a0 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7cfe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953506 cookie 94065447328576 notifier 4849 bl.length()=160 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(101 notify.0 [notify cookie 94065447589152 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7cfe7fc640 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f7c540032c0 con 0x7f7e180123a0 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(102 notify.0 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:14.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.023+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== watch-notify(notify_complete (2) cookie 94065447589152 notify 137438953506 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f7e100a6060 con 0x7f7e180123a0 2024-04-23T19:12:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.024+0000 7f7cd5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.024+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94065447352816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5193fe30 con 0x558d51922f10 2024-04-23T19:12:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.025+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94065447352816] v32'249 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:14.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.025+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94065447328576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519339f0 con 0x7f7e180123a0 2024-04-23T19:12:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.026+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94065447328576] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.026+0000 7f7e23f33800 20 remove_watcher() i=0 2024-04-23T19:12:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.026+0000 7f7e23f33800 2 removed watcher, disabling cache 2024-04-23T19:12:14.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.026+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94065447334352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519339f0 con 0x558d51922f10 2024-04-23T19:12:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.028+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94065447334352] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.028+0000 7f7e23f33800 20 remove_watcher() i=1 2024-04-23T19:12:14.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.028+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94065447335408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d5192f1d0 con 0x558d51929160 2024-04-23T19:12:14.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.029+0000 7f7e203f6640 1 -- 172.21.15.63:0/1228209910 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94065447335408] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e18029070 con 0x558d51929160 2024-04-23T19:12:14.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.029+0000 7f7e23f33800 20 remove_watcher() i=2 2024-04-23T19:12:14.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.029+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94065447341680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519339f0 con 0x7f7e180123a0 2024-04-23T19:12:14.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.030+0000 7f7e1fbf5640 1 -- 172.21.15.63:0/1228209910 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94065447341680] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e10075030 con 0x7f7e180123a0 2024-04-23T19:12:14.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.030+0000 7f7e23f33800 20 remove_watcher() i=3 2024-04-23T19:12:14.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.030+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94065447343888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51930e70 con 0x558d51922f10 2024-04-23T19:12:14.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.031+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94065447343888] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x558d51922f10 2024-04-23T19:12:14.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.031+0000 7f7e23f33800 20 remove_watcher() i=4 2024-04-23T19:12:14.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.031+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94065447349264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d519339f0 con 0x7f7e1009c040 2024-04-23T19:12:14.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.032+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94065447349264] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.032+0000 7f7e23f33800 20 remove_watcher() i=5 2024-04-23T19:12:14.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.032+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94065447350320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51931710 con 0x7f7e1009c040 2024-04-23T19:12:14.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.033+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94065447350320] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.033+0000 7f7e23f33800 20 remove_watcher() i=6 2024-04-23T19:12:14.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.033+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94065447354016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d51930e70 con 0x7f7e1009c040 2024-04-23T19:12:14.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94065447354016] v32'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7e1402a040 con 0x7f7e1009c040 2024-04-23T19:12:14.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 20 remove_watcher() i=7 2024-04-23T19:12:14.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7e1009c040 msgr2=0x7f7e10079990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f7e1009c040 0x7f7e10079990 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d51922f10 msgr2=0x558d519253d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d51922f10 0x558d519253d0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7e180123a0 msgr2=0x7f7e18022640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7e180123a0 0x7f7e18022640 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d51929160 msgr2=0x558d5192b550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d51929160 0x558d5192b550 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7ce8036eb0 msgr2=0x7f7ce8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7ce8036eb0 0x7f7ce8039370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e100064f0 tx=0x7f7e1008a000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 msgr2=0x558d519174d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e1f3f4640 1 -- 172.21.15.63:0/1228209910 reap_dead start 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d519174d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7e180127e0 tx=0x7f7e1804cc00 comp rx=0 tx=0).stop 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.034+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 shutdown_connections 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/1228209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51910050 0x558d519174d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 >> 172.21.15.63:0/1228209910 conn(0x558d5190c3b0 msgr2=0x558d51909a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 shutdown_connections 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/1228209910 wait complete. 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7cf8036e60 msgr2=0x7f7cf8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7cf8036e60 0x7f7cf8039320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x558d518e0ba0 tx=0x7f7e100063e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51902d70 msgr2=0x558d51903190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51902d70 0x558d51903190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x558d518e1400 tx=0x7f7e180178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 shutdown_connections 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7cf8036e60 0x7f7cf8039320 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3137844200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d51902d70 0x558d51903190 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 >> 172.21.15.63:0/3137844200 conn(0x558d518f6e10 msgr2=0x558d518f7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 shutdown_connections 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3137844200 wait complete. 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d518f1d90 msgr2=0x558d518f4180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d518f1d90 0x558d518f4180 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d517ec260 msgr2=0x558d518ee680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d517ec260 0x558d518ee680 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7d08036eb0 msgr2=0x7f7d08039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7d08036eb0 0x7f7d08039370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7e14000f60 tx=0x7f7e140012b0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 msgr2=0x558d518e16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518e16c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000a470 tx=0x7f7e10003020 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 shutdown_connections 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d518f1d90 0x558d518f4180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d517ec260 0x558d518ee680 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7d08036eb0 0x7f7d08039370 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 --2- 172.21.15.63:0/3670250618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d517eb000 0x558d518e16c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.035+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 >> 172.21.15.63:0/3670250618 conn(0x558d517e6600 msgr2=0x558d5175ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.036+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 shutdown_connections 2024-04-23T19:12:14.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.036+0000 7f7e23f33800 1 -- 172.21.15.63:0/3670250618 wait complete. 2024-04-23T19:12:14.045 ERROR:root:None 2024-04-23T19:12:14.045 ERROR:root:0 2024-04-23T19:12:14.045 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-04-23T19:12:14.045 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-04-23T19:12:14.045 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid a_user --display-name 'True Admin User' --access-key true_admin_akey --secret true_admin_skey --caps user-info-without-keys=read --admin 2024-04-23T19:12:14.098 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:14.099 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:14.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe125533800 1 Processor -- start 2024-04-23T19:12:14.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe125533800 1 -- start start 2024-04-23T19:12:14.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe125533800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563bb3a756e0 0x563bb3a77ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe125533800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563bb39237f0 con 0x563bb3a71bd0 2024-04-23T19:12:14.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe125533800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x563bb3912e20 con 0x563bb3a756e0 2024-04-23T19:12:14.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe1209f4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563bb3a756e0 0x563bb3a77ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe1211f5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x563bb3a71bd0 0x563bb3a73fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56742/0 (socket says 172.21.15.63:56742) 2024-04-23T19:12:14.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.099+0000 7fe1211f5640 1 -- 172.21.15.63:0/533692926 learned_addr learned my addr 172.21.15.63:0/533692926 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:14.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.100+0000 7fe1209f4640 1 -- 172.21.15.63:0/533692926 >> v1:172.21.15.16:6789/0 conn(0x563bb3a71bd0 legacy=0x563bb3a73fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.100+0000 7fe113fff640 1 -- 172.21.15.63:0/533692926 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605229160 0 0) 0x563bb39237f0 con 0x563bb3a71bd0 2024-04-23T19:12:14.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.100+0000 7fe1209f4640 1 -- 172.21.15.63:0/533692926 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb3931b50 con 0x563bb3a756e0 2024-04-23T19:12:14.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.101+0000 7fe1209f4640 1 --2- 172.21.15.63:0/533692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a756e0 0x563bb3a77ad0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe11800a470 tx=0x7fe11801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cf9952ab79eb7b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe113fff640 1 -- 172.21.15.63:0/533692926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe118057070 con 0x563bb3a756e0 2024-04-23T19:12:14.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe113fff640 1 -- 172.21.15.63:0/533692926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe118052570 con 0x563bb3a756e0 2024-04-23T19:12:14.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe113fff640 1 -- 172.21.15.63:0/533692926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe118052870 con 0x563bb3a756e0 2024-04-23T19:12:14.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe125533800 1 -- 172.21.15.63:0/533692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a756e0 msgr2=0x563bb3a77ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe125533800 1 --2- 172.21.15.63:0/533692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a756e0 0x563bb3a77ad0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe11800a470 tx=0x7fe11801ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:14.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe125533800 1 -- 172.21.15.63:0/533692926 shutdown_connections 2024-04-23T19:12:14.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe125533800 1 --2- 172.21.15.63:0/533692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a756e0 0x563bb3a77ad0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe125533800 1 -- 172.21.15.63:0/533692926 >> 172.21.15.63:0/533692926 conn(0x563bb3a6b5a0 msgr2=0x563bb3a6d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe125533800 1 -- 172.21.15.63:0/533692926 shutdown_connections 2024-04-23T19:12:14.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.102+0000 7fe125533800 1 -- 172.21.15.63:0/533692926 wait complete. 2024-04-23T19:12:14.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.115+0000 7fe125533800 1 Processor -- start 2024-04-23T19:12:14.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.115+0000 7fe125533800 1 -- start start 2024-04-23T19:12:14.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.115+0000 7fe125533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3a71510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.115+0000 7fe125533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bb39bc690 con 0x563bb3a71bd0 2024-04-23T19:12:14.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.115+0000 7fe1211f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3a71510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.115+0000 7fe1211f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3a71510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40702/0 (socket says 172.21.15.63:40702) 2024-04-23T19:12:14.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.115+0000 7fe1211f5640 1 -- 172.21.15.63:0/2177983871 learned_addr learned my addr 172.21.15.63:0/2177983871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.116+0000 7fe1211f5640 1 -- 172.21.15.63:0/2177983871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb3a6fad0 con 0x563bb3a71bd0 2024-04-23T19:12:14.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.116+0000 7fe1211f5640 1 --2- 172.21.15.63:0/2177983871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3a71510 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b210 tx=0x7fe11400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c652537b773fec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.116+0000 7fe00f606640 1 -- 172.21.15.63:0/2177983871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe114008070 con 0x563bb3a71bd0 2024-04-23T19:12:14.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.116+0000 7fe00f606640 1 -- 172.21.15.63:0/2177983871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11400d500 con 0x563bb3a71bd0 2024-04-23T19:12:14.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.116+0000 7fe00f606640 1 -- 172.21.15.63:0/2177983871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11400d840 con 0x563bb3a71bd0 2024-04-23T19:12:14.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 -- 172.21.15.63:0/2177983871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 msgr2=0x563bb3a71510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 --2- 172.21.15.63:0/2177983871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3a71510 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b210 tx=0x7fe11400b6d0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 -- 172.21.15.63:0/2177983871 shutdown_connections 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 --2- 172.21.15.63:0/2177983871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3a71510 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 -- 172.21.15.63:0/2177983871 >> 172.21.15.63:0/2177983871 conn(0x563bb3a6b5a0 msgr2=0x563bb3b5dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 -- 172.21.15.63:0/2177983871 shutdown_connections 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 -- 172.21.15.63:0/2177983871 wait complete. 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 Processor -- start 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.117+0000 7fe125533800 1 -- start start 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.118+0000 7fe125533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3b6bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.118+0000 7fe125533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bb3a7a320 con 0x563bb3a71bd0 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.118+0000 7fe1211f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3b6bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.118+0000 7fe1211f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3b6bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40704/0 (socket says 172.21.15.63:40704) 2024-04-23T19:12:14.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.118+0000 7fe1211f5640 1 -- 172.21.15.63:0/629864328 learned_addr learned my addr 172.21.15.63:0/629864328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.118+0000 7fe1211f5640 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb3b6c480 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.118+0000 7fe1211f5640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3b6bf40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b210 tx=0x7fe114000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.119+0000 7fe00de03640 1 -- 172.21.15.63:0/629864328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11400f950 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.119+0000 7fe00de03640 1 -- 172.21.15.63:0/629864328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11400faf0 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.119+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563bb3b6c710 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.119+0000 7fe00de03640 1 -- 172.21.15.63:0/629864328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11400fdd0 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.119+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb3b6f3c0 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.119+0000 7fe00de03640 1 -- 172.21.15.63:0/629864328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe114017040 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe00de03640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036e60 0x7fe008039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe00de03640 1 -- 172.21.15.63:0/629864328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe114015020 con 0x563bb3a71bd0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe1209f4640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036e60 0x7fe008039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3a756e0 0x563bb3b73180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3b736c0 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe1219f6640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3a756e0 0x563bb3b73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe1209f4640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036e60 0x7fe008039320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe118054000 tx=0x7fe11804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.120+0000 7fe1219f6640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3a756e0 0x563bb3b73180 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe1219f6640 1 -- 172.21.15.63:0/629864328 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3b70750 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe1219f6640 1 -- 172.21.15.63:0/629864328 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3b75530 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe1219f6640 1 -- 172.21.15.63:0/629864328 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3b74870 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe1219f6640 1 -- 172.21.15.63:0/629864328 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3a756e0 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563bb3b768e0 0x563bb3b78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.121+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3b79290 con 0x563bb3b768e0 2024-04-23T19:12:14.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.122+0000 7fe1211f5640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563bb3b768e0 0x563bb3b78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.122+0000 7fe1211f5640 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563bb3b768e0 0x563bb3b78cd0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.122+0000 7fe1211f5640 1 -- 172.21.15.63:0/629864328 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe11402d070 con 0x563bb3b768e0 2024-04-23T19:12:14.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe125533800 1 Processor -- start 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe125533800 1 -- start start 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe125533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b87690 0x563bb3b87ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe125533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bb3962e30 con 0x563bb3b87690 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe1219f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b87690 0x563bb3b87ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe1219f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b87690 0x563bb3b87ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40718/0 (socket says 172.21.15.63:40718) 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe1219f6640 1 -- 172.21.15.63:0/3644602288 learned_addr learned my addr 172.21.15.63:0/3644602288 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.123+0000 7fe1219f6640 1 -- 172.21.15.63:0/3644602288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb3b87ff0 con 0x563bb3b87690 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.124+0000 7fe1219f6640 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b87690 0x563bb3b87ab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x563bb3b73aa0 tx=0x7fe11c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.124+0000 7fe006ffd640 1 -- 172.21.15.63:0/3644602288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11c013440 con 0x563bb3b87690 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.124+0000 7fe006ffd640 1 -- 172.21.15.63:0/3644602288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c0135e0 con 0x563bb3b87690 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.124+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563bb3b88a60 con 0x563bb3b87690 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.124+0000 7fe006ffd640 1 -- 172.21.15.63:0/3644602288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11c0138c0 con 0x563bb3b87690 2024-04-23T19:12:14.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.124+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb3b88f60 con 0x563bb3b87690 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe006ffd640 1 -- 172.21.15.63:0/3644602288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe11c011040 con 0x563bb3b87690 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe006ffd640 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdff8036eb0 0x7fdff8039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe006ffd640 1 -- 172.21.15.63:0/3644602288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe11c01d070 con 0x563bb3b87690 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe1211f5640 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdff8036eb0 0x7fdff8039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe125533800 1 Processor -- start 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe125533800 1 -- start start 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe1211f5640 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdff8036eb0 0x7fdff8039370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b6f0 tx=0x7fe11404ba00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe125533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.125+0000 7fe125533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bb3a6fe50 con 0x563bb3b948e0 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.126+0000 7fe1219f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.126+0000 7fe1219f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b94d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40730/0 (socket says 172.21.15.63:40730) 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.126+0000 7fe1219f6640 1 -- 172.21.15.63:0/1252823782 learned_addr learned my addr 172.21.15.63:0/1252823782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.126+0000 7fe1219f6640 1 -- 172.21.15.63:0/1252823782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb3b952d0 con 0x563bb3b948e0 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.126+0000 7fe1219f6640 1 --2- 172.21.15.63:0/1252823782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b94d00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe11c04cc90 tx=0x7fe11c02b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4e39a920226ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.126+0000 7fe004ff9640 1 -- 172.21.15.63:0/1252823782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11c02b3a0 con 0x563bb3b948e0 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe004ff9640 1 -- 172.21.15.63:0/1252823782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c02b540 con 0x563bb3b948e0 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe004ff9640 1 -- 172.21.15.63:0/1252823782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11c021bc0 con 0x563bb3b948e0 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 -- 172.21.15.63:0/1252823782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 msgr2=0x563bb3b94d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 --2- 172.21.15.63:0/1252823782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b94d00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe11c04cc90 tx=0x7fe11c02b8e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 -- 172.21.15.63:0/1252823782 shutdown_connections 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 --2- 172.21.15.63:0/1252823782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b94d00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 -- 172.21.15.63:0/1252823782 >> 172.21.15.63:0/1252823782 conn(0x563bb3b90c40 msgr2=0x563bb3b8eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 -- 172.21.15.63:0/1252823782 shutdown_connections 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 -- 172.21.15.63:0/1252823782 wait complete. 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 Processor -- start 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.127+0000 7fe125533800 1 -- start start 2024-04-23T19:12:14.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe125533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b9bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe125533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563bb3b95a40 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe1219f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b9bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe1219f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b9bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40742/0 (socket says 172.21.15.63:40742) 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 learned_addr learned my addr 172.21.15.63:0/4022742732 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb3b9c420 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe1219f6640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b9bee0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe11c02b920 tx=0x7fe11c04cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fdff6ffd640 1 -- 172.21.15.63:0/4022742732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11c013c70 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fdff6ffd640 1 -- 172.21.15.63:0/4022742732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c013e10 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563bb3b9c6b0 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fdff6ffd640 1 -- 172.21.15.63:0/4022742732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe11c02a480 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.128+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb3b9f360 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.129+0000 7fdff6ffd640 1 -- 172.21.15.63:0/4022742732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fe11c02a620 con 0x563bb3b948e0 2024-04-23T19:12:14.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.129+0000 7fdff6ffd640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdfec036eb0 0x7fdfec039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.129+0000 7fdff6ffd640 1 -- 172.21.15.63:0/4022742732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fe11c032070 con 0x563bb3b948e0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.129+0000 7fe1211f5640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdfec036eb0 0x7fdfec039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.130+0000 7fe1211f5640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdfec036eb0 0x7fdfec039370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe11400f640 tx=0x7fe114034000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.130+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.130+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3ba7a30 0x563bb3ba9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.130+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3baa640 con 0x563bb3ba7a30 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.130+0000 7fe1209f4640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3ba7a30 0x563bb3ba9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.130+0000 7fe1209f4640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3ba7a30 0x563bb3ba9ef0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 realm 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3baaa10 con 0x563bb3ba7a30 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bab2e0 con 0x563bb3ba7a30 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bab680 con 0x563bb3ba7a30 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.131+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3babed0 con 0x563bb3ba7a30 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe125533800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe125533800 20 searching for the correct realm 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563bb3badc40 0x563bb3bb0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe1219f6640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563bb3badc40 0x563bb3bb0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe1219f6640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563bb3badc40 0x563bb3bb0030 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe1219f6640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe11c03b6c0 0x7fe11c046420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.132+0000 7fe1211f5640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe11c03b6c0 0x7fe11c046420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.133+0000 7fe1211f5640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe11c03b6c0 0x7fe11c046420 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.133+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11402d070 con 0x7fe11c03b6c0 2024-04-23T19:12:14.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.133+0000 7fe1211f5640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe11404d890 0x7fe11404a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.133+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.133+0000 7fe1209f4640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe11404d890 0x7fe11404a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.134+0000 7fe1209f4640 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe11404d890 0x7fe11404a0e0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.134+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.134+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.134+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.134+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.134+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.134+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11402d070 con 0x7fe11c03b6c0 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.135+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.136+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.136+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.136+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.136+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.136+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.136+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.137+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.138+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.139+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.140+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.140+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.140+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.140+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.140+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.140+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.141+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.141+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3badc40 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.141+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.141+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11404d890 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.141+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.141+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x563bb3ba7a30 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb05c0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:14.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bac850 con 0x563bb3ba7a30 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bad0f0 con 0x563bb3ba7a30 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.142+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb0a40 con 0x7fe11404d890 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 20 zone default found 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 4 Realm: () 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 20 zonegroup default 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb0ef0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.143+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb2c70 con 0x563bb3badc40 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb3150 con 0x563bb3badc40 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe125533800 20 started sync module instance, tier type = 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe125533800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.144+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb2110 con 0x7fe11c03b6c0 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.145+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.145+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94814418445568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb3a60 con 0x7fe11c03b6c0 2024-04-23T19:12:14.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.145+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb42b0 con 0x563bb3ba7a30 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.146+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94814418445568] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.146+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.146+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94814418449840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb4370 con 0x563bb3ba7a30 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.146+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb4e70 con 0x563bb3badc40 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.147+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94814418449840] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.147+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.148+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94814418454880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb4e30 con 0x563bb3badc40 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.148+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb58f0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.149+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94814418454880] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.149+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.149+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94814418451968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb5ce0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.149+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb6730 con 0x563bb3ba7a30 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.150+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94814418451968] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.150+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.150+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94814418461216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb6b20 con 0x563bb3ba7a30 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.150+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb7540 con 0x7fe11404d890 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.151+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.151+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94814418464816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb7500 con 0x7fe11404d890 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.151+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb8500 con 0x7fe11404d890 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.151+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94814418461216] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.152+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94814418464816] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.152+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.152+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94814418457600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb7780 con 0x7fe11404d890 2024-04-23T19:12:14.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.152+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb89b0 con 0x7fe11404d890 2024-04-23T19:12:14.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94814418457600] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94814418468864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb89b0 con 0x7fe11404d890 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 20 add_watcher() i=0 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 20 add_watcher() i=1 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 20 add_watcher() i=2 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 20 add_watcher() i=3 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 20 add_watcher() i=4 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 20 add_watcher() i=5 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.153+0000 7fe125533800 20 add_watcher() i=6 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.154+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94814418468864] v32'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.154+0000 7fe125533800 20 add_watcher() i=7 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.154+0000 7fe125533800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:14.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.154+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bbbdb0 con 0x563bb3ba7a30 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.155+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.155+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94814418470608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb9270 con 0x563bb3ba7a30 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94814418470608] v32'250 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fdfd77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fe125533800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fe125533800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fe125533800 5 note: GC not initialized 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fdfd67e4640 20 reqs_thread_entry: start 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fe125533800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.156+0000 7fe125533800 20 Filter name: none 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fdf47fff640 20 reqs_thread_entry: start 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-04-23T19:12:14.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:62 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bef250 con 0x563bb3badc40 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(62 a_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 234+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x0 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : miss 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:63 8.15 8:afe09604:users.keys::true_admin_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bed9f0 con 0x563bb3badc40 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(63 true_admin_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 201+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x0 2024-04-23T19:12:14.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 adding default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.157+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 8.14 8:2efb937b:users.uid::a_user:head [delete,create,call version.set in=58b,writefull 0~363 in=363b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bee290 con 0x563bb3badc40 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.158+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(64 a_user [delete,create,call,writefull 0~363] v32'1 uv1 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.158+0000 7fe125533800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.158+0000 7fe125533800 10 moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.158+0000 7fe125533800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.158+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94814418689504 in=554b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bf26e0 con 0x563bb3badc40 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== watch-notify(notify (1) cookie 94814418454880 notify 137438953481 ret 0) ==== 584+0+0 (crc 0 0 0) 0x563bb3beeca0 con 0x563bb3badc40 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe004ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953481 cookie 94814418454880 notifier 4856 bl.length()=542 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe004ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe004ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-23T19:12:14.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(65 notify.2 [notify cookie 94814418689504 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe004ff9640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fdfe4003ea0 con 0x563bb3badc40 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== watch-notify(notify_complete (2) cookie 94814418689504 notify 137438953481 ret 0) ==== 42+0+48 (crc 0 0 0) 0x563bb3b9c420 con 0x563bb3badc40 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.159+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 8.15 8:afe09604:users.keys::true_admin_akey:head [delete,create,writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bee240 con 0x563bb3badc40 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.160+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(67 true_admin_akey [delete,create,writefull 0~10] v32'3 uv3 ondisk = 0) ==== 243+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.160+0000 7fe125533800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.160+0000 7fe125533800 10 moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.160+0000 7fe125533800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.160+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94814418689424 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bf26e0 con 0x7fe11404d890 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== watch-notify(notify (1) cookie 94814418464816 notify 137438953516 ret 0) ==== 217+0+0 (crc 0 0 0) 0x563bb3931b50 con 0x7fe11404d890 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe004ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953516 cookie 94814418464816 notifier 4856 bl.length()=175 2024-04-23T19:12:14.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe004ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-04-23T19:12:14.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe004ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-04-23T19:12:14.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe004ff9640 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fdfe4003ea0 con 0x7fe11404d890 2024-04-23T19:12:14.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(68 notify.5 [notify cookie 94814418689424 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== watch-notify(notify_complete (2) cookie 94814418689424 notify 137438953516 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fe11805dd40 con 0x7fe11404d890 2024-04-23T19:12:14.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.161+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.173 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"a_user","display_name":"True Admin User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"a_user","access_key":"true_admin_akey","secret_key":"true_admin_skey","active":true,"create_date":"2024-04-23T19:12:14.158040Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","admin":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:14.158019Z","tags":[],"group_ids":[]} 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.162+0000 7fdfd77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.162+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94814418470608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bbb8a0 con 0x563bb3ba7a30 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.163+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94814418470608] v32'251 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.163+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94814418445568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb86a0 con 0x7fe11c03b6c0 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.164+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94814418445568] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.164+0000 7fe125533800 20 remove_watcher() i=0 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.164+0000 7fe125533800 2 removed watcher, disabling cache 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.164+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94814418449840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb2500 con 0x563bb3ba7a30 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.165+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94814418449840] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.165+0000 7fe125533800 20 remove_watcher() i=1 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.165+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94814418454880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb2500 con 0x563bb3badc40 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.166+0000 7fe1219f6640 1 -- 172.21.15.63:0/4022742732 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94814418454880] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11c00a4a0 con 0x563bb3badc40 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.166+0000 7fe125533800 20 remove_watcher() i=2 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.166+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94814418451968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb4960 con 0x7fe11c03b6c0 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.167+0000 7fe1211f5640 1 -- 172.21.15.63:0/4022742732 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94814418451968] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11405c040 con 0x7fe11c03b6c0 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.167+0000 7fe125533800 20 remove_watcher() i=3 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.167+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94814418461216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb3e00 con 0x563bb3ba7a30 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.169+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94814418461216] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x563bb3ba7a30 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.169+0000 7fe125533800 20 remove_watcher() i=4 2024-04-23T19:12:14.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.169+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94814418464816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb4960 con 0x7fe11404d890 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.170+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94814418464816] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.170+0000 7fe125533800 20 remove_watcher() i=5 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.170+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94814418457600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb6220 con 0x7fe11404d890 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.171+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94814418457600] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.171+0000 7fe125533800 20 remove_watcher() i=6 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.171+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94814418468864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563bb3bb4960 con 0x7fe11404d890 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94814418468864] v32'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe11805a040 con 0x7fe11404d890 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 20 remove_watcher() i=7 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe11404d890 msgr2=0x7fe11404a0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fe11404d890 0x7fe11404a0e0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3ba7a30 msgr2=0x563bb3ba9ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3ba7a30 0x563bb3ba9ef0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe11c03b6c0 msgr2=0x7fe11c046420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fe11c03b6c0 0x7fe11c046420 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563bb3badc40 msgr2=0x563bb3bb0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563bb3badc40 0x563bb3bb0030 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdfec036eb0 msgr2=0x7fdfec039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdfec036eb0 0x7fdfec039370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe11400f640 tx=0x7fe114034000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 msgr2=0x563bb3b9bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe1209f4640 1 -- 172.21.15.63:0/4022742732 reap_dead start 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b9bee0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe11c02b920 tx=0x7fe11c04cc20 comp rx=0 tx=0).stop 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 shutdown_connections 2024-04-23T19:12:14.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 --2- 172.21.15.63:0/4022742732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b948e0 0x563bb3b9bee0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 >> 172.21.15.63:0/4022742732 conn(0x563bb3b90c40 msgr2=0x563bb3b8e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 shutdown_connections 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.172+0000 7fe125533800 1 -- 172.21.15.63:0/4022742732 wait complete. 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdff8036eb0 msgr2=0x7fdff8039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdff8036eb0 0x7fdff8039370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b6f0 tx=0x7fe11404ba00 comp rx=0 tx=0).stop 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b87690 msgr2=0x563bb3b87ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b87690 0x563bb3b87ab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x563bb3b73aa0 tx=0x7fe11c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 shutdown_connections 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fdff8036eb0 0x7fdff8039370 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/3644602288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3b87690 0x563bb3b87ab0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 >> 172.21.15.63:0/3644602288 conn(0x563bb3b7b730 msgr2=0x563bb3b7bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 shutdown_connections 2024-04-23T19:12:14.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/3644602288 wait complete. 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563bb3b768e0 msgr2=0x563bb3b78cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563bb3b768e0 0x563bb3b78cd0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3a756e0 msgr2=0x563bb3b73180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3a756e0 0x563bb3b73180 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036e60 msgr2=0x7fe008039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036e60 0x7fe008039320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe118054000 tx=0x7fe11804e000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 msgr2=0x563bb3b6bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3b6bf40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b210 tx=0x7fe114000ed0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 shutdown_connections 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563bb3b768e0 0x563bb3b78cd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563bb3a756e0 0x563bb3b73180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fe008036e60 0x7fe008039320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 --2- 172.21.15.63:0/629864328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563bb3a71bd0 0x563bb3b6bf40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 >> 172.21.15.63:0/629864328 conn(0x563bb3a6b5a0 msgr2=0x563bb3a703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 shutdown_connections 2024-04-23T19:12:14.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.173+0000 7fe125533800 1 -- 172.21.15.63:0/629864328 wait complete. 2024-04-23T19:12:14.186 DEBUG:tasks.util.rgw: json result: {'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-04-23T19:12:14.158040Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:14.158019Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.186 ERROR:root:{'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-04-23T19:12:14.158040Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:14.158019Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.186 ERROR:root:0 2024-04-23T19:12:14.186 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:14.191 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:14.191 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'a_user'] 2024-04-23T19:12:14.191 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'a_user'] 2024-04-23T19:12:14.191 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid a_user 2024-04-23T19:12:14.235 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:14.235 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:14.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.235+0000 7f6c9f533800 1 Processor -- start 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.235+0000 7f6c9f533800 1 -- start start 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9f533800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55da3f694a60 0x55da3f694e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9f533800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55da3f5487f0 con 0x55da3f6953c0 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9f533800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55da3f537e20 con 0x55da3f694a60 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9b1f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55da3f694a60 0x55da3f694e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9b1f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55da3f694a60 0x55da3f694e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40746/0 (socket says 172.21.15.63:40746) 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9a9f4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55da3f6953c0 0x55da3f695870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56756/0 (socket says 172.21.15.63:56756) 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/3789677355 learned_addr learned my addr 172.21.15.63:0/3789677355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/3789677355 >> v1:172.21.15.16:6789/0 conn(0x55da3f6953c0 legacy=0x55da3f695870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.236+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/3789677355 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3f556b50 con 0x55da3f694a60 2024-04-23T19:12:14.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.238+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/3789677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f694e80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00a520 tx=0x7f6c8c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40adce61a30525a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.238+0000 7f6c9a1f3640 1 -- 172.21.15.63:0/3789677355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8c057070 con 0x55da3f694a60 2024-04-23T19:12:14.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.238+0000 7f6c9a1f3640 1 -- 172.21.15.63:0/3789677355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c052570 con 0x55da3f694a60 2024-04-23T19:12:14.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.238+0000 7f6c9a1f3640 1 -- 172.21.15.63:0/3789677355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8c052870 con 0x55da3f694a60 2024-04-23T19:12:14.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.239+0000 7f6c9f533800 1 -- 172.21.15.63:0/3789677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 msgr2=0x55da3f694e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.239+0000 7f6c9f533800 1 --2- 172.21.15.63:0/3789677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f694e80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00a520 tx=0x7f6c8c01d010 comp rx=0 tx=0).stop 2024-04-23T19:12:14.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.239+0000 7f6c9f533800 1 -- 172.21.15.63:0/3789677355 shutdown_connections 2024-04-23T19:12:14.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.239+0000 7f6c9f533800 1 --2- 172.21.15.63:0/3789677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f694e80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.239+0000 7f6c9f533800 1 -- 172.21.15.63:0/3789677355 >> 172.21.15.63:0/3789677355 conn(0x55da3f690600 msgr2=0x55da3f692a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.239+0000 7f6c9f533800 1 -- 172.21.15.63:0/3789677355 shutdown_connections 2024-04-23T19:12:14.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.239+0000 7f6c9f533800 1 -- 172.21.15.63:0/3789677355 wait complete. 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9f533800 1 Processor -- start 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9f533800 1 -- start start 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9f533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f789600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9f533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da3f5e16f0 con 0x55da3f694a60 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9b1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f789600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9b1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f789600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40752/0 (socket says 172.21.15.63:40752) 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/1707579648 learned_addr learned my addr 172.21.15.63:0/1707579648 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.252+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/1707579648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3f789bd0 con 0x55da3f694a60 2024-04-23T19:12:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/1707579648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f789600 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c01d050 tx=0x7f6c8c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1583df60b36d8cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6b8960a640 1 -- 172.21.15.63:0/1707579648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8c057040 con 0x55da3f694a60 2024-04-23T19:12:14.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6b8960a640 1 -- 172.21.15.63:0/1707579648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c052850 con 0x55da3f694a60 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6b8960a640 1 -- 172.21.15.63:0/1707579648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8c052b70 con 0x55da3f694a60 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6c9f533800 1 -- 172.21.15.63:0/1707579648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 msgr2=0x55da3f789600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6c9f533800 1 --2- 172.21.15.63:0/1707579648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f789600 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c01d050 tx=0x7f6c8c04c000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6c9f533800 1 -- 172.21.15.63:0/1707579648 shutdown_connections 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.253+0000 7f6c9f533800 1 --2- 172.21.15.63:0/1707579648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f789600 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9f533800 1 -- 172.21.15.63:0/1707579648 >> 172.21.15.63:0/1707579648 conn(0x55da3f690600 msgr2=0x55da3f784f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9f533800 1 -- 172.21.15.63:0/1707579648 shutdown_connections 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9f533800 1 -- 172.21.15.63:0/1707579648 wait complete. 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9f533800 1 Processor -- start 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9f533800 1 -- start start 2024-04-23T19:12:14.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9f533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f790fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9f533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da3f6961c0 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9b1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f790fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9b1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f790fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40756/0 (socket says 172.21.15.63:40756) 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.254+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/2888614839 learned_addr learned my addr 172.21.15.63:0/2888614839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.255+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3f791500 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.255+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f790fc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005930 tx=0x7f6c8c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.255+0000 7f6b87e07640 1 -- 172.21.15.63:0/2888614839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8c073070 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.255+0000 7f6b87e07640 1 -- 172.21.15.63:0/2888614839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c05d650 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.255+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da3f791790 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.255+0000 7f6b87e07640 1 -- 172.21.15.63:0/2888614839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8c057070 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.255+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3f794440 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6b87e07640 1 -- 172.21.15.63:0/2888614839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6c8c076070 con 0x55da3f694a60 2024-04-23T19:12:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6b87e07640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b80036e60 0x7f6b80039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6b87e07640 1 -- 172.21.15.63:0/2888614839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f6c8c08e650 con 0x55da3f694a60 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6c9a9f4640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b80036e60 0x7f6b80039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f6953c0 0x55da3f798200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7987d0 con 0x55da3f6953c0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f6953c0 0x55da3f798200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.256+0000 7f6c9a9f4640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b80036e60 0x7f6b80039320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000f60 tx=0x7f6c90001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.257+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f6953c0 0x55da3f798200 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.257+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/2888614839 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f6953c0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.257+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7957d0 con 0x55da3f6953c0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.257+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/2888614839 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f6953c0 2024-04-23T19:12:14.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.257+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f79a640 con 0x55da3f6953c0 2024-04-23T19:12:14.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.257+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/2888614839 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f6953c0 2024-04-23T19:12:14.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.258+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f799980 con 0x55da3f6953c0 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.258+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/2888614839 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f6953c0 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.258+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55da3f79b9f0 0x55da3f79dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.258+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f79e360 con 0x55da3f79b9f0 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.258+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55da3f79b9f0 0x55da3f79dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.259+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55da3f79b9f0 0x55da3f79dde0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.259+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/2888614839 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6c8c075030 con 0x55da3f79b9f0 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.259+0000 7f6c9f533800 1 Processor -- start 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9f533800 1 -- start start 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9f533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7ac780 0x55da3f7acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9f533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da3f587e30 con 0x55da3f7ac780 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9b9f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7ac780 0x55da3f7acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9b9f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7ac780 0x55da3f7acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40768/0 (socket says 172.21.15.63:40768) 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/2356922891 learned_addr learned my addr 172.21.15.63:0/2356922891 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/2356922891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3f7ad0e0 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.260+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7ac780 0x55da3f7acba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55da3f798bb0 tx=0x7f6c940178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.261+0000 7f6b85602640 1 -- 172.21.15.63:0/2356922891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c94013440 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.261+0000 7f6b85602640 1 -- 172.21.15.63:0/2356922891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c940135e0 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.261+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da3f7adb50 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.261+0000 7f6b85602640 1 -- 172.21.15.63:0/2356922891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c940138c0 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.261+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3f7ae050 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.261+0000 7f6b85602640 1 -- 172.21.15.63:0/2356922891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6c94011040 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6b85602640 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b74036eb0 0x7f6b74039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6b85602640 1 -- 172.21.15.63:0/2356922891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f6c9401d070 con 0x55da3f7ac780 2024-04-23T19:12:14.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b74036eb0 0x7f6b74039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9f533800 1 Processor -- start 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9f533800 1 -- start start 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b74036eb0 0x7f6b74039370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c052a70 tx=0x7f6c8c06ff20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9f533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9f533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da3f789e90 con 0x55da3f7b9a90 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9b9f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9b9f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7b9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40776/0 (socket says 172.21.15.63:40776) 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.262+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/1904497932 learned_addr learned my addr 172.21.15.63:0/1904497932 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.263+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/1904497932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3f7ba480 con 0x55da3f7b9a90 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.263+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/1904497932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7b9eb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6c9404ccf0 tx=0x7f6c9402b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5be72d37dd23340 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.263+0000 7f6b7effd640 1 -- 172.21.15.63:0/1904497932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c9402bdc0 con 0x55da3f7b9a90 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.263+0000 7f6b7effd640 1 -- 172.21.15.63:0/1904497932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9402b3a0 con 0x55da3f7b9a90 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.263+0000 7f6b7effd640 1 -- 172.21.15.63:0/1904497932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c94021a50 con 0x55da3f7b9a90 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.263+0000 7f6c9f533800 1 -- 172.21.15.63:0/1904497932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 msgr2=0x55da3f7b9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.263+0000 7f6c9f533800 1 --2- 172.21.15.63:0/1904497932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7b9eb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6c9404ccf0 tx=0x7f6c9402b8e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 -- 172.21.15.63:0/1904497932 shutdown_connections 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 --2- 172.21.15.63:0/1904497932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7b9eb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 -- 172.21.15.63:0/1904497932 >> 172.21.15.63:0/1904497932 conn(0x55da3f7b5da0 msgr2=0x55da3f7b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 -- 172.21.15.63:0/1904497932 shutdown_connections 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 -- 172.21.15.63:0/1904497932 wait complete. 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 Processor -- start 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 -- start start 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9f533800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da3f7babd0 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9b9f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9b9f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7c0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40782/0 (socket says 172.21.15.63:40782) 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.264+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 learned_addr learned my addr 172.21.15.63:0/623903279 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.265+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3f7c1530 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.265+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7c0ff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6c9402aff0 tx=0x7f6c9404cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.265+0000 7f6b7d7fa640 1 -- 172.21.15.63:0/623903279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c94013b90 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.265+0000 7f6b7d7fa640 1 -- 172.21.15.63:0/623903279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c94013d30 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.265+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da3f7c17c0 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.265+0000 7f6b7d7fa640 1 -- 172.21.15.63:0/623903279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c9402a4a0 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.265+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3f7c4470 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.266+0000 7f6b7d7fa640 1 -- 172.21.15.63:0/623903279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f6c9402a640 con 0x55da3f7b9a90 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.266+0000 7f6b7d7fa640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b64036eb0 0x7f6b64039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.266+0000 7f6b7d7fa640 1 -- 172.21.15.63:0/623903279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f6c94032070 con 0x55da3f7b9a90 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.266+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b64036eb0 0x7f6b64039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.266+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b64036eb0 0x7f6b64039370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c006b30 tx=0x7f6c8c091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f7ccae0 0x55da3f7cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7cf6f0 con 0x55da3f7ccae0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9a9f4640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f7ccae0 0x55da3f7cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9a9f4640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f7ccae0 0x55da3f7cefa0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9f533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9f533800 20 realm 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.267+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7cbf30 con 0x55da3f7ccae0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d01e0 con 0x55da3f7ccae0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d0ab0 con 0x55da3f7ccae0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d0e50 con 0x55da3f7ccae0 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 20 searching for the correct realm 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55da3f7d2d30 0x55da3f7d5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.268+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.269+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55da3f7d2d30 0x55da3f7d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.269+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55da3f7d2d30 0x55da3f7d5160 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.269+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.269+0000 7f6c9b9f6640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f6c9401a9b0 0x7f6c94046860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.269+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c9401a9b0 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.269+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f6c9401a9b0 0x7f6c94046860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.270+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f6c9401a9b0 0x7f6c94046860 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.270+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.270+0000 7f6c9b1f5640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6c8c078060 0x7f6c8c08e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.270+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.270+0000 7f6c9a9f4640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6c8c078060 0x7f6c8c08e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.270+0000 7f6c9a9f4640 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6c8c078060 0x7f6c8c08e9b0 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.271+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.271+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.271+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.271+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.271+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.271+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c9401a9b0 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.272+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.273+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.273+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.273+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.273+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.273+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.273+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c9401a9b0 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.274+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.275+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c9401a9b0 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.276+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c9401a9b0 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.277+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.277+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.277+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.277+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.277+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.277+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c9401a9b0 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7d2d30 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c8c078060 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x55da3f7ccae0 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.278+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5830 con 0x7f6c9401a9b0 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d17a0 con 0x55da3f7ccae0 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d2040 con 0x55da3f7ccae0 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.279+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5cb0 con 0x7f6c8c078060 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 zone default found 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 4 Realm: () 2024-04-23T19:12:14.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 zonegroup default 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d5cf0 con 0x7f6c9401a9b0 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d7fc0 con 0x55da3f7d2d30 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d8360 con 0x55da3f7d2d30 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 started sync module instance, tier type = 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.280+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d7450 con 0x7f6c9401a9b0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.282+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.282+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94395856418928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d8c00 con 0x7f6c9401a9b0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.282+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d9450 con 0x55da3f7ccae0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.283+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94395856418928] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.283+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.283+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94395856422736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d9450 con 0x55da3f7ccae0 2024-04-23T19:12:14.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.283+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d9f70 con 0x55da3f7d2d30 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.284+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94395856422736] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.284+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.284+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94395856427584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d9450 con 0x55da3f7d2d30 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.284+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7da9f0 con 0x7f6c9401a9b0 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.285+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94395856427584] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.285+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.285+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94395856424864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d93c0 con 0x7f6c9401a9b0 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.285+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7db8e0 con 0x55da3f7ccae0 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.286+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.287+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94395856434224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7db8b0 con 0x55da3f7ccae0 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.287+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dc8c0 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.287+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94395856424864] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.288+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.288+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94395856438128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dc880 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.288+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dd2d0 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.288+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94395856434224] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.289+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94395856438128] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.289+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.289+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94395856432656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dce20 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.289+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dd710 con 0x7f6c8c078060 2024-04-23T19:12:14.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94395856440864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7ddda0 con 0x7f6c8c078060 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94395856432656] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 20 add_watcher() i=0 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 20 add_watcher() i=1 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 20 add_watcher() i=2 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 20 add_watcher() i=3 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 20 add_watcher() i=4 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 20 add_watcher() i=5 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.290+0000 7f6c9f533800 20 add_watcher() i=6 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.291+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94395856440864] v32'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.291+0000 7f6c9f533800 20 add_watcher() i=7 2024-04-23T19:12:14.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.291+0000 7f6c9f533800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.291+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7e0cc0 con 0x55da3f7ccae0 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.291+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.291+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94395856443360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7de0b0 con 0x55da3f7ccae0 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.292+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94395856443360] v32'252 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.292+0000 7f6b51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.293+0000 7f6c9f533800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.293+0000 7f6c9f533800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.293+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7e94d0 con 0x55da3f7ccae0 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.294+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'37 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.294+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7e9b20 con 0x7f6c8c078060 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.295+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.295+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7e9f60 con 0x55da3f7d2d30 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.296+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.296+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7e9f70 con 0x55da3f7ccae0 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.297+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'253 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.297+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7e9f90 con 0x7f6c9401a9b0 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.298+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'80 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.298+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7ea5e0 con 0x55da3f7d2d30 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.299+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'53 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.299+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7ea5e0 con 0x55da3f7ccae0 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.300+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.300+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x7f6c8c078060 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.301+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'53 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.301+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x7f6c9401a9b0 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.302+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.302+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x55da3f7d2d30 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.303+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.303+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x7f6c9401a9b0 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.304+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'81 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.304+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x7f6c8c078060 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.305+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'36 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.305+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x7f6c9401a9b0 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.306+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.306+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x7f6c8c078060 2024-04-23T19:12:14.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.307+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'61 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.307+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x55da3f7ccae0 2024-04-23T19:12:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.308+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'254 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.308+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x55da3f7d2d30 2024-04-23T19:12:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.309+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'75 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.309+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab50 con 0x7f6c9401a9b0 2024-04-23T19:12:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.310+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'82 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.310+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab50 con 0x55da3f7d2d30 2024-04-23T19:12:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.311+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.311+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab20 con 0x55da3f7ccae0 2024-04-23T19:12:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.312+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'38 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.313+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab20 con 0x55da3f7ccae0 2024-04-23T19:12:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.314+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'255 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.314+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab20 con 0x55da3f7d2d30 2024-04-23T19:12:14.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.315+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'76 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.315+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab50 con 0x7f6c9401a9b0 2024-04-23T19:12:14.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.316+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'40 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.316+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab50 con 0x55da3f7d2d30 2024-04-23T19:12:14.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.317+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.317+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eab20 con 0x55da3f7d2d30 2024-04-23T19:12:14.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.318+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.318+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x55da3f7d2d30 2024-04-23T19:12:14.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.318+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'26 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.319+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x55da3f7d2d30 2024-04-23T19:12:14.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.319+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'54 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.319+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x7f6c8c078060 2024-04-23T19:12:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.321+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.321+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eaa90 con 0x55da3f7d2d30 2024-04-23T19:12:14.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.322+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.322+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eb0d0 con 0x7f6c8c078060 2024-04-23T19:12:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.323+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'54 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.323+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eb0d0 con 0x55da3f7d2d30 2024-04-23T19:12:14.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.324+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'77 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.324+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eb0d0 con 0x7f6c8c078060 2024-04-23T19:12:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.325+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'62 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.325+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7eb0d0 con 0x7f6c9401a9b0 2024-04-23T19:12:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.326+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.326+0000 7f6b50fe1640 20 reqs_thread_entry: start 2024-04-23T19:12:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 20 Filter name: none 2024-04-23T19:12:14.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6aca7fc640 20 reqs_thread_entry: start 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:94 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f815d30 con 0x55da3f7d2d30 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(94 a_user [call out=48b,read 0~363 out=363b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 234+0+486 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 20 rados_obj.operate() r=0 bl.length=363 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x13 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.327+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:95 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f818640 con 0x55da3f7d2d30 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.328+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(95 a_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7f6c9400a4a0 con 0x55da3f7d2d30 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.328+0000 7f6c9f533800 10 removing key index: true_admin_akey 2024-04-23T19:12:14.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.328+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:96 8.15 8:afe09604:users.keys::true_admin_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f816620 con 0x55da3f7d2d30 2024-04-23T19:12:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(96 true_admin_akey [delete] v32'4 uv4 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6c9406d040 con 0x55da3f7d2d30 2024-04-23T19:12:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6c9f533800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-04-23T19:12:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify cookie 94395856679216 in=177b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f818cf0 con 0x7f6c8c078060 2024-04-23T19:12:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== watch-notify(notify (1) cookie 94395856438128 notify 137438953517 ret 0) ==== 207+0+0 (crc 0 0 0) 0x55da3f7e9b20 con 0x7f6c8c078060 2024-04-23T19:12:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6b7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953517 cookie 94395856438128 notifier 4863 bl.length()=165 2024-04-23T19:12:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6b7effd640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f6b60003d60 con 0x7f6c8c078060 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(97 notify.5 [notify cookie 94395856679216 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.329+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(98 notify.5 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.330+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== watch-notify(notify_complete (2) cookie 94395856679216 notify 137438953517 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55da3f7e9490 con 0x7f6c8c078060 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.330+0000 7f6c9f533800 10 removing email index: 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.330+0000 7f6c9f533800 10 removing user buckets index 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.330+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:99 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f818c50 con 0x55da3f7d2d30 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.330+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(99 a_user.buckets [delete] v32'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7f6c9406d040 con 0x55da3f7d2d30 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.330+0000 7f6c9f533800 10 removing user index: a_user 2024-04-23T19:12:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.330+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:100 8.14 8:2efb937b:users.uid::a_user:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f818140 con 0x55da3f7d2d30 2024-04-23T19:12:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.331+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(100 a_user [call,call,delete] v32'2 uv2 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f6c9406d040 con 0x55da3f7d2d30 2024-04-23T19:12:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.331+0000 7f6c9f533800 10 removing default.rgw.meta+users.uid+a_user from cache 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.331+0000 7f6c9f533800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.331+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:101 7.15 7:a93a5511:::notify.2:head [notify cookie 94395856686160 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55da3f817170 con 0x55da3f7d2d30 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.332+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== watch-notify(notify (1) cookie 94395856427584 notify 137438953482 ret 0) ==== 197+0+0 (crc 0 0 0) 0x55da3f7ea330 con 0x55da3f7d2d30 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.332+0000 7f6b7e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953482 cookie 94395856427584 notifier 4863 bl.length()=155 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.332+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(101 notify.2 [notify cookie 94395856686160 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6c9406d040 con 0x55da3f7d2d30 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.332+0000 7f6b7e7fc640 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:102 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f6ad00032c0 con 0x55da3f7d2d30 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.332+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(102 notify.2 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9406d040 con 0x55da3f7d2d30 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.332+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== watch-notify(notify_complete (2) cookie 94395856686160 notify 137438953482 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55da3f7c1530 con 0x55da3f7d2d30 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.333+0000 7f6b51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.333+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94395856443360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f818c50 con 0x55da3f7ccae0 2024-04-23T19:12:14.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.334+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94395856443360] v32'256 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.334+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94395856418928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dd7e0 con 0x7f6c9401a9b0 2024-04-23T19:12:14.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.335+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94395856418928] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.335+0000 7f6c9f533800 20 remove_watcher() i=0 2024-04-23T19:12:14.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.335+0000 7f6c9f533800 2 removed watcher, disabling cache 2024-04-23T19:12:14.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.335+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94395856422736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dd7e0 con 0x55da3f7ccae0 2024-04-23T19:12:14.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.336+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94395856422736] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.336+0000 7f6c9f533800 20 remove_watcher() i=1 2024-04-23T19:12:14.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.336+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94395856427584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dd7e0 con 0x55da3f7d2d30 2024-04-23T19:12:14.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.337+0000 7f6c9b9f6640 1 -- 172.21.15.63:0/623903279 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94395856427584] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9406d040 con 0x55da3f7d2d30 2024-04-23T19:12:14.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.337+0000 7f6c9f533800 20 remove_watcher() i=2 2024-04-23T19:12:14.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.337+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94395856424864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d9a40 con 0x7f6c9401a9b0 2024-04-23T19:12:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.338+0000 7f6c9b1f5640 1 -- 172.21.15.63:0/623903279 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94395856424864] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c8c075030 con 0x7f6c9401a9b0 2024-04-23T19:12:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.338+0000 7f6c9f533800 20 remove_watcher() i=3 2024-04-23T19:12:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.339+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94395856434224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7dd7e0 con 0x55da3f7ccae0 2024-04-23T19:12:14.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.339+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94395856434224] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x55da3f7ccae0 2024-04-23T19:12:14.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.340+0000 7f6c9f533800 20 remove_watcher() i=4 2024-04-23T19:12:14.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.340+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94395856438128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d8fa0 con 0x7f6c8c078060 2024-04-23T19:12:14.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.341+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94395856438128] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.341+0000 7f6c9f533800 20 remove_watcher() i=5 2024-04-23T19:12:14.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.341+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94395856432656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7d9a40 con 0x7f6c8c078060 2024-04-23T19:12:14.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.342+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94395856432656] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.342+0000 7f6c9f533800 20 remove_watcher() i=6 2024-04-23T19:12:14.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.342+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94395856440864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55da3f7db430 con 0x7f6c8c078060 2024-04-23T19:12:14.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94395856440864] v32'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6c9000f610 con 0x7f6c8c078060 2024-04-23T19:12:14.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 20 remove_watcher() i=7 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6c8c078060 msgr2=0x7f6c8c08e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f6c8c078060 0x7f6c8c08e9b0 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f7ccae0 msgr2=0x55da3f7cefa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f7ccae0 0x55da3f7cefa0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f6c9401a9b0 msgr2=0x7f6c94046860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f6c9401a9b0 0x7f6c94046860 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55da3f7d2d30 msgr2=0x55da3f7d5160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55da3f7d2d30 0x55da3f7d5160 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b64036eb0 msgr2=0x7f6b64039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b64036eb0 0x7f6b64039370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c006b30 tx=0x7f6c8c091000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 msgr2=0x55da3f7c0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7c0ff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6c9402aff0 tx=0x7f6c9404cc80 comp rx=0 tx=0).stop 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9a9f4640 1 -- 172.21.15.63:0/623903279 reap_dead start 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 shutdown_connections 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/623903279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7b9a90 0x55da3f7c0ff0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 >> 172.21.15.63:0/623903279 conn(0x55da3f7b5da0 msgr2=0x55da3f7b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 shutdown_connections 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/623903279 wait complete. 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b74036eb0 msgr2=0x7f6b74039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b74036eb0 0x7f6b74039370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c052a70 tx=0x7f6c8c06ff20 comp rx=0 tx=0).stop 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7ac780 msgr2=0x55da3f7acba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7ac780 0x55da3f7acba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55da3f798bb0 tx=0x7f6c940178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 shutdown_connections 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b74036eb0 0x7f6b74039370 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2356922891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f7ac780 0x55da3f7acba0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 >> 172.21.15.63:0/2356922891 conn(0x55da3f7a0820 msgr2=0x55da3f7a0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.343+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 shutdown_connections 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2356922891 wait complete. 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55da3f79b9f0 msgr2=0x55da3f79dde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55da3f79b9f0 0x55da3f79dde0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f6953c0 msgr2=0x55da3f798200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f6953c0 0x55da3f798200 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b80036e60 msgr2=0x7f6b80039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b80036e60 0x7f6b80039320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000f60 tx=0x7f6c90001370 comp rx=0 tx=0).stop 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 msgr2=0x55da3f790fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f790fc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005930 tx=0x7f6c8c003020 comp rx=0 tx=0).stop 2024-04-23T19:12:14.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 shutdown_connections 2024-04-23T19:12:14.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55da3f79b9f0 0x55da3f79dde0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55da3f6953c0 0x55da3f798200 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f6b80036e60 0x7f6b80039320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 --2- 172.21.15.63:0/2888614839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da3f694a60 0x55da3f790fc0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 >> 172.21.15.63:0/2888614839 conn(0x55da3f690600 msgr2=0x55da3f783720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 shutdown_connections 2024-04-23T19:12:14.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.344+0000 7f6c9f533800 1 -- 172.21.15.63:0/2888614839 wait complete. 2024-04-23T19:12:14.353 ERROR:root:None 2024-04-23T19:12:14.353 ERROR:root:0 2024-04-23T19:12:14.353 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-04-23T19:12:14.353 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-04-23T19:12:14.353 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=read 2024-04-23T19:12:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f1251975800 1 Processor -- start 2024-04-23T19:12:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f1251975800 1 -- start start 2024-04-23T19:12:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f1251975800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55ff1320d7b0 0x55ff1320dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f1251975800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ff130be7f0 con 0x55ff1320e110 2024-04-23T19:12:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f1251975800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55ff130ade20 con 0x55ff1320d7b0 2024-04-23T19:12:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f124f474640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55ff1320d7b0 0x55ff1320dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f124f474640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55ff1320d7b0 0x55ff1320dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40790/0 (socket says 172.21.15.63:40790) 2024-04-23T19:12:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f124f474640 1 -- 172.21.15.63:0/95362241 learned_addr learned my addr 172.21.15.63:0/95362241 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f124ec73640 1 --1- 172.21.15.63:0/95362241 >> v1:172.21.15.16:6789/0 conn(0x55ff1320e110 0x55ff1320a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56762/0 (socket says 172.21.15.63:56762) 2024-04-23T19:12:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f124f474640 1 -- 172.21.15.63:0/95362241 >> v1:172.21.15.16:6789/0 conn(0x55ff1320e110 legacy=0x55ff1320a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.400+0000 7f124f474640 1 -- 172.21.15.63:0/95362241 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff130ccb50 con 0x55ff1320d7b0 2024-04-23T19:12:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.402+0000 7f124f474640 1 --2- 172.21.15.63:0/95362241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320dbd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f124000a520 tx=0x7f124001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c0f8211ec2f041 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f124e472640 1 -- 172.21.15.63:0/95362241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240057070 con 0x55ff1320d7b0 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f124e472640 1 -- 172.21.15.63:0/95362241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240052570 con 0x55ff1320d7b0 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f124e472640 1 -- 172.21.15.63:0/95362241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f124004ae60 con 0x55ff1320d7b0 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f1251975800 1 -- 172.21.15.63:0/95362241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 msgr2=0x55ff1320dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f1251975800 1 --2- 172.21.15.63:0/95362241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320dbd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f124000a520 tx=0x7f124001d010 comp rx=0 tx=0).stop 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f1251975800 1 -- 172.21.15.63:0/95362241 shutdown_connections 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f1251975800 1 --2- 172.21.15.63:0/95362241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320dbd0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f1251975800 1 -- 172.21.15.63:0/95362241 >> 172.21.15.63:0/95362241 conn(0x55ff13205db0 msgr2=0x55ff132081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f1251975800 1 -- 172.21.15.63:0/95362241 shutdown_connections 2024-04-23T19:12:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.403+0000 7f1251975800 1 -- 172.21.15.63:0/95362241 wait complete. 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.416+0000 7f1251975800 1 Processor -- start 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.416+0000 7f1251975800 1 -- start start 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.416+0000 7f1251975800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.416+0000 7f1251975800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff131576f0 con 0x55ff1320d7b0 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.416+0000 7f124f474640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.416+0000 7f124f474640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40792/0 (socket says 172.21.15.63:40792) 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.416+0000 7f124f474640 1 -- 172.21.15.63:0/697577343 learned_addr learned my addr 172.21.15.63:0/697577343 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.417+0000 7f124f474640 1 -- 172.21.15.63:0/697577343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1320b410 con 0x55ff1320d7b0 2024-04-23T19:12:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.417+0000 7f124f474640 1 --2- 172.21.15.63:0/697577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320ce50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f124001d050 tx=0x7f124004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f52f084ff14a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.417+0000 7f113d60a640 1 -- 172.21.15.63:0/697577343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240057040 con 0x55ff1320d7b0 2024-04-23T19:12:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.417+0000 7f113d60a640 1 -- 172.21.15.63:0/697577343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240052720 con 0x55ff1320d7b0 2024-04-23T19:12:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.417+0000 7f113d60a640 1 -- 172.21.15.63:0/697577343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240052a40 con 0x55ff1320d7b0 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 -- 172.21.15.63:0/697577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 msgr2=0x55ff1320ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 --2- 172.21.15.63:0/697577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320ce50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f124001d050 tx=0x7f124004c000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 -- 172.21.15.63:0/697577343 shutdown_connections 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 --2- 172.21.15.63:0/697577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff1320ce50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 -- 172.21.15.63:0/697577343 >> 172.21.15.63:0/697577343 conn(0x55ff13205db0 msgr2=0x55ff132f93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 -- 172.21.15.63:0/697577343 shutdown_connections 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 -- 172.21.15.63:0/697577343 wait complete. 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 Processor -- start 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 -- start start 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff13307640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.418+0000 7f1251975800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff13215a10 con 0x55ff1320d7b0 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f124f474640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff13307640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f124f474640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff13307640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40794/0 (socket says 172.21.15.63:40794) 2024-04-23T19:12:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f124f474640 1 -- 172.21.15.63:0/2701244273 learned_addr learned my addr 172.21.15.63:0/2701244273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f124f474640 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff13307b80 con 0x55ff1320d7b0 2024-04-23T19:12:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f124f474640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff13307640 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1240005930 tx=0x7f1240003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f113be07640 1 -- 172.21.15.63:0/2701244273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240073070 con 0x55ff1320d7b0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f113be07640 1 -- 172.21.15.63:0/2701244273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124005d570 con 0x55ff1320d7b0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff13307e10 con 0x55ff1320d7b0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f113be07640 1 -- 172.21.15.63:0/2701244273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240057070 con 0x55ff1320d7b0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.419+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1330aac0 con 0x55ff1320d7b0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.420+0000 7f113be07640 1 -- 172.21.15.63:0/2701244273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1240076070 con 0x55ff1320d7b0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.420+0000 7f113be07640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1134036e60 0x7f1134039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.420+0000 7f113be07640 1 -- 172.21.15.63:0/2701244273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f124008e700 con 0x55ff1320d7b0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.420+0000 7f124ec73640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1134036e60 0x7f1134039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.421+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff1320e110 0x55ff1330e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.421+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1330ee00 con 0x55ff1320e110 2024-04-23T19:12:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.421+0000 7f124fc75640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff1320e110 0x55ff1330e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.421+0000 7f124ec73640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1134036e60 0x7f1134039320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1238000f60 tx=0x7f1238001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.421+0000 7f124fc75640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff1320e110 0x55ff1330e880 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.421+0000 7f124fc75640 1 -- 172.21.15.63:0/2701244273 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f124800a4a0 con 0x55ff1320e110 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.421+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1330be50 con 0x55ff1320e110 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f124fc75640 1 -- 172.21.15.63:0/2701244273 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f124800a4a0 con 0x55ff1320e110 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff13310c40 con 0x55ff1320e110 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f124fc75640 1 -- 172.21.15.63:0/2701244273 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f124800a4a0 con 0x55ff1320e110 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1330ff80 con 0x55ff1320e110 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f124fc75640 1 -- 172.21.15.63:0/2701244273 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f124800a4a0 con 0x55ff1320e110 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ff13311ff0 0x55ff133143e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff13314960 con 0x55ff13311ff0 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.422+0000 7f124f474640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ff13311ff0 0x55ff133143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.423+0000 7f124f474640 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ff13311ff0 0x55ff133143e0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.423+0000 7f124f474640 1 -- 172.21.15.63:0/2701244273 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f1240075030 con 0x55ff13311ff0 2024-04-23T19:12:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f1251975800 1 Processor -- start 2024-04-23T19:12:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f1251975800 1 -- start start 2024-04-23T19:12:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f1251975800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff13322d80 0x55ff133231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f1251975800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff130fde30 con 0x55ff13322d80 2024-04-23T19:12:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f124fc75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff13322d80 0x55ff133231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f124fc75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff13322d80 0x55ff133231a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40800/0 (socket says 172.21.15.63:40800) 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f124fc75640 1 -- 172.21.15.63:0/2588216025 learned_addr learned my addr 172.21.15.63:0/2588216025 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.424+0000 7f124fc75640 1 -- 172.21.15.63:0/2588216025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff133236e0 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.425+0000 7f124fc75640 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff13322d80 0x55ff133231a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55ff1330f200 tx=0x7f12480178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.425+0000 7f1139602640 1 -- 172.21.15.63:0/2588216025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1248013440 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.425+0000 7f1139602640 1 -- 172.21.15.63:0/2588216025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12480135e0 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.425+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff13324150 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.425+0000 7f1139602640 1 -- 172.21.15.63:0/2588216025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12480138c0 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.425+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff13324650 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.426+0000 7f1139602640 1 -- 172.21.15.63:0/2588216025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1248011040 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.426+0000 7f1139602640 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1128036eb0 0x7f1128039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.426+0000 7f1139602640 1 -- 172.21.15.63:0/2588216025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f124801d070 con 0x55ff13322d80 2024-04-23T19:12:14.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.426+0000 7f124f474640 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1128036eb0 0x7f1128039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.426+0000 7f1251975800 1 Processor -- start 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.426+0000 7f1251975800 1 -- start start 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.426+0000 7f124f474640 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1128036eb0 0x7f1128039370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55ff13314d40 tx=0x7f124001d010 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.427+0000 7f1251975800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff13330410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.427+0000 7f1251975800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff1320b6d0 con 0x55ff1332fff0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.427+0000 7f124fc75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff13330410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.427+0000 7f124fc75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff13330410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40802/0 (socket says 172.21.15.63:40802) 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.427+0000 7f124fc75640 1 -- 172.21.15.63:0/2606309020 learned_addr learned my addr 172.21.15.63:0/2606309020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.427+0000 7f124fc75640 1 -- 172.21.15.63:0/2606309020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff133309e0 con 0x55ff1332fff0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.427+0000 7f124fc75640 1 --2- 172.21.15.63:0/2606309020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff13330410 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f124804cc30 tx=0x7f12480129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca63e9bda1a596 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1132ffd640 1 -- 172.21.15.63:0/2606309020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12480123a0 con 0x55ff1332fff0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1132ffd640 1 -- 172.21.15.63:0/2606309020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124804a020 con 0x55ff1332fff0 2024-04-23T19:12:14.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1132ffd640 1 -- 172.21.15.63:0/2606309020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1248001310 con 0x55ff1332fff0 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 -- 172.21.15.63:0/2606309020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 msgr2=0x55ff13330410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 --2- 172.21.15.63:0/2606309020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff13330410 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f124804cc30 tx=0x7f12480129d0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 -- 172.21.15.63:0/2606309020 shutdown_connections 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 --2- 172.21.15.63:0/2606309020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff13330410 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 -- 172.21.15.63:0/2606309020 >> 172.21.15.63:0/2606309020 conn(0x55ff1332c3a0 msgr2=0x55ff1332a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 -- 172.21.15.63:0/2606309020 shutdown_connections 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 -- 172.21.15.63:0/2606309020 wait complete. 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.428+0000 7f1251975800 1 Processor -- start 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f1251975800 1 -- start start 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f1251975800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff133375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f1251975800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff13331140 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f124fc75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff133375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f124fc75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff133375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40804/0 (socket says 172.21.15.63:40804) 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 learned_addr learned my addr 172.21.15.63:0/4228560187 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff13337b00 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f124fc75640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff133375c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f124804cc30 tx=0x7f124804cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f11317fa640 1 -- 172.21.15.63:0/4228560187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f124802aa60 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.429+0000 7f11317fa640 1 -- 172.21.15.63:0/4228560187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124802ac00 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.430+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff13337d90 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.430+0000 7f11317fa640 1 -- 172.21.15.63:0/4228560187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f124802a350 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.430+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1333aa40 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.430+0000 7f11317fa640 1 -- 172.21.15.63:0/4228560187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1248045040 con 0x55ff1332fff0 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f11317fa640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1118036eb0 0x7f1118039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f11317fa640 1 -- 172.21.15.63:0/4228560187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f1248043020 con 0x55ff1332fff0 2024-04-23T19:12:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f124f474640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1118036eb0 0x7f1118039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f124f474640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1118036eb0 0x7f1118039370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1240052cd0 tx=0x7f1240083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff13343260 0x55ff13345720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff13345e70 con 0x55ff13343260 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.431+0000 7f124ec73640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff13343260 0x55ff13345720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f124ec73640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff13343260 0x55ff13345720 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 20 realm 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff133426b0 con 0x55ff13343260 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff13346960 con 0x55ff13343260 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.432+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff13347230 con 0x55ff13343260 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f1251975800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff133475d0 con 0x55ff13343260 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f1251975800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f1251975800 20 searching for the correct realm 2024-04-23T19:12:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ff133494b0 0x55ff1334b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x55ff133494b0 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.433+0000 7f124fc75640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ff133494b0 0x55ff1334b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.434+0000 7f124fc75640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ff133494b0 0x55ff1334b8e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.434+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.434+0000 7f124fc75640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1248047b80 0x7f12480368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.434+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x7f1248047b80 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.434+0000 7f124f474640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1248047b80 0x7f12480368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.434+0000 7f124f474640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1248047b80 0x7f12480368c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.435+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.435+0000 7f124f474640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f1240078060 0x7f124008e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.435+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x7f1240078060 2024-04-23T19:12:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.435+0000 7f124ec73640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f1240078060 0x7f124008e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.435+0000 7f124ec73640 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f1240078060 0x7f124008e050 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x55ff13343260 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x55ff13343260 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x7f1240078060 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.436+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x7f1240078060 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x7f1248047b80 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x55ff133494b0 2024-04-23T19:12:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x55ff13343260 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.437+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x7f1240078060 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.438+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.438+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x55ff13343260 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.438+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.438+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334bfb0 con 0x7f1248047b80 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.438+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.438+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff133494b0 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff133494b0 2024-04-23T19:12:14.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1240078060 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff133494b0 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.439+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff133494b0 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.440+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.440+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff13343260 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.440+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.440+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff13343260 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.440+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.440+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1240078060 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.441+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.441+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff133494b0 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.441+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.441+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1248047b80 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.441+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.441+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff13343260 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1248047b80 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff133494b0 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1240078060 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.442+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1248047b80 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff133494b0 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1240078060 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x55ff13343260 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.443+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7f12400a9060 con 0x7f1248047b80 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:14.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff13347f20 con 0x55ff13343260 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff133487c0 con 0x55ff13343260 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.444+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334c430 con 0x7f1240078060 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 20 zone default found 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 4 Realm: () 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 20 zonegroup default 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334c4c0 con 0x7f1248047b80 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.445+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334e790 con 0x55ff133494b0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f1251975800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334eb30 con 0x55ff133494b0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f1251975800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f1251975800 20 started sync module instance, tier type = 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f1251975800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.446+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334dc20 con 0x7f1248047b80 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.447+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.447+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94554027253776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334f3f0 con 0x7f1248047b80 2024-04-23T19:12:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.447+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334fc40 con 0x55ff13343260 2024-04-23T19:12:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.448+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94554027253776] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.448+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.448+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94554027257664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334fc80 con 0x55ff13343260 2024-04-23T19:12:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.448+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13350770 con 0x55ff133494b0 2024-04-23T19:12:14.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.449+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.449+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94554027262528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13351010 con 0x55ff133494b0 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.449+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13351b10 con 0x7f1248047b80 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.450+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94554027257664] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.450+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.451+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94554027267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13351f00 con 0x7f1248047b80 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.451+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13352940 con 0x55ff13343260 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.451+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94554027262528] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.452+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.452+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94554027271184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13352900 con 0x55ff13343260 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.452+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13353450 con 0x7f1240078060 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.452+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94554027267584] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.453+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.453+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94554027274112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13353450 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.453+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13353e50 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.453+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94554027271184] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.454+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94554027274112] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.454+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.454+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94554027259792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13353960 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.454+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff133546f0 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94554027259792] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94554027278912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13353960 con 0x7f1240078060 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 20 add_watcher() i=0 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 20 add_watcher() i=1 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 20 add_watcher() i=2 2024-04-23T19:12:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 20 add_watcher() i=3 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 20 add_watcher() i=4 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 20 add_watcher() i=5 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.455+0000 7f1251975800 20 add_watcher() i=6 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.456+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94554027278912] v32'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.456+0000 7f1251975800 20 add_watcher() i=7 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.456+0000 7f1251975800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.456+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff13350b60 con 0x55ff13343260 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.457+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.457+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94554027279968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13350b60 con 0x55ff13343260 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94554027279968] v32'257 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1105fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 5 note: GC not initialized 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1104fe1640 20 reqs_thread_entry: start 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 20 Filter name: none 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f107e7fc640 20 reqs_thread_entry: start 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.458+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1338ac50 con 0x7f1240078060 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.459+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.459+0000 7f1251975800 20 rados_obj.operate() r=0 bl.length=460 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.459+0000 7f1251975800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.459+0000 7f1251975800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.459+0000 7f1251975800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.459+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13390f90 con 0x7f1240078060 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.460+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v32'6 uv6 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.460+0000 7f1251975800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.460+0000 7f1251975800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.460+0000 7f1251975800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94554027526656 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55ff1338fae0 con 0x55ff13343260 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94554027271184 notify 137438953500 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7f1238018040 con 0x55ff13343260 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f1132ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953500 cookie 94554027271184 notifier 4870 bl.length()=666 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f1132ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f1132ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94554027526656 out=8b] v0'0 uv258 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f1132ffd640 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f1114003ea0 con 0x55ff13343260 2024-04-23T19:12:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.461+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 94554027526656 notify 137438953500 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f1238009050 con 0x55ff13343260 2024-04-23T19:12:14.472 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"read"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.462+0000 7f1105fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.462+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94554027279968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1338b150 con 0x55ff13343260 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.463+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94554027279968] v32'258 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.463+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94554027253776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13354660 con 0x7f1248047b80 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.464+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94554027253776] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.464+0000 7f1251975800 20 remove_watcher() i=0 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.464+0000 7f1251975800 2 removed watcher, disabling cache 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.464+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94554027257664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334e010 con 0x55ff13343260 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.465+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94554027257664] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.466+0000 7f1251975800 20 remove_watcher() i=1 2024-04-23T19:12:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.466+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94554027262528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334ef40 con 0x55ff133494b0 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.467+0000 7f124fc75640 1 -- 172.21.15.63:0/4228560187 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94554027262528] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f124804cd70 con 0x55ff133494b0 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.467+0000 7f1251975800 20 remove_watcher() i=2 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.467+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94554027267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334ef40 con 0x7f1248047b80 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.468+0000 7f124f474640 1 -- 172.21.15.63:0/4228560187 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94554027267584] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1240089030 con 0x7f1248047b80 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.468+0000 7f1251975800 20 remove_watcher() i=3 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.468+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94554027271184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334ef40 con 0x55ff13343260 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.469+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94554027271184] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x55ff13343260 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.469+0000 7f1251975800 20 remove_watcher() i=4 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.469+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94554027274112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13352410 con 0x7f1240078060 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.470+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94554027274112] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.470+0000 7f1251975800 20 remove_watcher() i=5 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.470+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94554027259792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff1334ef40 con 0x7f1240078060 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.471+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94554027259792] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.472+0000 7f1251975800 20 remove_watcher() i=6 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.472+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94554027278912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55ff13352410 con 0x7f1240078060 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94554027278912] v32'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f123800f610 con 0x7f1240078060 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 20 remove_watcher() i=7 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f1240078060 msgr2=0x7f124008e050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f1240078060 0x7f124008e050 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff13343260 msgr2=0x55ff13345720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff13343260 0x55ff13345720 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1248047b80 msgr2=0x7f12480368c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1248047b80 0x7f12480368c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ff133494b0 msgr2=0x55ff1334b8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55ff133494b0 0x55ff1334b8e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1118036eb0 msgr2=0x7f1118039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1118036eb0 0x7f1118039370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1240052cd0 tx=0x7f1240083000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 msgr2=0x55ff133375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f124ec73640 1 -- 172.21.15.63:0/4228560187 reap_dead start 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff133375c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f124804cc30 tx=0x7f124804cbc0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 shutdown_connections 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/4228560187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1332fff0 0x55ff133375c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 >> 172.21.15.63:0/4228560187 conn(0x55ff1332c3a0 msgr2=0x55ff13331860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 shutdown_connections 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/4228560187 wait complete. 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1128036eb0 msgr2=0x7f1128039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1128036eb0 0x7f1128039370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55ff13314d40 tx=0x7f124001d010 comp rx=0 tx=0).stop 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff13322d80 msgr2=0x55ff133231a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff13322d80 0x55ff133231a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55ff1330f200 tx=0x7f12480178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 shutdown_connections 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.473+0000 7f1251975800 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1128036eb0 0x7f1128039370 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2588216025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff13322d80 0x55ff133231a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 >> 172.21.15.63:0/2588216025 conn(0x55ff13316e20 msgr2=0x55ff13317220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 shutdown_connections 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2588216025 wait complete. 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ff13311ff0 msgr2=0x55ff133143e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ff13311ff0 0x55ff133143e0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff1320e110 msgr2=0x55ff1330e880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff1320e110 0x55ff1330e880 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1134036e60 msgr2=0x7f1134039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1134036e60 0x7f1134039320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1238000f60 tx=0x7f1238001370 comp rx=0 tx=0).stop 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 msgr2=0x55ff13307640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff13307640 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1240005930 tx=0x7f1240003020 comp rx=0 tx=0).stop 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 shutdown_connections 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55ff13311ff0 0x55ff133143e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55ff1320e110 0x55ff1330e880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1134036e60 0x7f1134039320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 --2- 172.21.15.63:0/2701244273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff1320d7b0 0x55ff13307640 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 >> 172.21.15.63:0/2701244273 conn(0x55ff13205db0 msgr2=0x55ff132f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 shutdown_connections 2024-04-23T19:12:14.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.474+0000 7f1251975800 1 -- 172.21.15.63:0/2701244273 wait complete. 2024-04-23T19:12:14.484 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.484 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.484 ERROR:root:0 2024-04-23T19:12:14.484 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:14.488 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:14.488 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-04-23T19:12:14.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-04-23T19:12:14.489 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'user-info-without-keys=*' 2024-04-23T19:12:14.534 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:14.534 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.534+0000 7f9e0272f800 1 Processor -- start 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9e0272f800 1 -- start start 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9e0272f800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55665b132580 0x55665b13b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9e0272f800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55665afe47f0 con 0x55665b130a40 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9e0272f800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55665afd3e20 con 0x55665b132580 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9dfdbf4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55665b132580 0x55665b13b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9dfe3f5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55665b130a40 0x55665b130e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56772/0 (socket says 172.21.15.63:56772) 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/1759404333 learned_addr learned my addr 172.21.15.63:0/1759404333 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/1759404333 >> v1:172.21.15.16:6789/0 conn(0x55665b130a40 legacy=0x55665b130e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.535+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/1759404333 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665aff2b50 con 0x55665b132580 2024-04-23T19:12:14.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.537+0000 7f9dfdbf4640 1 --2- 172.21.15.63:0/1759404333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b132580 0x55665b13b080 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9df400a470 tx=0x7f9df401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d7cff55dcf8842 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.537+0000 7f9dfd3f3640 1 -- 172.21.15.63:0/1759404333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df4057070 con 0x55665b132580 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.537+0000 7f9dfd3f3640 1 -- 172.21.15.63:0/1759404333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df4052440 con 0x55665b132580 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.537+0000 7f9dfd3f3640 1 -- 172.21.15.63:0/1759404333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df4052740 con 0x55665b132580 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.538+0000 7f9e0272f800 1 -- 172.21.15.63:0/1759404333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b132580 msgr2=0x55665b13b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.538+0000 7f9e0272f800 1 --2- 172.21.15.63:0/1759404333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b132580 0x55665b13b080 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9df400a470 tx=0x7f9df401ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.538+0000 7f9e0272f800 1 -- 172.21.15.63:0/1759404333 shutdown_connections 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.538+0000 7f9e0272f800 1 --2- 172.21.15.63:0/1759404333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b132580 0x55665b13b080 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.538+0000 7f9e0272f800 1 -- 172.21.15.63:0/1759404333 >> 172.21.15.63:0/1759404333 conn(0x55665b12c600 msgr2=0x55665b12ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.538+0000 7f9e0272f800 1 -- 172.21.15.63:0/1759404333 shutdown_connections 2024-04-23T19:12:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.538+0000 7f9e0272f800 1 -- 172.21.15.63:0/1759404333 wait complete. 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.551+0000 7f9e0272f800 1 Processor -- start 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.551+0000 7f9e0272f800 1 -- start start 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.551+0000 7f9e0272f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b2257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.551+0000 7f9e0272f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55665b07d6f0 con 0x55665b130a40 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.551+0000 7f9dfe3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b2257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.551+0000 7f9dfe3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b2257f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40816/0 (socket says 172.21.15.63:40816) 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.551+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3427693117 learned_addr learned my addr 172.21.15.63:0/3427693117 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.552+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3427693117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665b225dc0 con 0x55665b130a40 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.552+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3427693117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b2257f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9df0001de0 tx=0x7f9df000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bc5368531f4e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.552+0000 7f9cec608640 1 -- 172.21.15.63:0/3427693117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0012020 con 0x55665b130a40 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.552+0000 7f9cec608640 1 -- 172.21.15.63:0/3427693117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df000e5a0 con 0x55665b130a40 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.552+0000 7f9cec608640 1 -- 172.21.15.63:0/3427693117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df000e8c0 con 0x55665b130a40 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 -- 172.21.15.63:0/3427693117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 msgr2=0x55665b2257f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3427693117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b2257f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9df0001de0 tx=0x7f9df000b6c0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 -- 172.21.15.63:0/3427693117 shutdown_connections 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3427693117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b2257f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 -- 172.21.15.63:0/3427693117 >> 172.21.15.63:0/3427693117 conn(0x55665b12c600 msgr2=0x55665b221110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 -- 172.21.15.63:0/3427693117 shutdown_connections 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 -- 172.21.15.63:0/3427693117 wait complete. 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 Processor -- start 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 -- start start 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b22d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9e0272f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55665b13d8a0 con 0x55665b130a40 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.553+0000 7f9dfe3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b22d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9dfe3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b22d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40822/0 (socket says 172.21.15.63:40822) 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3795130753 learned_addr learned my addr 172.21.15.63:0/3795130753 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665b22d8a0 con 0x55665b130a40 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b22d360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9df0001de0 tx=0x7f9df0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9ceae05640 1 -- 172.21.15.63:0/3795130753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0012030 con 0x55665b130a40 2024-04-23T19:12:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9ceae05640 1 -- 172.21.15.63:0/3795130753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df001a7a0 con 0x55665b130a40 2024-04-23T19:12:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9ceae05640 1 -- 172.21.15.63:0/3795130753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df001aac0 con 0x55665b130a40 2024-04-23T19:12:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55665b22db30 con 0x55665b130a40 2024-04-23T19:12:14.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.554+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665b2307e0 con 0x55665b130a40 2024-04-23T19:12:14.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.555+0000 7f9ceae05640 1 -- 172.21.15.63:0/3795130753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9df0015020 con 0x55665b130a40 2024-04-23T19:12:14.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.555+0000 7f9ceae05640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ce4036e60 0x7f9ce4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.555+0000 7f9ceae05640 1 -- 172.21.15.63:0/3795130753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9df0020070 con 0x55665b130a40 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.555+0000 7f9dfdbf4640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ce4036e60 0x7f9ce4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.555+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b132580 0x55665b2345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.555+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b234ae0 con 0x55665b132580 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.556+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b132580 0x55665b2345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.556+0000 7f9dfdbf4640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ce4036e60 0x7f9ce4039320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9df4054000 tx=0x7f9df404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.556+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b132580 0x55665b2345a0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.556+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3795130753 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9df800a4a0 con 0x55665b132580 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.556+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b231b70 con 0x55665b132580 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3795130753 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9df800a4a0 con 0x55665b132580 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b236920 con 0x55665b132580 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3795130753 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9df800a4a0 con 0x55665b132580 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b235c60 con 0x55665b132580 2024-04-23T19:12:14.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3795130753 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9df800a4a0 con 0x55665b132580 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55665b237cd0 0x55665b23a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b23a680 con 0x55665b237cd0 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.557+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55665b237cd0 0x55665b23a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.558+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55665b237cd0 0x55665b23a0c0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.558+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3795130753 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9df0024980 con 0x55665b237cd0 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9e0272f800 1 Processor -- start 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9e0272f800 1 -- start start 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9e0272f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b248af0 0x55665b248f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9e0272f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55665b023e30 con 0x55665b248af0 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9dfebf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b248af0 0x55665b248f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9dfebf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b248af0 0x55665b248f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40832/0 (socket says 172.21.15.63:40832) 2024-04-23T19:12:14.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9dfebf6640 1 -- 172.21.15.63:0/747327513 learned_addr learned my addr 172.21.15.63:0/747327513 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9dfebf6640 1 -- 172.21.15.63:0/747327513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665b249450 con 0x55665b248af0 2024-04-23T19:12:14.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.559+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b248af0 0x55665b248f10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55665b22e760 tx=0x7f9df80178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.560+0000 7f9ce3fff640 1 -- 172.21.15.63:0/747327513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df8013440 con 0x55665b248af0 2024-04-23T19:12:14.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.560+0000 7f9ce3fff640 1 -- 172.21.15.63:0/747327513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df80135e0 con 0x55665b248af0 2024-04-23T19:12:14.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.560+0000 7f9ce3fff640 1 -- 172.21.15.63:0/747327513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df80138c0 con 0x55665b248af0 2024-04-23T19:12:14.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.560+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55665b249ec0 con 0x55665b248af0 2024-04-23T19:12:14.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.560+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665b24a3c0 con 0x55665b248af0 2024-04-23T19:12:14.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.560+0000 7f9ce3fff640 1 -- 172.21.15.63:0/747327513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9df8011040 con 0x55665b248af0 2024-04-23T19:12:14.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9ce3fff640 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cd8036e60 0x7f9cd8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9ce3fff640 1 -- 172.21.15.63:0/747327513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9df801d070 con 0x55665b248af0 2024-04-23T19:12:14.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cd8036e60 0x7f9cd8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9e0272f800 1 Processor -- start 2024-04-23T19:12:14.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9e0272f800 1 -- start start 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cd8036e60 0x7f9cd8039320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9df000b6e0 tx=0x7f9df001ae40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9e0272f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b256280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.561+0000 7f9e0272f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55665b226140 con 0x55665b255e60 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9dfebf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b256280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9dfebf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b256280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40840/0 (socket says 172.21.15.63:40840) 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9dfebf6640 1 -- 172.21.15.63:0/4153060161 learned_addr learned my addr 172.21.15.63:0/4153060161 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9dfebf6640 1 -- 172.21.15.63:0/4153060161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665b256850 con 0x55665b255e60 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/4153060161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b256280 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9df804cca0 tx=0x7f9df80127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a65f847cfa269f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9ce1ffb640 1 -- 172.21.15.63:0/4153060161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df8042020 con 0x55665b255e60 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9ce1ffb640 1 -- 172.21.15.63:0/4153060161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df8012cc0 con 0x55665b255e60 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.562+0000 7f9ce1ffb640 1 -- 172.21.15.63:0/4153060161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df8027040 con 0x55665b255e60 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 -- 172.21.15.63:0/4153060161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 msgr2=0x55665b256280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 --2- 172.21.15.63:0/4153060161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b256280 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9df804cca0 tx=0x7f9df80127a0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 -- 172.21.15.63:0/4153060161 shutdown_connections 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 --2- 172.21.15.63:0/4153060161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b256280 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 -- 172.21.15.63:0/4153060161 >> 172.21.15.63:0/4153060161 conn(0x55665b252130 msgr2=0x55665b250190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 -- 172.21.15.63:0/4153060161 shutdown_connections 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 -- 172.21.15.63:0/4153060161 wait complete. 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 Processor -- start 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 -- start start 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b25d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9e0272f800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55665b256fa0 con 0x55665b255e60 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9dfebf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b25d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9dfebf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b25d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40856/0 (socket says 172.21.15.63:40856) 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.563+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 learned_addr learned my addr 172.21.15.63:0/3776402211 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.564+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665b25d820 con 0x55665b255e60 2024-04-23T19:12:14.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.564+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b25d2e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9df804bb90 tx=0x7f9df804cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.564+0000 7f9ccffff640 1 -- 172.21.15.63:0/3776402211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df8027040 con 0x55665b255e60 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.564+0000 7f9ccffff640 1 -- 172.21.15.63:0/3776402211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df8021c60 con 0x55665b255e60 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.564+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55665b25db10 con 0x55665b255e60 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.564+0000 7f9ccffff640 1 -- 172.21.15.63:0/3776402211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df802a350 con 0x55665b255e60 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.564+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665b260760 con 0x55665b255e60 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.565+0000 7f9ccffff640 1 -- 172.21.15.63:0/3776402211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9df8045020 con 0x55665b255e60 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.565+0000 7f9ccffff640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cc8036e60 0x7f9cc8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.565+0000 7f9ccffff640 1 -- 172.21.15.63:0/3776402211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9df8033070 con 0x55665b255e60 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.565+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cc8036e60 0x7f9cc8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.565+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cc8036e60 0x7f9cc8039320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9df0024c90 tx=0x7f9df0035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b268fd0 0x55665b26b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26bbe0 con 0x55665b268fd0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9dfdbf4640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b268fd0 0x55665b26b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9dfdbf4640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b268fd0 0x55665b26b490 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 20 realm 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26bfb0 con 0x55665b268fd0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.566+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26c880 con 0x55665b268fd0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26cc20 con 0x55665b268fd0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26d470 con 0x55665b268fd0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 20 searching for the correct realm 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55665b26f1e0 0x55665b2715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.567+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55665b26f1e0 0x55665b2715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.568+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55665b26f1e0 0x55665b2715d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.568+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.568+0000 7f9dfebf6640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9df80123a0 0x7f9df803c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.568+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df80123a0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.568+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9df80123a0 0x7f9df803c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.569+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9df80123a0 0x7f9df803c990 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.569+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.569+0000 7f9dfe3f5640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9df0017050 0x7f9df0016590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.569+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.569+0000 7f9dfdbf4640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9df0017050 0x7f9df0016590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.569+0000 7f9dfdbf4640 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9df0017050 0x7f9df0016590 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.570+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.571+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.571+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df80123a0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.571+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.571+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.571+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.571+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.571+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.572+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.572+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.572+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.572+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.572+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df80123a0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.572+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.572+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.573+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.574+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df80123a0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df80123a0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.575+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.576+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.576+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.576+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.576+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df80123a0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.576+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.576+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b26f1e0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df0017050 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x55665b268fd0 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55665b271c00 con 0x7f9df80123a0 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.577+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26ddf0 con 0x55665b268fd0 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26e690 con 0x55665b268fd0 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b272080 con 0x7f9df0017050 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 zone default found 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 4 Realm: () 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:14.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 zonegroup default 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.578+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b26e2f0 con 0x7f9df80123a0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b274410 con 0x55665b26f1e0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b2747b0 con 0x55665b26f1e0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 20 started sync module instance, tier type = 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.579+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b2738a0 con 0x7f9df80123a0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.580+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.580+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93898104323264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b275050 con 0x7f9df80123a0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.580+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b2758a0 con 0x55665b268fd0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.581+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93898104323264] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.582+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.582+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93898104327072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b2758a0 con 0x55665b268fd0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.582+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b276390 con 0x55665b26f1e0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.583+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.583+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93898104331872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b276c30 con 0x55665b26f1e0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.583+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b277710 con 0x7f9df80123a0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.583+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93898104327072] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.584+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.584+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93898104336928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b277b00 con 0x7f9df80123a0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.584+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93898104331872] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.584+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b278500 con 0x55665b268fd0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.585+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93898104336928] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.585+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.585+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93898104329200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b2784c0 con 0x55665b268fd0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.585+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b278e80 con 0x7f9df0017050 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.586+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.586+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93898104329200] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.586+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93898104342992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b2792a0 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.586+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b279810 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.587+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93898104342992] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.587+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.587+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93898104344048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b279210 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.587+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b279210 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.588+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93898104344048] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.588+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93898104346640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b27a0e0 con 0x7f9df0017050 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=0 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=1 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=2 2024-04-23T19:12:14.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=3 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=4 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=5 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=6 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93898104346640] v32'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 20 add_watcher() i=7 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.589+0000 7f9e0272f800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.590+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b27d5a0 con 0x55665b268fd0 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.590+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.590+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93898104348160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b27aa60 con 0x55665b268fd0 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.591+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93898104348160] v32'259 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.591+0000 7f9cb4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.591+0000 7f9e0272f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.591+0000 7f9e0272f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.591+0000 7f9e0272f800 5 note: GC not initialized 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.591+0000 7f9c2f7fe640 20 reqs_thread_entry: start 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.591+0000 7f9e0272f800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 20 Filter name: none 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9c2d7fa640 20 reqs_thread_entry: start 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b2b0a70 con 0x7f9df0017050 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv6 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 20 rados_obj.operate() r=0 bl.length=490 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.592+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b2b7110 con 0x7f9df0017050 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v32'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9e0272f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9e0272f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9e0272f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93898104597376 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55665b2b5c60 con 0x55665b268fd0 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 93898104329200 notify 137438953501 ret 0) ==== 708+0+0 (crc 0 0 0) 0x55665aff2b50 con 0x55665b268fd0 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9ce1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953501 cookie 93898104329200 notifier 4877 bl.length()=666 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9ce1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9ce1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9ce1ffb640 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f9cc4003ea0 con 0x55665b268fd0 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 93898104597376 out=8b] v0'0 uv261 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.594+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 93898104597376 notify 137438953501 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f9df405dc30 con 0x55665b268fd0 2024-04-23T19:12:14.605 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.595+0000 7f9cb4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.595+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93898104348160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b27cc20 con 0x55665b268fd0 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.596+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93898104348160] v32'260 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.596+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93898104323264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b279dd0 con 0x7f9df80123a0 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.597+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93898104323264] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.597+0000 7f9e0272f800 20 remove_watcher() i=0 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.597+0000 7f9e0272f800 2 removed watcher, disabling cache 2024-04-23T19:12:14.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.597+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93898104327072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b273cc0 con 0x55665b268fd0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.598+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93898104327072] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.598+0000 7f9e0272f800 20 remove_watcher() i=1 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.598+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93898104331872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b273cc0 con 0x55665b26f1e0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.599+0000 7f9dfebf6640 1 -- 172.21.15.63:0/3776402211 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93898104331872] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df8029070 con 0x55665b26f1e0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.599+0000 7f9e0272f800 20 remove_watcher() i=2 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.599+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93898104336928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b273cc0 con 0x7f9df80123a0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.600+0000 7f9dfe3f5640 1 -- 172.21.15.63:0/3776402211 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93898104336928] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df0024980 con 0x7f9df80123a0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.600+0000 7f9e0272f800 20 remove_watcher() i=3 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.600+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93898104329200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b277220 con 0x55665b268fd0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.601+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93898104329200] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x55665b268fd0 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.601+0000 7f9e0272f800 20 remove_watcher() i=4 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.602+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93898104342992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b273cc0 con 0x7f9df0017050 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.603+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93898104342992] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.603+0000 7f9e0272f800 20 remove_watcher() i=5 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.603+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93898104344048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b2789d0 con 0x7f9df0017050 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.604+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93898104344048] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.604+0000 7f9e0272f800 20 remove_watcher() i=6 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.604+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93898104346640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55665b273cc0 con 0x7f9df0017050 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93898104346640] v32'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9df405a040 con 0x7f9df0017050 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 20 remove_watcher() i=7 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9df0017050 msgr2=0x7f9df0016590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f9df0017050 0x7f9df0016590 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b268fd0 msgr2=0x55665b26b490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b268fd0 0x55665b26b490 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9df80123a0 msgr2=0x7f9df803c990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f9df80123a0 0x7f9df803c990 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55665b26f1e0 msgr2=0x55665b2715d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55665b26f1e0 0x55665b2715d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cc8036e60 msgr2=0x7f9cc8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cc8036e60 0x7f9cc8039320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9df0024c90 tx=0x7f9df0035000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 msgr2=0x55665b25d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3776402211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b255e60 0x55665b25d2e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9df804bb90 tx=0x7f9df804cc00 comp rx=0 tx=0).stop 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 reap_dead start 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9dfdbf4640 1 -- 172.21.15.63:0/3776402211 reap_dead start 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 shutdown_connections 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 >> 172.21.15.63:0/3776402211 conn(0x55665b252130 msgr2=0x55665b24f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 shutdown_connections 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/3776402211 wait complete. 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cd8036e60 msgr2=0x7f9cd8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cd8036e60 0x7f9cd8039320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9df000b6e0 tx=0x7f9df001ae40 comp rx=0 tx=0).stop 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b248af0 msgr2=0x55665b248f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.605+0000 7f9e0272f800 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b248af0 0x55665b248f10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55665b22e760 tx=0x7f9df80178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 shutdown_connections 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9cd8036e60 0x7f9cd8039320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/747327513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b248af0 0x55665b248f10 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 >> 172.21.15.63:0/747327513 conn(0x55665b23cb90 msgr2=0x55665b23cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 shutdown_connections 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/747327513 wait complete. 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55665b237cd0 msgr2=0x55665b23a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55665b237cd0 0x55665b23a0c0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b132580 msgr2=0x55665b2345a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b132580 0x55665b2345a0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ce4036e60 msgr2=0x7f9ce4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ce4036e60 0x7f9ce4039320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9df4054000 tx=0x7f9df404e000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 msgr2=0x55665b22d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b22d360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9df0001de0 tx=0x7f9df0000ed0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 shutdown_connections 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55665b237cd0 0x55665b23a0c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55665b132580 0x55665b2345a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9ce4036e60 0x7f9ce4039320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 --2- 172.21.15.63:0/3795130753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55665b130a40 0x55665b22d360 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 >> 172.21.15.63:0/3795130753 conn(0x55665b12c600 msgr2=0x55665b21f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 shutdown_connections 2024-04-23T19:12:14.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.606+0000 7f9e0272f800 1 -- 172.21.15.63:0/3795130753 wait complete. 2024-04-23T19:12:14.616 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.616 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.616 ERROR:root:0 2024-04-23T19:12:14.617 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:14.621 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:14.621 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:14.621 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:14.621 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-04-23T19:12:14.667 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:14.667 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:14.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.667+0000 7f48aba38800 1 Processor -- start 2024-04-23T19:12:14.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.667+0000 7f48aba38800 1 -- start start 2024-04-23T19:12:14.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48aba38800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55f03d244c00 0x55f03d243160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48aba38800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f03d1817f0 con 0x55f03d2436a0 2024-04-23T19:12:14.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48aba38800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x55f03d170e20 con 0x55f03d244c00 2024-04-23T19:12:14.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48a9537640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x55f03d244c00 0x55f03d243160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48a8d36640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x55f03d2436a0 0x55f03d2d7010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56774/0 (socket says 172.21.15.63:56774) 2024-04-23T19:12:14.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48a8d36640 1 -- 172.21.15.63:0/653937670 learned_addr learned my addr 172.21.15.63:0/653937670 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:14.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48a9537640 1 -- 172.21.15.63:0/653937670 >> v1:172.21.15.16:6789/0 conn(0x55f03d2436a0 legacy=0x55f03d2d7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.668+0000 7f48a9537640 1 -- 172.21.15.63:0/653937670 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03d18fb50 con 0x55f03d244c00 2024-04-23T19:12:14.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.670+0000 7f48a9537640 1 --2- 172.21.15.63:0/653937670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d244c00 0x55f03d243160 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f489c00a470 tx=0x7f489c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d431c2c4cd315a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.670+0000 7f489bfff640 1 -- 172.21.15.63:0/653937670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c057070 con 0x55f03d244c00 2024-04-23T19:12:14.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.670+0000 7f489bfff640 1 -- 172.21.15.63:0/653937670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489c04ac00 con 0x55f03d244c00 2024-04-23T19:12:14.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.670+0000 7f489bfff640 1 -- 172.21.15.63:0/653937670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c052440 con 0x55f03d244c00 2024-04-23T19:12:14.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.671+0000 7f48aba38800 1 -- 172.21.15.63:0/653937670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d244c00 msgr2=0x55f03d243160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.671+0000 7f48aba38800 1 --2- 172.21.15.63:0/653937670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d244c00 0x55f03d243160 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f489c00a470 tx=0x7f489c01cbd0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.671+0000 7f48aba38800 1 -- 172.21.15.63:0/653937670 shutdown_connections 2024-04-23T19:12:14.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.671+0000 7f48aba38800 1 --2- 172.21.15.63:0/653937670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d244c00 0x55f03d243160 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.671+0000 7f48aba38800 1 -- 172.21.15.63:0/653937670 >> 172.21.15.63:0/653937670 conn(0x55f03d2ca7e0 msgr2=0x55f03d2ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.671+0000 7f48aba38800 1 -- 172.21.15.63:0/653937670 shutdown_connections 2024-04-23T19:12:14.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.671+0000 7f48aba38800 1 -- 172.21.15.63:0/653937670 wait complete. 2024-04-23T19:12:14.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.684+0000 7f48aba38800 1 Processor -- start 2024-04-23T19:12:14.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.684+0000 7f48aba38800 1 -- start start 2024-04-23T19:12:14.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.684+0000 7f48aba38800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.684+0000 7f48aba38800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f03d21a6f0 con 0x55f03d2436a0 2024-04-23T19:12:14.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f48a9537640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f48a9537640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40880/0 (socket says 172.21.15.63:40880) 2024-04-23T19:12:14.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f48a9537640 1 -- 172.21.15.63:0/2223443196 learned_addr learned my addr 172.21.15.63:0/2223443196 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f48a9537640 1 -- 172.21.15.63:0/2223443196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03d3c2fe0 con 0x55f03d2436a0 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f48a9537640 1 --2- 172.21.15.63:0/2223443196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3c2a10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f489c055040 tx=0x7f489c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ad39f85cc006fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f4797606640 1 -- 172.21.15.63:0/2223443196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c057040 con 0x55f03d2436a0 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f4797606640 1 -- 172.21.15.63:0/2223443196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489c063070 con 0x55f03d2436a0 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.685+0000 7f4797606640 1 -- 172.21.15.63:0/2223443196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c052850 con 0x55f03d2436a0 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 -- 172.21.15.63:0/2223443196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 msgr2=0x55f03d3c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 --2- 172.21.15.63:0/2223443196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3c2a10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f489c055040 tx=0x7f489c04c000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 -- 172.21.15.63:0/2223443196 shutdown_connections 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 --2- 172.21.15.63:0/2223443196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3c2a10 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 -- 172.21.15.63:0/2223443196 >> 172.21.15.63:0/2223443196 conn(0x55f03d2ca7e0 msgr2=0x55f03d3be330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 -- 172.21.15.63:0/2223443196 shutdown_connections 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 -- 172.21.15.63:0/2223443196 wait complete. 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.686+0000 7f48aba38800 1 Processor -- start 2024-04-23T19:12:14.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.687+0000 7f48aba38800 1 -- start start 2024-04-23T19:12:14.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.687+0000 7f48aba38800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.687+0000 7f48aba38800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f03d2da9b0 con 0x55f03d2436a0 2024-04-23T19:12:14.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.687+0000 7f48a9537640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.687+0000 7f48a9537640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40892/0 (socket says 172.21.15.63:40892) 2024-04-23T19:12:14.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.687+0000 7f48a9537640 1 -- 172.21.15.63:0/2495063720 learned_addr learned my addr 172.21.15.63:0/2495063720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.687+0000 7f48a9537640 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03d3ca950 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.688+0000 7f48a9537640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3ca410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f489c04abd0 tx=0x7f489c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.688+0000 7f4795e03640 1 -- 172.21.15.63:0/2495063720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c06d070 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.688+0000 7f4795e03640 1 -- 172.21.15.63:0/2495063720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489c052bf0 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.688+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f03d3cabe0 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.688+0000 7f4795e03640 1 -- 172.21.15.63:0/2495063720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c057050 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.688+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03d3cd840 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f4795e03640 1 -- 172.21.15.63:0/2495063720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f489c05d470 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f4795e03640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f4795e03640 1 -- 172.21.15.63:0/2495063720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f489c08ddf0 con 0x55f03d2436a0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f48a8d36640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d244c00 0x55f03d3d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d3d1be0 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f48a9d38640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d244c00 0x55f03d3d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.689+0000 7f48a8d36640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f48a0000f60 tx=0x7f48a00012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.690+0000 7f48a9d38640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d244c00 0x55f03d3d1600 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.690+0000 7f48a9d38640 1 -- 172.21.15.63:0/2495063720 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48a400a4a0 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.690+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d3cebd0 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.690+0000 7f48a9d38640 1 -- 172.21.15.63:0/2495063720 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f48a400a4a0 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.690+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d3d3a00 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.690+0000 7f48a9d38640 1 -- 172.21.15.63:0/2495063720 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f48a400a4a0 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.690+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d3d2d40 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.691+0000 7f48a9d38640 1 -- 172.21.15.63:0/2495063720 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f48a400a4a0 con 0x55f03d244c00 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.691+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f03d3d4db0 0x55f03d3d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.691+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d3d7720 con 0x55f03d3d4db0 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.691+0000 7f48a9537640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f03d3d4db0 0x55f03d3d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.691+0000 7f48a9537640 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f03d3d4db0 0x55f03d3d71a0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48a9537640 1 -- 172.21.15.63:0/2495063720 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f489c05a040 con 0x55f03d3d4db0 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48aba38800 1 Processor -- start 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48aba38800 1 -- start start 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48aba38800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3e5b40 0x55f03d3e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48aba38800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f03d1c0e30 con 0x55f03d3e5b40 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48a9d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3e5b40 0x55f03d3e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48a9d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3e5b40 0x55f03d3e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40902/0 (socket says 172.21.15.63:40902) 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.692+0000 7f48a9d38640 1 -- 172.21.15.63:0/1628868319 learned_addr learned my addr 172.21.15.63:0/1628868319 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.693+0000 7f48a9d38640 1 -- 172.21.15.63:0/1628868319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03d3e64a0 con 0x55f03d3e5b40 2024-04-23T19:12:14.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.693+0000 7f48a9d38640 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3e5b40 0x55f03d3e5f60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55f03d3d1fc0 tx=0x7f48a40178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.693+0000 7f478effd640 1 -- 172.21.15.63:0/1628868319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a4013440 con 0x55f03d3e5b40 2024-04-23T19:12:14.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.693+0000 7f478effd640 1 -- 172.21.15.63:0/1628868319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a40135e0 con 0x55f03d3e5b40 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.693+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f03d3e6f10 con 0x55f03d3e5b40 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.693+0000 7f478effd640 1 -- 172.21.15.63:0/1628868319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a40138c0 con 0x55f03d3e5b40 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.693+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03d3e7410 con 0x55f03d3e5b40 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f478effd640 1 -- 172.21.15.63:0/1628868319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f48a4011040 con 0x55f03d3e5b40 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f478effd640 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4780036eb0 0x7f4780039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f478effd640 1 -- 172.21.15.63:0/1628868319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f48a401d070 con 0x55f03d3e5b40 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f48a9537640 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4780036eb0 0x7f4780039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f48aba38800 1 Processor -- start 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f48aba38800 1 -- start start 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f48a9537640 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4780036eb0 0x7f4780039370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f489c08c230 tx=0x7f489c05c980 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f48aba38800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d3f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.694+0000 7f48aba38800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f03d3c32a0 con 0x55f03d3f2de0 2024-04-23T19:12:14.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f48a9d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d3f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f48a9d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d3f3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40908/0 (socket says 172.21.15.63:40908) 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f48a9d38640 1 -- 172.21.15.63:0/3082119949 learned_addr learned my addr 172.21.15.63:0/3082119949 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f48a9d38640 1 -- 172.21.15.63:0/3082119949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03d3f37d0 con 0x55f03d3f2de0 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f48a9d38640 1 --2- 172.21.15.63:0/3082119949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d3f3200 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f48a404cc30 tx=0x7f48a4012770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1622bca52c9390b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f478cff9640 1 -- 172.21.15.63:0/3082119949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a4012c90 con 0x55f03d3f2de0 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f478cff9640 1 -- 172.21.15.63:0/3082119949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a4012e30 con 0x55f03d3f2de0 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.695+0000 7f478cff9640 1 -- 172.21.15.63:0/3082119949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a40124d0 con 0x55f03d3f2de0 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 -- 172.21.15.63:0/3082119949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 msgr2=0x55f03d3f3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 --2- 172.21.15.63:0/3082119949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d3f3200 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f48a404cc30 tx=0x7f48a4012770 comp rx=0 tx=0).stop 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 -- 172.21.15.63:0/3082119949 shutdown_connections 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 --2- 172.21.15.63:0/3082119949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d3f3200 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 -- 172.21.15.63:0/3082119949 >> 172.21.15.63:0/3082119949 conn(0x55f03d3ef0f0 msgr2=0x55f03d3ed1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 -- 172.21.15.63:0/3082119949 shutdown_connections 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 -- 172.21.15.63:0/3082119949 wait complete. 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 Processor -- start 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.696+0000 7f48aba38800 1 -- start start 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48aba38800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d2d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48aba38800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f03d3f3fa0 con 0x55f03d3f2de0 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48a9d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d2d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48a9d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d2d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40910/0 (socket says 172.21.15.63:40910) 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 learned_addr learned my addr 172.21.15.63:0/869865023 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03d2d29f0 con 0x55f03d3f2de0 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48a9d38640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d2d24b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f48a4015fd0 tx=0x7f48a404cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f477effd640 1 -- 172.21.15.63:0/869865023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a403a070 con 0x55f03d3f2de0 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f477effd640 1 -- 172.21.15.63:0/869865023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a402aa60 con 0x55f03d3f2de0 2024-04-23T19:12:14.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f477effd640 1 -- 172.21.15.63:0/869865023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a402adc0 con 0x55f03d3f2de0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f03d2d2c80 con 0x55f03d3f2de0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.697+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03d2cf240 con 0x55f03d3f2de0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.698+0000 7f477effd640 1 -- 172.21.15.63:0/869865023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f48a402a350 con 0x55f03d3f2de0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.698+0000 7f477effd640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4774036eb0 0x7f4774039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.698+0000 7f477effd640 1 -- 172.21.15.63:0/869865023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f48a4032070 con 0x55f03d3f2de0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.698+0000 7f48a9537640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4774036eb0 0x7f4774039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.699+0000 7f48a9537640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4774036eb0 0x7f4774039370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55f03d3d7b00 tx=0x7f489c05d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.699+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.699+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d40a3e0 0x55f03d40c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.699+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d40cff0 con 0x55f03d40a3e0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.699+0000 7f48a8d36640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d40a3e0 0x55f03d40c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.699+0000 7f48a8d36640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d40a3e0 0x55f03d40c8a0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48a0009050 con 0x55f03d40a3e0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 realm 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d409830 con 0x55f03d40a3e0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48a0009050 con 0x55f03d40a3e0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d40dae0 con 0x55f03d40a3e0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48a0009050 con 0x55f03d40a3e0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d40e3b0 con 0x55f03d40a3e0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f48a0009050 con 0x55f03d40a3e0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d40e750 con 0x55f03d40a3e0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.700+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f48a0009050 con 0x55f03d40a3e0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48aba38800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48aba38800 20 searching for the correct realm 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f03d410630 0x55f03d412a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48a9d38640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f03d410630 0x55f03d412a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48a9d38640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f03d410630 0x55f03d412a40 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48a9d38640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48a402c940 0x7f48a4022420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f48a402c940 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.701+0000 7f48a9537640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48a402c940 0x7f48a4022420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.702+0000 7f48a9537640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48a402c940 0x7f48a4022420 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.702+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.702+0000 7f48a9537640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f489c08d320 0x7f489c08d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.702+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.702+0000 7f48a8d36640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f489c08d320 0x7f489c08d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.703+0000 7f48a8d36640 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f489c08d320 0x7f489c08d760 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.703+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a0009050 con 0x7f489c08d320 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.703+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.703+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a0009050 con 0x55f03d40a3e0 2024-04-23T19:12:14.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.703+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.703+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.703+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f48a402c940 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.704+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.705+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.705+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.705+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.705+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.705+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.705+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f48a402c940 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.706+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.707+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f48a402c940 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.708+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f48a402c940 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f48a402c940 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.709+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d410630 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.710+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.710+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f489c08d320 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.710+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.710+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x55f03d40a3e0 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.710+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.710+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413110 con 0x7f48a402c940 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d40f0a0 con 0x55f03d40a3e0 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d40f940 con 0x55f03d40a3e0 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.711+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413590 con 0x7f489c08d320 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 zone default found 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 4 Realm: () 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:14.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 zonegroup default 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d413640 con 0x7f48a402c940 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d415910 con 0x55f03d410630 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d415cb0 con 0x55f03d410630 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 started sync module instance, tier type = 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.712+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d414da0 con 0x7f48a402c940 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.714+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.714+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94490308202896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d414a00 con 0x7f48a402c940 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.714+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d416a20 con 0x55f03d40a3e0 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.715+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94490308202896] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.715+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.715+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94490308206784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d416990 con 0x55f03d40a3e0 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.715+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d4174c0 con 0x55f03d410630 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.716+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.716+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94490308210656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d4174c0 con 0x55f03d410630 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.716+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d4184e0 con 0x7f48a402c940 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.716+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94490308206784] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.717+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94490308210656] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.717+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.717+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94490308212960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d4184a0 con 0x7f48a402c940 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.717+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d418d80 con 0x55f03d40a3e0 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.718+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94490308212960] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.718+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.718+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94490308214704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d4184a0 con 0x55f03d40a3e0 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.718+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d4194d0 con 0x7f489c08d320 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.719+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94490308214704] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.720+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.720+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94490308218784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d419490 con 0x7f489c08d320 2024-04-23T19:12:14.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.720+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d419f00 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.721+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94490308218784] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.721+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.721+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94490308221488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d419f00 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.721+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d41a910 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94490308221488] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94490308217040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d41a910 con 0x7f489c08d320 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 20 add_watcher() i=0 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 20 add_watcher() i=1 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 20 add_watcher() i=2 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 20 add_watcher() i=3 2024-04-23T19:12:14.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 20 add_watcher() i=4 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 20 add_watcher() i=5 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.722+0000 7f48aba38800 20 add_watcher() i=6 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.723+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94490308217040] v32'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.723+0000 7f48aba38800 20 add_watcher() i=7 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.723+0000 7f48aba38800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.723+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d41dd40 con 0x55f03d40a3e0 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.723+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.723+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94490308224560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d41b290 con 0x55f03d40a3e0 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.724+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94490308224560] v32'261 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.724+0000 7f475f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.724+0000 7f48aba38800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.724+0000 7f48aba38800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:14.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f48aba38800 5 note: GC not initialized 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f475e7e4640 20 reqs_thread_entry: start 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f48aba38800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f48aba38800 20 Filter name: none 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f46cffff640 20 reqs_thread_entry: start 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f48aba38800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f48aba38800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.725+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d451290 con 0x7f489c08d320 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.726+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.726+0000 7f48aba38800 20 rados_obj.operate() r=0 bl.length=490 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.726+0000 7f48aba38800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.726+0000 7f48aba38800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.726+0000 7f48aba38800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:14.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.726+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~503 in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d457b60 con 0x7f489c08d320 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~503] v32'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48aba38800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48aba38800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48aba38800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94490308459296 in=691b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f03d455400 con 0x55f03d40a3e0 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94490308214704 notify 137438953502 ret 0) ==== 721+0+0 (crc 0 0 0) 0x55f03d1817f0 con 0x55f03d40a3e0 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f478cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953502 cookie 94490308214704 notifier 4884 bl.length()=679 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f478cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f478cff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94490308459296 out=8b] v0'0 uv264 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f478cff9640 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f476c003ea0 con 0x55f03d40a3e0 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.727+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.728+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 94490308459296 notify 137438953502 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f48a003fa70 con 0x55f03d40a3e0 2024-04-23T19:12:14.736 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.728+0000 7f475f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.728+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94490308224560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d406360 con 0x55f03d40a3e0 2024-04-23T19:12:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.729+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94490308224560] v32'262 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.730+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94490308202896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d41a600 con 0x7f48a402c940 2024-04-23T19:12:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.731+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94490308202896] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.731+0000 7f48aba38800 20 remove_watcher() i=0 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.731+0000 7f48aba38800 2 removed watcher, disabling cache 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.731+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94490308206784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d415190 con 0x55f03d40a3e0 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.732+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94490308206784] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.732+0000 7f48aba38800 20 remove_watcher() i=1 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.732+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94490308210656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d415190 con 0x55f03d410630 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.733+0000 7f48a9d38640 1 -- 172.21.15.63:0/869865023 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94490308210656] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a4029070 con 0x55f03d410630 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.733+0000 7f48aba38800 20 remove_watcher() i=2 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.733+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94490308212960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d415190 con 0x7f48a402c940 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.734+0000 7f48a9537640 1 -- 172.21.15.63:0/869865023 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94490308212960] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f489c05a040 con 0x7f48a402c940 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.734+0000 7f48aba38800 20 remove_watcher() i=3 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.734+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94490308214704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d4178e0 con 0x55f03d40a3e0 2024-04-23T19:12:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.735+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94490308214704] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x55f03d40a3e0 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.735+0000 7f48aba38800 20 remove_watcher() i=4 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.735+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94490308218784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d415190 con 0x7f489c08d320 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.736+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94490308218784] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.736+0000 7f48aba38800 20 remove_watcher() i=5 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.736+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94490308221488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d418fa0 con 0x7f489c08d320 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.737+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94490308221488] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.737+0000 7f48aba38800 20 remove_watcher() i=6 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.737+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94490308217040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55f03d415190 con 0x7f489c08d320 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94490308217040] v32'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f48a002a040 con 0x7f489c08d320 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 20 remove_watcher() i=7 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f489c08d320 msgr2=0x7f489c08d760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f489c08d320 0x7f489c08d760 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d40a3e0 msgr2=0x55f03d40c8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d40a3e0 0x55f03d40c8a0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48a402c940 msgr2=0x7f48a4022420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f48a402c940 0x7f48a4022420 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f03d410630 msgr2=0x55f03d412a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x55f03d410630 0x55f03d412a40 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4774036eb0 msgr2=0x7f4774039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4774036eb0 0x7f4774039370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55f03d3d7b00 tx=0x7f489c05d000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 msgr2=0x55f03d2d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48a8d36640 1 -- 172.21.15.63:0/869865023 reap_dead start 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.738+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d2d24b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f48a4015fd0 tx=0x7f48a404cbc0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 shutdown_connections 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/869865023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3f2de0 0x55f03d2d24b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 >> 172.21.15.63:0/869865023 conn(0x55f03d3ef0f0 msgr2=0x55f03d3ec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 shutdown_connections 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/869865023 wait complete. 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4780036eb0 msgr2=0x7f4780039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4780036eb0 0x7f4780039370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f489c08c230 tx=0x7f489c05c980 comp rx=0 tx=0).stop 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3e5b40 msgr2=0x55f03d3e5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3e5b40 0x55f03d3e5f60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55f03d3d1fc0 tx=0x7f48a40178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 shutdown_connections 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4780036eb0 0x7f4780039370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/1628868319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d3e5b40 0x55f03d3e5f60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 >> 172.21.15.63:0/1628868319 conn(0x55f03d3d9be0 msgr2=0x55f03d3d9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 shutdown_connections 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/1628868319 wait complete. 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f03d3d4db0 msgr2=0x55f03d3d71a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f03d3d4db0 0x55f03d3d71a0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d244c00 msgr2=0x55f03d3d1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d244c00 0x55f03d3d1600 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 msgr2=0x7f4790039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f48a0000f60 tx=0x7f48a00012b0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.739+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 msgr2=0x55f03d3ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3ca410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f489c04abd0 tx=0x7f489c002fb0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 shutdown_connections 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x55f03d3d4db0 0x55f03d3d71a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55f03d244c00 0x55f03d3d1600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f4790036e60 0x7f4790039320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 --2- 172.21.15.63:0/2495063720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f03d2436a0 0x55f03d3ca410 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 >> 172.21.15.63:0/2495063720 conn(0x55f03d2ca7e0 msgr2=0x55f03d3bcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 shutdown_connections 2024-04-23T19:12:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.740+0000 7f48aba38800 1 -- 172.21.15.63:0/2495063720 wait complete. 2024-04-23T19:12:14.749 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.749 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.749 ERROR:root:0 2024-04-23T19:12:14.749 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:14.753 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:14.753 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-04-23T19:12:14.754 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-04-23T19:12:14.754 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write; user-info-without-keys=*' 2024-04-23T19:12:14.799 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:14.800 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee97a2800 1 Processor -- start 2024-04-23T19:12:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee97a2800 1 -- start start 2024-04-23T19:12:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee97a2800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5639515c0740 0x5639515c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee97a2800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56395146e7f0 con 0x5639515bcc30 2024-04-23T19:12:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee97a2800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x56395145de20 con 0x5639515c0740 2024-04-23T19:12:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee6aa7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5639515c0740 0x5639515c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee72a8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5639515bcc30 0x5639515bf020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56778/0 (socket says 172.21.15.63:56778) 2024-04-23T19:12:14.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.800+0000 7faee72a8640 1 -- 172.21.15.63:0/2342010522 learned_addr learned my addr 172.21.15.63:0/2342010522 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:14.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.801+0000 7faee6aa7640 1 -- 172.21.15.63:0/2342010522 >> v1:172.21.15.16:6789/0 conn(0x5639515bcc30 legacy=0x5639515bf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.801+0000 7faee6aa7640 1 -- 172.21.15.63:0/2342010522 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56395147cb50 con 0x5639515c0740 2024-04-23T19:12:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee6aa7640 1 --2- 172.21.15.63:0/2342010522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515c0740 0x5639515c2b30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faedc00a470 tx=0x7faedc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36046153db1462f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee62a6640 1 -- 172.21.15.63:0/2342010522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faedc057070 con 0x5639515c0740 2024-04-23T19:12:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee62a6640 1 -- 172.21.15.63:0/2342010522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faedc052570 con 0x5639515c0740 2024-04-23T19:12:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee62a6640 1 -- 172.21.15.63:0/2342010522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faedc052870 con 0x5639515c0740 2024-04-23T19:12:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee97a2800 1 -- 172.21.15.63:0/2342010522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515c0740 msgr2=0x5639515c2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee97a2800 1 --2- 172.21.15.63:0/2342010522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515c0740 0x5639515c2b30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faedc00a470 tx=0x7faedc01ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee97a2800 1 -- 172.21.15.63:0/2342010522 shutdown_connections 2024-04-23T19:12:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee97a2800 1 --2- 172.21.15.63:0/2342010522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515c0740 0x5639515c2b30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee97a2800 1 -- 172.21.15.63:0/2342010522 >> 172.21.15.63:0/2342010522 conn(0x5639515b6600 msgr2=0x5639515b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee97a2800 1 -- 172.21.15.63:0/2342010522 shutdown_connections 2024-04-23T19:12:14.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.803+0000 7faee97a2800 1 -- 172.21.15.63:0/2342010522 wait complete. 2024-04-23T19:12:14.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.816+0000 7faee97a2800 1 Processor -- start 2024-04-23T19:12:14.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.816+0000 7faee97a2800 1 -- start start 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.816+0000 7faee97a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639515bc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.816+0000 7faee97a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639515076f0 con 0x5639515bcc30 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.816+0000 7faee72a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639515bc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.816+0000 7faee72a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639515bc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40926/0 (socket says 172.21.15.63:40926) 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.816+0000 7faee72a8640 1 -- 172.21.15.63:0/3754465884 learned_addr learned my addr 172.21.15.63:0/3754465884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.817+0000 7faee72a8640 1 -- 172.21.15.63:0/3754465884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639515bab30 con 0x5639515bcc30 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.817+0000 7faee72a8640 1 --2- 172.21.15.63:0/3754465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639515bc570 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faed8001d50 tx=0x7faed800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feafc6dd91993c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.817+0000 7fadd560a640 1 -- 172.21.15.63:0/3754465884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faed8013030 con 0x5639515bcc30 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.817+0000 7fadd560a640 1 -- 172.21.15.63:0/3754465884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed800e470 con 0x5639515bcc30 2024-04-23T19:12:14.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.817+0000 7fadd560a640 1 -- 172.21.15.63:0/3754465884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faed800e810 con 0x5639515bcc30 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 -- 172.21.15.63:0/3754465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 msgr2=0x5639515bc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 --2- 172.21.15.63:0/3754465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639515bc570 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faed8001d50 tx=0x7faed800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 -- 172.21.15.63:0/3754465884 shutdown_connections 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 --2- 172.21.15.63:0/3754465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639515bc570 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 -- 172.21.15.63:0/3754465884 >> 172.21.15.63:0/3754465884 conn(0x5639515b6600 msgr2=0x5639516a8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 -- 172.21.15.63:0/3754465884 shutdown_connections 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 -- 172.21.15.63:0/3754465884 wait complete. 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 Processor -- start 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 -- start start 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639516b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee97a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639515c5380 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee72a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639516b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee72a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639516b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40928/0 (socket says 172.21.15.63:40928) 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.818+0000 7faee72a8640 1 -- 172.21.15.63:0/1847684389 learned_addr learned my addr 172.21.15.63:0/1847684389 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.819+0000 7faee72a8640 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639516b7650 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.819+0000 7faee72a8640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639516b7110 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x5639515bbba0 tx=0x7faed8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.819+0000 7fadd3e07640 1 -- 172.21.15.63:0/1847684389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faed8027070 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.819+0000 7fadd3e07640 1 -- 172.21.15.63:0/1847684389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed8019900 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.819+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639516b78e0 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.819+0000 7fadd3e07640 1 -- 172.21.15.63:0/1847684389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faed8019c40 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.819+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639516ba590 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.820+0000 7fadd3e07640 1 -- 172.21.15.63:0/1847684389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7faed802a070 con 0x5639515bcc30 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.820+0000 7fadd3e07640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadcc036e60 0x7fadcc039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.820+0000 7fadd3e07640 1 -- 172.21.15.63:0/1847684389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7faed801f070 con 0x5639515bcc30 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.820+0000 7faee6aa7640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadcc036e60 0x7fadcc039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.820+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639515c0740 0x5639516be350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.820+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516be920 con 0x5639515c0740 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.820+0000 7faee7aa9640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639515c0740 0x5639516be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.821+0000 7faee6aa7640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadcc036e60 0x7fadcc039320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faedc054000 tx=0x7faedc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.821+0000 7faee7aa9640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639515c0740 0x5639516be350 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.821+0000 7faee7aa9640 1 -- 172.21.15.63:0/1847684389 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7faee000a4a0 con 0x5639515c0740 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.821+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516bb920 con 0x5639515c0740 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.821+0000 7faee7aa9640 1 -- 172.21.15.63:0/1847684389 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7faee000a4a0 con 0x5639515c0740 2024-04-23T19:12:14.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.821+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516c0720 con 0x5639515c0740 2024-04-23T19:12:14.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.822+0000 7faee7aa9640 1 -- 172.21.15.63:0/1847684389 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7faee000a4a0 con 0x5639515c0740 2024-04-23T19:12:14.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.822+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516bfa60 con 0x5639515c0740 2024-04-23T19:12:14.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.822+0000 7faee7aa9640 1 -- 172.21.15.63:0/1847684389 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7faee000a4a0 con 0x5639515c0740 2024-04-23T19:12:14.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.822+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5639516c1ad0 0x5639516c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.822+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516c4440 con 0x5639516c1ad0 2024-04-23T19:12:14.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.822+0000 7faee72a8640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5639516c1ad0 0x5639516c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.823+0000 7faee72a8640 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5639516c1ad0 0x5639516c3ec0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.823+0000 7faee72a8640 1 -- 172.21.15.63:0/1847684389 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7faed8029030 con 0x5639516c1ad0 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.823+0000 7faee97a2800 1 Processor -- start 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.823+0000 7faee97a2800 1 -- start start 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee97a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516d2840 0x5639516d2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee97a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639514ade30 con 0x5639516d2840 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee7aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516d2840 0x5639516d2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee7aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516d2840 0x5639516d2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40930/0 (socket says 172.21.15.63:40930) 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee7aa9640 1 -- 172.21.15.63:0/3336904977 learned_addr learned my addr 172.21.15.63:0/3336904977 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee7aa9640 1 -- 172.21.15.63:0/3336904977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639516d31a0 con 0x5639516d2840 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee7aa9640 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516d2840 0x5639516d2c60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x5639516bece0 tx=0x7faee00178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7fadd1602640 1 -- 172.21.15.63:0/3336904977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee0013440 con 0x5639516d2840 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7fadd1602640 1 -- 172.21.15.63:0/3336904977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee00135e0 con 0x5639516d2840 2024-04-23T19:12:14.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7fadd1602640 1 -- 172.21.15.63:0/3336904977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee00138c0 con 0x5639516d2840 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.824+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639516d3c10 con 0x5639516d2840 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.825+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639516d4110 con 0x5639516d2840 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.825+0000 7fadd1602640 1 -- 172.21.15.63:0/3336904977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7faee0011040 con 0x5639516d2840 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.825+0000 7fadd1602640 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadc0036e60 0x7fadc0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.825+0000 7fadd1602640 1 -- 172.21.15.63:0/3336904977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7faee001d070 con 0x5639516d2840 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.825+0000 7faee72a8640 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadc0036e60 0x7fadc0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee97a2800 1 Processor -- start 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee97a2800 1 -- start start 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee72a8640 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadc0036e60 0x7fadc0039320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5639516c4820 tx=0x7faed800ef30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee97a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516dfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee97a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639515baeb0 con 0x5639516dfbd0 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee7aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516dfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee7aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516dfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40946/0 (socket says 172.21.15.63:40946) 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee7aa9640 1 -- 172.21.15.63:0/4227614479 learned_addr learned my addr 172.21.15.63:0/4227614479 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee7aa9640 1 -- 172.21.15.63:0/4227614479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639516e05c0 con 0x5639516dfbd0 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.826+0000 7faee7aa9640 1 --2- 172.21.15.63:0/4227614479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516dfff0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faee0001470 tx=0x7faee0012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb0a9ffb23ad9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7fadcaffd640 1 -- 172.21.15.63:0/4227614479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee0043070 con 0x5639516dfbd0 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7fadcaffd640 1 -- 172.21.15.63:0/4227614479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee0015430 con 0x5639516dfbd0 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7fadcaffd640 1 -- 172.21.15.63:0/4227614479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee002ac50 con 0x5639516dfbd0 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 -- 172.21.15.63:0/4227614479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 msgr2=0x5639516dfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 --2- 172.21.15.63:0/4227614479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516dfff0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faee0001470 tx=0x7faee0012a30 comp rx=0 tx=0).stop 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 -- 172.21.15.63:0/4227614479 shutdown_connections 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 --2- 172.21.15.63:0/4227614479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516dfff0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 -- 172.21.15.63:0/4227614479 >> 172.21.15.63:0/4227614479 conn(0x5639516dbea0 msgr2=0x5639516d9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 -- 172.21.15.63:0/4227614479 shutdown_connections 2024-04-23T19:12:14.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 -- 172.21.15.63:0/4227614479 wait complete. 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 Processor -- start 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.827+0000 7faee97a2800 1 -- start start 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee97a2800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee97a2800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639516e0d50 con 0x5639516dfbd0 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee7aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee7aa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516e70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40962/0 (socket says 172.21.15.63:40962) 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 learned_addr learned my addr 172.21.15.63:0/449949345 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639516e7620 con 0x5639516dfbd0 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee7aa9640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516e70e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faee004cb80 tx=0x7faee004cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7fadc97fa640 1 -- 172.21.15.63:0/449949345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee0043030 con 0x5639516dfbd0 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7fadc97fa640 1 -- 172.21.15.63:0/449949345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee00017f0 con 0x5639516dfbd0 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7fadc97fa640 1 -- 172.21.15.63:0/449949345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee00124d0 con 0x5639516dfbd0 2024-04-23T19:12:14.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639516e7910 con 0x5639516dfbd0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.828+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639516ea560 con 0x5639516dfbd0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.829+0000 7fadc97fa640 1 -- 172.21.15.63:0/449949345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7faee004e0d0 con 0x5639516dfbd0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.829+0000 7fadc97fa640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadb0036e60 0x7fadb0039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.830+0000 7fadc97fa640 1 -- 172.21.15.63:0/449949345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7faee0044070 con 0x5639516dfbd0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.830+0000 7faee72a8640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadb0036e60 0x7fadb0039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.830+0000 7faee72a8640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadb0036e60 0x7fadb0039320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faed8046010 tx=0x7faed8043000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.830+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.830+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639516f2db0 0x5639516f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.830+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516f59c0 con 0x5639516f2db0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.830+0000 7faee6aa7640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639516f2db0 0x5639516f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee6aa7640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639516f2db0 0x5639516f5270 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 20 realm 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516f5d90 con 0x5639516f2db0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516f6660 con 0x5639516f2db0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.831+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516f6a00 con 0x5639516f2db0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee97a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516f7250 con 0x5639516f2db0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee97a2800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee97a2800 20 searching for the correct realm 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5639516f8fc0 0x5639516fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee7aa9640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5639516f8fc0 0x5639516fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.832+0000 7faee7aa9640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5639516f8fc0 0x5639516fb3b0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee7aa9640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7faee002bc00 0x7faee002c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faee002bc00 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee72a8640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7faee002bc00 0x7faee002c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee72a8640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7faee002bc00 0x7faee002c450 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee72a8640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7faed802c060 0x7faed802a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.833+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.834+0000 7faee6aa7640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7faed802c060 0x7faed802a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.834+0000 7faee6aa7640 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7faed802c060 0x7faed802a580 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.834+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.834+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.835+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.835+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.835+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.835+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.835+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.835+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faee002bc00 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.836+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faee002bc00 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.837+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.838+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.839+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.839+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.839+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.839+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.839+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.839+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.839+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faee002bc00 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faee002bc00 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.840+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.841+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.841+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.841+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faee002bc00 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.841+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.841+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f8fc0 2024-04-23T19:12:14.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.842+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.842+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faed802c060 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.842+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.842+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x5639516f2db0 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.842+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5639516fb940 con 0x7faee002bc00 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:14.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516f7ba0 con 0x5639516f2db0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516f8440 con 0x5639516f2db0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.843+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516fbdc0 con 0x7faed802c060 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 20 zone default found 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 4 Realm: () 2024-04-23T19:12:14.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 20 zonegroup default 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516fc270 con 0x7faee002bc00 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516fdff0 con 0x5639516f8fc0 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.844+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee97a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5639516fe490 con 0x5639516f8fc0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee97a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee97a2800 20 started sync module instance, tier type = 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee97a2800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.845+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516fd490 con 0x7faee002bc00 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.846+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.846+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94804179409072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516feda0 con 0x7faee002bc00 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.846+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516ff5f0 con 0x5639516f2db0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.847+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94804179409072] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.847+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.847+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94804179413232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516ff6d0 con 0x5639516f2db0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.847+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639517001f0 con 0x5639516f8fc0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.848+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94804179413232] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.848+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.849+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94804179418304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639517005e0 con 0x5639516f8fc0 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.849+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951700fe0 con 0x7faee002bc00 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.850+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94804179418304] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.850+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.850+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94804179415360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639517013d0 con 0x7faee002bc00 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.850+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951701e10 con 0x5639516f2db0 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.851+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.851+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94804179425504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951701e10 con 0x5639516f2db0 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.851+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951702e10 con 0x7faed802c060 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.851+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94804179415360] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.852+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94804179425504] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.852+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.852+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94804179429632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951701ff0 con 0x7faed802c060 2024-04-23T19:12:14.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.852+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639517036b0 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.853+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94804179429632] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.853+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.853+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94804179421936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951702e10 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.853+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951703cd0 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94804179421936] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94804179431376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951703cd0 con 0x7faed802c060 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 20 add_watcher() i=0 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 20 add_watcher() i=1 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 20 add_watcher() i=2 2024-04-23T19:12:14.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 20 add_watcher() i=3 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 20 add_watcher() i=4 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 20 add_watcher() i=5 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.854+0000 7faee97a2800 20 add_watcher() i=6 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.855+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94804179431376] v32'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.855+0000 7faee97a2800 20 add_watcher() i=7 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.855+0000 7faee97a2800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.855+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563951707010 con 0x5639516f2db0 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.855+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.855+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94804179446320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951703e40 con 0x5639516f2db0 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94804179446320] v32'263 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7fad9dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:14.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 5 note: GC not initialized 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7fad9cfe1640 20 reqs_thread_entry: start 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 20 Filter name: none 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7fad167fc640 20 reqs_thread_entry: start 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.857+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56395173a530 con 0x7faed802c060 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.858+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~503 out=503b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+626 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.858+0000 7faee97a2800 20 rados_obj.operate() r=0 bl.length=503 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.858+0000 7faee97a2800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.858+0000 7faee97a2800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.858+0000 7faee97a2800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.858+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951740c60 con 0x7faed802c060 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v32'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee97a2800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee97a2800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee97a2800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94804179683024 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56395173f7b0 con 0x5639516f2db0 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94804179425504 notify 137438953503 ret 0) ==== 678+0+0 (crc 0 0 0) 0x56395147cb50 con 0x5639516f2db0 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7fadcaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953503 cookie 94804179425504 notifier 4891 bl.length()=636 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94804179683024 out=8b] v0'0 uv267 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7fadcaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7fadcaffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7fadcaffd640 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fadac003ea0 con 0x5639516f2db0 2024-04-23T19:12:14.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== watch-notify(notify_complete (2) cookie 94804179683024 notify 137438953503 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7faedc08f550 con 0x5639516f2db0 2024-04-23T19:12:14.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.860+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.870 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:14.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.861+0000 7fad9dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.861+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94804179446320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x56395173fbc0 con 0x5639516f2db0 2024-04-23T19:12:14.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.862+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94804179446320] v32'264 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.862+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94804179409072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563951706a30 con 0x7faee002bc00 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.863+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94804179409072] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.863+0000 7faee97a2800 20 remove_watcher() i=0 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.863+0000 7faee97a2800 2 removed watcher, disabling cache 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.863+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94804179413232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516fd8b0 con 0x5639516f2db0 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.865+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94804179413232] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.865+0000 7faee97a2800 20 remove_watcher() i=1 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.865+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94804179418304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516fe8f0 con 0x5639516f8fc0 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.866+0000 7faee7aa9640 1 -- 172.21.15.63:0/449949345 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94804179418304] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faee0029070 con 0x5639516f8fc0 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.866+0000 7faee97a2800 20 remove_watcher() i=2 2024-04-23T19:12:14.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.866+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94804179415360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516fe8f0 con 0x7faee002bc00 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.867+0000 7faee72a8640 1 -- 172.21.15.63:0/449949345 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94804179415360] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faed803f070 con 0x7faee002bc00 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.867+0000 7faee97a2800 20 remove_watcher() i=3 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.867+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94804179425504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516ff140 con 0x5639516f2db0 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.868+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94804179425504] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x5639516f2db0 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.868+0000 7faee97a2800 20 remove_watcher() i=4 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.868+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94804179429632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516fe8f0 con 0x7faed802c060 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.869+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94804179429632] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.869+0000 7faee97a2800 20 remove_watcher() i=5 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.869+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94804179421936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516ff140 con 0x7faed802c060 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.870+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94804179421936] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.870+0000 7faee97a2800 20 remove_watcher() i=6 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.870+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94804179431376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5639516fe8f0 con 0x7faed802c060 2024-04-23T19:12:14.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94804179431376] v32'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7faedc05a040 con 0x7faed802c060 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 20 remove_watcher() i=7 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7faed802c060 msgr2=0x7faed802a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7faed802c060 0x7faed802a580 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639516f2db0 msgr2=0x5639516f5270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639516f2db0 0x5639516f5270 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7faee002bc00 msgr2=0x7faee002c450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7faee002bc00 0x7faee002c450 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5639516f8fc0 msgr2=0x5639516fb3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5639516f8fc0 0x5639516fb3b0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadb0036e60 msgr2=0x7fadb0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadb0036e60 0x7fadb0039320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faed8046010 tx=0x7faed8043000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 msgr2=0x5639516e70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516e70e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faee004cb80 tx=0x7faee004cbb0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee6aa7640 1 -- 172.21.15.63:0/449949345 reap_dead start 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 shutdown_connections 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/449949345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516dfbd0 0x5639516e70e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 >> 172.21.15.63:0/449949345 conn(0x5639516dbea0 msgr2=0x5639516d9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 shutdown_connections 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/449949345 wait complete. 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadc0036e60 msgr2=0x7fadc0039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadc0036e60 0x7fadc0039320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5639516c4820 tx=0x7faed800ef30 comp rx=0 tx=0).stop 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516d2840 msgr2=0x5639516d2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516d2840 0x5639516d2c60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x5639516bece0 tx=0x7faee00178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 shutdown_connections 2024-04-23T19:12:14.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadc0036e60 0x7fadc0039320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 --2- 172.21.15.63:0/3336904977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639516d2840 0x5639516d2c60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 >> 172.21.15.63:0/3336904977 conn(0x5639516c68e0 msgr2=0x5639516c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.872+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 shutdown_connections 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/3336904977 wait complete. 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5639516c1ad0 msgr2=0x5639516c3ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5639516c1ad0 0x5639516c3ec0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639515c0740 msgr2=0x5639516be350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639515c0740 0x5639516be350 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadcc036e60 msgr2=0x7fadcc039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadcc036e60 0x7fadcc039320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faedc054000 tx=0x7faedc04e000 comp rx=0 tx=0).stop 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 msgr2=0x5639516b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639516b7110 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x5639515bbba0 tx=0x7faed8000bc0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 shutdown_connections 2024-04-23T19:12:14.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5639516c1ad0 0x5639516c3ec0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5639515c0740 0x5639516be350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fadcc036e60 0x7fadcc039320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 --2- 172.21.15.63:0/1847684389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639515bcc30 0x5639516b7110 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 >> 172.21.15.63:0/1847684389 conn(0x5639515b6600 msgr2=0x5639515bb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 shutdown_connections 2024-04-23T19:12:14.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.873+0000 7faee97a2800 1 -- 172.21.15.63:0/1847684389 wait complete. 2024-04-23T19:12:14.882 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.882 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:14.882 ERROR:root:0 2024-04-23T19:12:14.882 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-23T19:12:14.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-23T19:12:14.883 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=write 2024-04-23T19:12:14.929 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:14.929 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:14.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b7e010800 1 Processor -- start 2024-04-23T19:12:14.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b7e010800 1 -- start start 2024-04-23T19:12:14.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b7e010800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558d441c04a0 0x558d441c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b7e010800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558d4406f7f0 con 0x558d441bc010 2024-04-23T19:12:14.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b7e010800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x558d4405ee20 con 0x558d441c04a0 2024-04-23T19:12:14.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b78ded640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558d441c04a0 0x558d441c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b78ded640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x558d441c04a0 0x558d441c2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40972/0 (socket says 172.21.15.63:40972) 2024-04-23T19:12:14.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b78ded640 1 -- 172.21.15.63:0/1429911716 learned_addr learned my addr 172.21.15.63:0/1429911716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.929+0000 7f2b795ee640 1 --1- 172.21.15.63:0/1429911716 >> v1:172.21.15.16:6789/0 conn(0x558d441bc010 0x558d441bc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56780/0 (socket says 172.21.15.63:56780) 2024-04-23T19:12:14.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.930+0000 7f2b78ded640 1 -- 172.21.15.63:0/1429911716 >> v1:172.21.15.16:6789/0 conn(0x558d441bc010 legacy=0x558d441bc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.930+0000 7f2b78ded640 1 -- 172.21.15.63:0/1429911716 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4407db50 con 0x558d441c04a0 2024-04-23T19:12:14.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.931+0000 7f2b78ded640 1 --2- 172.21.15.63:0/1429911716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441c04a0 0x558d441c2890 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000a520 tx=0x7f2b7001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee746a318001cd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b6bfff640 1 -- 172.21.15.63:0/1429911716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b70057070 con 0x558d441c04a0 2024-04-23T19:12:14.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b6bfff640 1 -- 172.21.15.63:0/1429911716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b70052570 con 0x558d441c04a0 2024-04-23T19:12:14.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b6bfff640 1 -- 172.21.15.63:0/1429911716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b7004ae60 con 0x558d441c04a0 2024-04-23T19:12:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b7e010800 1 -- 172.21.15.63:0/1429911716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441c04a0 msgr2=0x558d441c2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1429911716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441c04a0 0x558d441c2890 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000a520 tx=0x7f2b7001d010 comp rx=0 tx=0).stop 2024-04-23T19:12:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b7e010800 1 -- 172.21.15.63:0/1429911716 shutdown_connections 2024-04-23T19:12:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1429911716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441c04a0 0x558d441c2890 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b7e010800 1 -- 172.21.15.63:0/1429911716 >> 172.21.15.63:0/1429911716 conn(0x558d441b7630 msgr2=0x558d441b9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b7e010800 1 -- 172.21.15.63:0/1429911716 shutdown_connections 2024-04-23T19:12:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.932+0000 7f2b7e010800 1 -- 172.21.15.63:0/1429911716 wait complete. 2024-04-23T19:12:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b7e010800 1 Processor -- start 2024-04-23T19:12:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b7e010800 1 -- start start 2024-04-23T19:12:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b7e010800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d441300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b7e010800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d441086f0 con 0x558d441bc010 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b795ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d441300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b795ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d441300e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40976/0 (socket says 172.21.15.63:40976) 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b795ee640 1 -- 172.21.15.63:0/2840218130 learned_addr learned my addr 172.21.15.63:0/2840218130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.945+0000 7f2b795ee640 1 -- 172.21.15.63:0/2840218130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d442b10b0 con 0x558d441bc010 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.946+0000 7f2b795ee640 1 --2- 172.21.15.63:0/2840218130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d441300e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c001e40 tx=0x7f2b6c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65e0938b5ea5244 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.946+0000 7f2a67606640 1 -- 172.21.15.63:0/2840218130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b6c00e5c0 con 0x558d441bc010 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.946+0000 7f2a67606640 1 -- 172.21.15.63:0/2840218130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c006070 con 0x558d441bc010 2024-04-23T19:12:14.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.946+0000 7f2a67606640 1 -- 172.21.15.63:0/2840218130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b6c00e910 con 0x558d441bc010 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 -- 172.21.15.63:0/2840218130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 msgr2=0x558d441300e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 --2- 172.21.15.63:0/2840218130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d441300e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c001e40 tx=0x7f2b6c001e70 comp rx=0 tx=0).stop 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 -- 172.21.15.63:0/2840218130 shutdown_connections 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 --2- 172.21.15.63:0/2840218130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d441300e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 -- 172.21.15.63:0/2840218130 >> 172.21.15.63:0/2840218130 conn(0x558d441b7630 msgr2=0x558d442a8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 -- 172.21.15.63:0/2840218130 shutdown_connections 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 -- 172.21.15.63:0/2840218130 wait complete. 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 Processor -- start 2024-04-23T19:12:14.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 -- start start 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d442b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b7e010800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d441c3160 con 0x558d441bc010 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b795ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d442b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b795ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d442b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40984/0 (socket says 172.21.15.63:40984) 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.947+0000 7f2b795ee640 1 -- 172.21.15.63:0/1431881001 learned_addr learned my addr 172.21.15.63:0/1431881001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.948+0000 7f2b795ee640 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d442b64d0 con 0x558d441bc010 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.948+0000 7f2b795ee640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d442b5f90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c004000 tx=0x7f2b6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.948+0000 7f2a65e03640 1 -- 172.21.15.63:0/1431881001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b6c01a7c0 con 0x558d441bc010 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.948+0000 7f2a65e03640 1 -- 172.21.15.63:0/1431881001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c01a960 con 0x558d441bc010 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.948+0000 7f2a65e03640 1 -- 172.21.15.63:0/1431881001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b6c01acc0 con 0x558d441bc010 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.948+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d442b6760 con 0x558d441bc010 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.948+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d442b9410 con 0x558d441bc010 2024-04-23T19:12:14.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.949+0000 7f2a65e03640 1 -- 172.21.15.63:0/1431881001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f2b6c017070 con 0x558d441bc010 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.949+0000 7f2a65e03640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a60036e60 0x7f2a60039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.949+0000 7f2a65e03640 1 -- 172.21.15.63:0/1431881001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f2b6c015020 con 0x558d441bc010 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.949+0000 7f2b78ded640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a60036e60 0x7f2a60039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.949+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d441c04a0 0x558d442bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.949+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442bd760 con 0x558d441c04a0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.949+0000 7f2b79def640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d441c04a0 0x558d442bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.950+0000 7f2b78ded640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a60036e60 0x7f2a60039320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2b70054000 tx=0x7f2b7004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.950+0000 7f2b79def640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d441c04a0 0x558d442bd1d0 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.950+0000 7f2b79def640 1 -- 172.21.15.63:0/1431881001 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b7400a4a0 con 0x558d441c04a0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.950+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442ba7a0 con 0x558d441c04a0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.950+0000 7f2b79def640 1 -- 172.21.15.63:0/1431881001 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2b7400a4a0 con 0x558d441c04a0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.950+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442bf5a0 con 0x558d441c04a0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.951+0000 7f2b79def640 1 -- 172.21.15.63:0/1431881001 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2b7400a4a0 con 0x558d441c04a0 2024-04-23T19:12:14.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.951+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442be8e0 con 0x558d441c04a0 2024-04-23T19:12:14.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.951+0000 7f2b79def640 1 -- 172.21.15.63:0/1431881001 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2b7400a4a0 con 0x558d441c04a0 2024-04-23T19:12:14.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.951+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d442c0950 0x558d442c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.951+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442c32c0 con 0x558d442c0950 2024-04-23T19:12:14.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.951+0000 7f2b795ee640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d442c0950 0x558d442c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.951+0000 7f2b795ee640 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d442c0950 0x558d442c2d40 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b795ee640 1 -- 172.21.15.63:0/1431881001 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2b6c02e070 con 0x558d442c0950 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b7e010800 1 Processor -- start 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b7e010800 1 -- start start 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b7e010800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442d1670 0x558d442d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b7e010800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d440aee30 con 0x558d442d1670 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b79def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442d1670 0x558d442d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b79def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442d1670 0x558d442d1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:40990/0 (socket says 172.21.15.63:40990) 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.952+0000 7f2b79def640 1 -- 172.21.15.63:0/3621544488 learned_addr learned my addr 172.21.15.63:0/3621544488 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.953+0000 7f2b79def640 1 -- 172.21.15.63:0/3621544488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d442d1fd0 con 0x558d442d1670 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.953+0000 7f2b79def640 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442d1670 0x558d442d1a90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x558d442b7390 tx=0x7f2b740178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.953+0000 7f2a5effd640 1 -- 172.21.15.63:0/3621544488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b74013440 con 0x558d442d1670 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.953+0000 7f2a5effd640 1 -- 172.21.15.63:0/3621544488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b740135e0 con 0x558d442d1670 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.953+0000 7f2a5effd640 1 -- 172.21.15.63:0/3621544488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b740138c0 con 0x558d442d1670 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.953+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d442d2a40 con 0x558d442d1670 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.953+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d442d2f40 con 0x558d442d1670 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.954+0000 7f2a5effd640 1 -- 172.21.15.63:0/3621544488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f2b74011040 con 0x558d442d1670 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.954+0000 7f2a5effd640 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a54036e60 0x7f2a54039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.954+0000 7f2a5effd640 1 -- 172.21.15.63:0/3621544488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f2b7401d070 con 0x558d442d1670 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.954+0000 7f2b795ee640 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a54036e60 0x7f2a54039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.954+0000 7f2b7e010800 1 Processor -- start 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.954+0000 7f2b7e010800 1 -- start start 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.954+0000 7f2b795ee640 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a54036e60 0x7f2a54039320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c01be30 tx=0x7f2b6c041040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2b7e010800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2b7e010800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d442b1400 con 0x558d442de990 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2b79def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2b79def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442dedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41002/0 (socket says 172.21.15.63:41002) 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2b79def640 1 -- 172.21.15.63:0/3389346606 learned_addr learned my addr 172.21.15.63:0/3389346606 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2b79def640 1 -- 172.21.15.63:0/3389346606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d442df380 con 0x558d442de990 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2b79def640 1 --2- 172.21.15.63:0/3389346606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442dedb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2b7404ccb0 tx=0x7f2b74012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23832f826575dde server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2a5cff9640 1 -- 172.21.15.63:0/3389346606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b74012df0 con 0x558d442de990 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2a5cff9640 1 -- 172.21.15.63:0/3389346606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74001310 con 0x558d442de990 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.955+0000 7f2a5cff9640 1 -- 172.21.15.63:0/3389346606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b74001630 con 0x558d442de990 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 -- 172.21.15.63:0/3389346606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 msgr2=0x558d442dedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 --2- 172.21.15.63:0/3389346606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442dedb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2b7404ccb0 tx=0x7f2b74012910 comp rx=0 tx=0).stop 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 -- 172.21.15.63:0/3389346606 shutdown_connections 2024-04-23T19:12:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 --2- 172.21.15.63:0/3389346606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442dedb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 -- 172.21.15.63:0/3389346606 >> 172.21.15.63:0/3389346606 conn(0x558d442dacf0 msgr2=0x558d442d8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 -- 172.21.15.63:0/3389346606 shutdown_connections 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 -- 172.21.15.63:0/3389346606 wait complete. 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 Processor -- start 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 -- start start 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b7e010800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d442dfaf0 con 0x558d442de990 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b79def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b79def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41010/0 (socket says 172.21.15.63:41010) 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.956+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 learned_addr learned my addr 172.21.15.63:0/4226323170 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.957+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d442e64a0 con 0x558d442de990 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.957+0000 7f2b79def640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442e5f60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2b7404bb80 tx=0x7f2b7404cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.957+0000 7f2a4affd640 1 -- 172.21.15.63:0/4226323170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b74001c30 con 0x558d442de990 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.957+0000 7f2a4affd640 1 -- 172.21.15.63:0/4226323170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7402ae30 con 0x558d442de990 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.957+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d442e6730 con 0x558d442de990 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.957+0000 7f2a4affd640 1 -- 172.21.15.63:0/4226323170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b74021db0 con 0x558d442de990 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.957+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d442e93e0 con 0x558d442de990 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2a4affd640 1 -- 172.21.15.63:0/4226323170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f2b74040040 con 0x558d442de990 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2a4affd640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a44036eb0 0x7f2a44039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2a4affd640 1 -- 172.21.15.63:0/4226323170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f2b74032070 con 0x558d442de990 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2b795ee640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a44036eb0 0x7f2a44039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d442f1c10 0x558d442f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2b795ee640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a44036eb0 0x7f2a44039370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c01cfd0 tx=0x7f2b6c043000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.958+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442f4820 con 0x558d442f1c10 2024-04-23T19:12:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b78ded640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d442f1c10 0x558d442f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b78ded640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d442f1c10 0x558d442f40d0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 20 realm 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442f4bf0 con 0x558d442f1c10 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.959+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442f54c0 con 0x558d442f1c10 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442f5860 con 0x558d442f1c10 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442f60b0 con 0x558d442f1c10 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 20 searching for the correct realm 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d442f7e20 0x558d442fa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.960+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.961+0000 7f2b79def640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d442f7e20 0x558d442fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.961+0000 7f2b79def640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d442f7e20 0x558d442fa210 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.961+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.961+0000 7f2b79def640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2b740123a0 0x7f2b7402b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.961+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b740123a0 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.961+0000 7f2b795ee640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2b740123a0 0x7f2b7402b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.962+0000 7f2b795ee640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2b740123a0 0x7f2b7402b840 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.962+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.962+0000 7f2b795ee640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f2b6c00f070 0x7f2b6c04a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.962+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.962+0000 7f2b78ded640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f2b6c00f070 0x7f2b6c04a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.962+0000 7f2b78ded640 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f2b6c00f070 0x7f2b6c04a0e0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.963+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.964+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.964+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b740123a0 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.964+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.964+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.964+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.964+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.964+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.965+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.965+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.965+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.965+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.965+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b740123a0 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.965+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.965+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.966+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.967+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b740123a0 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b740123a0 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.968+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b740123a0 2024-04-23T19:12:14.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f7e20 2024-04-23T19:12:14.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.969+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b6c00f070 2024-04-23T19:12:14.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x558d442f1c10 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x558d442fa7a0 con 0x7f2b740123a0 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.970+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442f6a30 con 0x558d442f1c10 2024-04-23T19:12:14.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442f72d0 con 0x558d442f1c10 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442fac20 con 0x7f2b6c00f070 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 zone default found 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 4 Realm: () 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 zonegroup default 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.971+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442fb0d0 con 0x7f2b740123a0 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442fce50 con 0x558d442f7e20 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d442fd300 con 0x558d442f7e20 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 20 started sync module instance, tier type = 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.972+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fc2f0 con 0x7f2b740123a0 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.973+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.973+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94065222731488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fdc10 con 0x7f2b740123a0 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.973+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fe460 con 0x558d442f1c10 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.974+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94065222731488] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.975+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.975+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94065222735712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fe4e0 con 0x558d442f1c10 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.975+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442ff000 con 0x558d442f7e20 2024-04-23T19:12:14.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.976+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.976+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94065222740688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fefc0 con 0x558d442f7e20 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.976+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fffe0 con 0x7f2b740123a0 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.976+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94065222735712] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.977+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94065222740688] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.977+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.977+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94065222743024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fdfb0 con 0x7f2b740123a0 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.977+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44300880 con 0x558d442f1c10 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.978+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94065222743024] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.978+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.978+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94065222744816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44300880 con 0x558d442f1c10 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.978+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d443012f0 con 0x7f2b6c00f070 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.979+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.979+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94065222749664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d443012b0 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.979+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44302280 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.979+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94065222744816] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.980+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94065222749664] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.980+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.980+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94065222753680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44301490 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.980+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44302b20 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94065222753680] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94065222755952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44301490 con 0x7f2b6c00f070 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 20 add_watcher() i=0 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 20 add_watcher() i=1 2024-04-23T19:12:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 20 add_watcher() i=2 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 20 add_watcher() i=3 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 20 add_watcher() i=4 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 20 add_watcher() i=5 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.981+0000 7f2b7e010800 20 add_watcher() i=6 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.982+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94065222755952] v32'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.982+0000 7f2b7e010800 20 add_watcher() i=7 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.982+0000 7f2b7e010800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.983+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d44300490 con 0x558d442f1c10 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.983+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.983+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94065222757008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44300490 con 0x558d442f1c10 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.984+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94065222757008] v32'265 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:14.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.984+0000 7f2a2f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.984+0000 7f2b7e010800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.984+0000 7f2b7e010800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.984+0000 7f2b7e010800 5 note: GC not initialized 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.984+0000 7f2a2e7e4640 20 reqs_thread_entry: start 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.984+0000 7f2b7e010800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 20 Filter name: none 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f299ffff640 20 reqs_thread_entry: start 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 20 rados->read ofs=0 len=0 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d44339050 con 0x7f2b6c00f070 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 20 rados_obj.operate() r=0 bl.length=460 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:14.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.985+0000 7f2b7e010800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.986+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d4433f320 con 0x7f2b6c00f070 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v32'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b7e010800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b7e010800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b7e010800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94065223003536 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558d4433ab70 con 0x558d442f1c10 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94065222744816 notify 137438953504 ret 0) ==== 708+0+0 (crc 0 0 0) 0x558d4407db50 con 0x558d442f1c10 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2a5cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953504 cookie 94065222744816 notifier 4898 bl.length()=666 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2a5cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2a5cff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2a5cff9640 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f2a3c003ea0 con 0x558d442f1c10 2024-04-23T19:12:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94065223003536 out=8b] v0'0 uv270 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.987+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.988+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 94065223003536 notify 137438953504 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f2b7005dd00 con 0x558d442f1c10 2024-04-23T19:12:15.001 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"write"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.988+0000 7f2a2f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.988+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94065222757008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44339500 con 0x558d442f1c10 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.989+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94065222757008] v32'266 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.989+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94065222731488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d44302a90 con 0x7f2b740123a0 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.991+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94065222731488] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.991+0000 7f2b7e010800 20 remove_watcher() i=0 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.991+0000 7f2b7e010800 2 removed watcher, disabling cache 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.991+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94065222735712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fc6e0 con 0x558d442f1c10 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.992+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94065222735712] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.992+0000 7f2b7e010800 20 remove_watcher() i=1 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.992+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94065222740688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fd760 con 0x558d442f7e20 2024-04-23T19:12:15.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.993+0000 7f2b79def640 1 -- 172.21.15.63:0/4226323170 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94065222740688] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b74029070 con 0x558d442f7e20 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.993+0000 7f2b7e010800 20 remove_watcher() i=2 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.993+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94065222743024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fd760 con 0x7f2b740123a0 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.994+0000 7f2b795ee640 1 -- 172.21.15.63:0/4226323170 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94065222743024] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b6c02e070 con 0x7f2b740123a0 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.994+0000 7f2b7e010800 20 remove_watcher() i=3 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.994+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94065222744816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fd760 con 0x558d442f1c10 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.995+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94065222744816] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x558d442f1c10 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.995+0000 7f2b7e010800 20 remove_watcher() i=4 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.995+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94065222749664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442ffaf0 con 0x7f2b6c00f070 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.996+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94065222749664] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.996+0000 7f2b7e010800 20 remove_watcher() i=5 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.996+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94065222753680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fd760 con 0x7f2b6c00f070 2024-04-23T19:12:15.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.997+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94065222753680] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.997+0000 7f2b7e010800 20 remove_watcher() i=6 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.997+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94065222755952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558d442fd760 con 0x7f2b6c00f070 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.998+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94065222755952] v32'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b7005a040 con 0x7f2b6c00f070 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.998+0000 7f2b7e010800 20 remove_watcher() i=7 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f2b6c00f070 msgr2=0x7f2b6c04a0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f2b6c00f070 0x7f2b6c04a0e0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d442f1c10 msgr2=0x558d442f40d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d442f1c10 0x558d442f40d0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2b740123a0 msgr2=0x7f2b7402b840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f2b740123a0 0x7f2b7402b840 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d442f7e20 msgr2=0x558d442fa210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558d442f7e20 0x558d442fa210 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a44036eb0 msgr2=0x7f2a44039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a44036eb0 0x7f2a44039370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c01cfd0 tx=0x7f2b6c043000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 msgr2=0x558d442e5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/4226323170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442de990 0x558d442e5f60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2b7404bb80 tx=0x7f2b7404cc40 comp rx=0 tx=0).stop 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 reap_dead start 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b78ded640 1 -- 172.21.15.63:0/4226323170 reap_dead start 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 shutdown_connections 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 >> 172.21.15.63:0/4226323170 conn(0x558d442dacf0 msgr2=0x558d442d8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 shutdown_connections 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/4226323170 wait complete. 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a54036e60 msgr2=0x7f2a54039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a54036e60 0x7f2a54039320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c01be30 tx=0x7f2b6c041040 comp rx=0 tx=0).stop 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442d1670 msgr2=0x558d442d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442d1670 0x558d442d1a90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x558d442b7390 tx=0x7f2b740178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 shutdown_connections 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a54036e60 0x7f2a54039320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 --2- 172.21.15.63:0/3621544488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d442d1670 0x558d442d1a90 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 >> 172.21.15.63:0/3621544488 conn(0x558d442c5710 msgr2=0x558d442c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 shutdown_connections 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:14.999+0000 7f2b7e010800 1 -- 172.21.15.63:0/3621544488 wait complete. 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d442c0950 msgr2=0x558d442c2d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d442c0950 0x558d442c2d40 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d441c04a0 msgr2=0x558d442bd1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d441c04a0 0x558d442bd1d0 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a60036e60 msgr2=0x7f2a60039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a60036e60 0x7f2a60039320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2b70054000 tx=0x7f2b7004e000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 msgr2=0x558d442b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d442b5f90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c004000 tx=0x7f2b6c000bc0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 shutdown_connections 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x558d442c0950 0x558d442c2d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x558d441c04a0 0x558d442bd1d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2a60036e60 0x7f2a60039320 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 --2- 172.21.15.63:0/1431881001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d441bc010 0x558d442b5f90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 >> 172.21.15.63:0/1431881001 conn(0x558d441b7630 msgr2=0x558d442a73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 shutdown_connections 2024-04-23T19:12:15.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.000+0000 7f2b7e010800 1 -- 172.21.15.63:0/1431881001 wait complete. 2024-04-23T19:12:15.013 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.013 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.014 ERROR:root:0 2024-04-23T19:12:15.014 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.017 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'AccessDenied', 'Message': '', 'RequestId': 'tx00000c86981bdebe718fb-006628080f-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:12:15.018 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-23T19:12:15.018 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-23T19:12:15.018 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps user-info-without-keys=write 2024-04-23T19:12:15.064 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:15.064 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.064+0000 7f749a525800 1 Processor -- start 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.064+0000 7f749a525800 1 -- start start 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f749a525800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5589693da620 0x5589693dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f749a525800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5589692887f0 con 0x5589693d4400 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f749a525800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x558969277e20 con 0x5589693da620 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f7495ab6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5589693da620 0x5589693dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f74962b7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5589693d4400 0x5589693d4820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56782/0 (socket says 172.21.15.63:56782) 2024-04-23T19:12:15.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f74962b7640 1 -- 172.21.15.63:0/2462221634 learned_addr learned my addr 172.21.15.63:0/2462221634 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:15.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f7495ab6640 1 -- 172.21.15.63:0/2462221634 >> v1:172.21.15.16:6789/0 conn(0x5589693d4400 legacy=0x5589693d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.065+0000 7f7495ab6640 1 -- 172.21.15.63:0/2462221634 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558969296b50 con 0x5589693da620 2024-04-23T19:12:15.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.067+0000 7f7495ab6640 1 --2- 172.21.15.63:0/2462221634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693da620 0x5589693dca10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f748c00a470 tx=0x7f748c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba1a59d947cd964 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.067+0000 7f74952b5640 1 -- 172.21.15.63:0/2462221634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f748c057070 con 0x5589693da620 2024-04-23T19:12:15.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.067+0000 7f74952b5640 1 -- 172.21.15.63:0/2462221634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f748c052570 con 0x5589693da620 2024-04-23T19:12:15.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.067+0000 7f74952b5640 1 -- 172.21.15.63:0/2462221634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f748c052870 con 0x5589693da620 2024-04-23T19:12:15.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.068+0000 7f749a525800 1 -- 172.21.15.63:0/2462221634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693da620 msgr2=0x5589693dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.068+0000 7f749a525800 1 --2- 172.21.15.63:0/2462221634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693da620 0x5589693dca10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f748c00a470 tx=0x7f748c01ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:15.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.068+0000 7f749a525800 1 -- 172.21.15.63:0/2462221634 shutdown_connections 2024-04-23T19:12:15.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.068+0000 7f749a525800 1 --2- 172.21.15.63:0/2462221634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693da620 0x5589693dca10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.068+0000 7f749a525800 1 -- 172.21.15.63:0/2462221634 >> 172.21.15.63:0/2462221634 conn(0x5589693cfdb0 msgr2=0x5589693d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.068+0000 7f749a525800 1 -- 172.21.15.63:0/2462221634 shutdown_connections 2024-04-23T19:12:15.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.068+0000 7f749a525800 1 -- 172.21.15.63:0/2462221634 wait complete. 2024-04-23T19:12:15.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.080+0000 7f749a525800 1 Processor -- start 2024-04-23T19:12:15.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.081+0000 7f749a525800 1 -- start start 2024-04-23T19:12:15.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.081+0000 7f749a525800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.081+0000 7f749a525800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589693216f0 con 0x5589693d4400 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.081+0000 7f74962b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.081+0000 7f74962b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41038/0 (socket says 172.21.15.63:41038) 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.081+0000 7f74962b7640 1 -- 172.21.15.63:0/4031019877 learned_addr learned my addr 172.21.15.63:0/4031019877 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.081+0000 7f74962b7640 1 -- 172.21.15.63:0/4031019877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589694c7960 con 0x5589693d4400 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.082+0000 7f74962b7640 1 --2- 172.21.15.63:0/4031019877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694c7390 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7488001d50 tx=0x7f748800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb7af3a21c718b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.082+0000 7f7384608640 1 -- 172.21.15.63:0/4031019877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488013030 con 0x5589693d4400 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.082+0000 7f7384608640 1 -- 172.21.15.63:0/4031019877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f748800e470 con 0x5589693d4400 2024-04-23T19:12:15.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.082+0000 7f7384608640 1 -- 172.21.15.63:0/4031019877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f748800e810 con 0x5589693d4400 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.082+0000 7f749a525800 1 -- 172.21.15.63:0/4031019877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 msgr2=0x5589694c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.082+0000 7f749a525800 1 --2- 172.21.15.63:0/4031019877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694c7390 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7488001d50 tx=0x7f748800b6c0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 -- 172.21.15.63:0/4031019877 shutdown_connections 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 --2- 172.21.15.63:0/4031019877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694c7390 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 -- 172.21.15.63:0/4031019877 >> 172.21.15.63:0/4031019877 conn(0x5589693cfdb0 msgr2=0x5589694c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 -- 172.21.15.63:0/4031019877 shutdown_connections 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 -- 172.21.15.63:0/4031019877 wait complete. 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 Processor -- start 2024-04-23T19:12:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 -- start start 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f749a525800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589693dd2b0 con 0x5589693d4400 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f74962b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f74962b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41040/0 (socket says 172.21.15.63:41040) 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.083+0000 7f74962b7640 1 -- 172.21.15.63:0/3534609814 learned_addr learned my addr 172.21.15.63:0/3534609814 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.084+0000 7f74962b7640 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589694cf4e0 con 0x5589693d4400 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.084+0000 7f74962b7640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694cefa0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x5589694c8120 tx=0x7f7488000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.084+0000 7f7382e05640 1 -- 172.21.15.63:0/3534609814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488027070 con 0x5589693d4400 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.084+0000 7f7382e05640 1 -- 172.21.15.63:0/3534609814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7488019900 con 0x5589693d4400 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.084+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5589694cf770 con 0x5589693d4400 2024-04-23T19:12:15.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.084+0000 7f7382e05640 1 -- 172.21.15.63:0/3534609814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488019c40 con 0x5589693d4400 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.084+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589694d23d0 con 0x5589693d4400 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.085+0000 7f7382e05640 1 -- 172.21.15.63:0/3534609814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f748802a070 con 0x5589693d4400 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.085+0000 7f7382e05640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f737c036e60 0x7f737c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.085+0000 7f7382e05640 1 -- 172.21.15.63:0/3534609814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f748801f070 con 0x5589693d4400 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.085+0000 7f7495ab6640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f737c036e60 0x7f737c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.085+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5589693da620 0x5589694d6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.085+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5589694d6760 con 0x5589693da620 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.085+0000 7f7496ab8640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5589693da620 0x5589694d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.086+0000 7f7495ab6640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f737c036e60 0x7f737c039320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f748c054000 tx=0x7f748c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.086+0000 7f7496ab8640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5589693da620 0x5589694d6190 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.086+0000 7f7496ab8640 1 -- 172.21.15.63:0/3534609814 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f749000a4a0 con 0x5589693da620 2024-04-23T19:12:15.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.086+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5589694d3760 con 0x5589693da620 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.086+0000 7f7496ab8640 1 -- 172.21.15.63:0/3534609814 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f749000a4a0 con 0x5589693da620 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.086+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5589694d8580 con 0x5589693da620 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.087+0000 7f7496ab8640 1 -- 172.21.15.63:0/3534609814 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f749000a4a0 con 0x5589693da620 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.087+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5589694d78c0 con 0x5589693da620 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.087+0000 7f7496ab8640 1 -- 172.21.15.63:0/3534609814 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f749000a4a0 con 0x5589693da620 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.087+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5589694d9930 0x5589694dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.087+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5589694dc2a0 con 0x5589694d9930 2024-04-23T19:12:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.087+0000 7f74962b7640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5589694d9930 0x5589694dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.088+0000 7f74962b7640 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5589694d9930 0x5589694dbd20 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.088+0000 7f74962b7640 1 -- 172.21.15.63:0/3534609814 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7488029030 con 0x5589694d9930 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.088+0000 7f749a525800 1 Processor -- start 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.088+0000 7f749a525800 1 -- start start 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f749a525800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694ea630 0x5589694eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f749a525800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589692c7e30 con 0x5589694ea630 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f7496ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694ea630 0x5589694eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f7496ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694ea630 0x5589694eaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41056/0 (socket says 172.21.15.63:41056) 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f7496ab8640 1 -- 172.21.15.63:0/4048690648 learned_addr learned my addr 172.21.15.63:0/4048690648 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f7496ab8640 1 -- 172.21.15.63:0/4048690648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589694eaf90 con 0x5589694ea630 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f7496ab8640 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694ea630 0x5589694eaa50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x5589694d6b40 tx=0x7f74900178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f737bfff640 1 -- 172.21.15.63:0/4048690648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7490013440 con 0x5589694ea630 2024-04-23T19:12:15.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f737bfff640 1 -- 172.21.15.63:0/4048690648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74900135e0 con 0x5589694ea630 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5589694eba00 con 0x5589694ea630 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.089+0000 7f737bfff640 1 -- 172.21.15.63:0/4048690648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74900138c0 con 0x5589694ea630 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.090+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589694ebf00 con 0x5589694ea630 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.090+0000 7f737bfff640 1 -- 172.21.15.63:0/4048690648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f7490011040 con 0x5589694ea630 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.090+0000 7f737bfff640 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7370036eb0 0x7f7370039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.090+0000 7f737bfff640 1 -- 172.21.15.63:0/4048690648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f749001d070 con 0x5589694ea630 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.090+0000 7f74962b7640 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7370036eb0 0x7f7370039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f749a525800 1 Processor -- start 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f749a525800 1 -- start start 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f74962b7640 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7370036eb0 0x7f7370039370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f748800c520 tx=0x7f748800ef30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f749a525800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f749a525800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589694c7ce0 con 0x5589694f7960 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f7496ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f7496ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694f7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41066/0 (socket says 172.21.15.63:41066) 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f7496ab8640 1 -- 172.21.15.63:0/2047894711 learned_addr learned my addr 172.21.15.63:0/2047894711 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f7496ab8640 1 -- 172.21.15.63:0/2047894711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589694f8350 con 0x5589694f7960 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.091+0000 7f7496ab8640 1 --2- 172.21.15.63:0/2047894711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694f7d80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f749004ccf0 tx=0x7f749002b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2cf99368ba73f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f7379ffb640 1 -- 172.21.15.63:0/2047894711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749002bc90 con 0x5589694f7960 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f7379ffb640 1 -- 172.21.15.63:0/2047894711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749002be30 con 0x5589694f7960 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f7379ffb640 1 -- 172.21.15.63:0/2047894711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749002b4f0 con 0x5589694f7960 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 -- 172.21.15.63:0/2047894711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 msgr2=0x5589694f7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 --2- 172.21.15.63:0/2047894711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694f7d80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f749004ccf0 tx=0x7f749002b8e0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 -- 172.21.15.63:0/2047894711 shutdown_connections 2024-04-23T19:12:15.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 --2- 172.21.15.63:0/2047894711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694f7d80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 -- 172.21.15.63:0/2047894711 >> 172.21.15.63:0/2047894711 conn(0x5589694f3c30 msgr2=0x5589694f1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 -- 172.21.15.63:0/2047894711 shutdown_connections 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 -- 172.21.15.63:0/2047894711 wait complete. 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 Processor -- start 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.092+0000 7f749a525800 1 -- start start 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f749a525800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f749a525800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589694f8ae0 con 0x5589694f7960 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7496ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7496ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694fee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41070/0 (socket says 172.21.15.63:41070) 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 learned_addr learned my addr 172.21.15.63:0/3420038712 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589694ff3b0 con 0x5589694f7960 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7496ab8640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694fee70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f749002aff0 tx=0x7f749004cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7367fff640 1 -- 172.21.15.63:0/3420038712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7490013a60 con 0x5589694f7960 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7367fff640 1 -- 172.21.15.63:0/3420038712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7490013c00 con 0x5589694f7960 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f7367fff640 1 -- 172.21.15.63:0/3420038712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749002a350 con 0x5589694f7960 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5589694ff6a0 con 0x5589694f7960 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.093+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589695022f0 con 0x5589694f7960 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.094+0000 7f7367fff640 1 -- 172.21.15.63:0/3420038712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f749002a4f0 con 0x5589694f7960 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.094+0000 7f7367fff640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7360036eb0 0x7f7360039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.094+0000 7f7367fff640 1 -- 172.21.15.63:0/3420038712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f7490032070 con 0x5589694f7960 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.094+0000 7f74962b7640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7360036eb0 0x7f7360039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.095+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.095+0000 7f74962b7640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7360036eb0 0x7f7360039370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7488017a60 tx=0x7f7488042000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.095+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55896950aa20 0x55896950cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.095+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55896950d630 con 0x55896950aa20 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.095+0000 7f7495ab6640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55896950aa20 0x55896950cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.095+0000 7f7495ab6640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55896950aa20 0x55896950cee0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 realm 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55896950da00 con 0x55896950aa20 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55896950e2d0 con 0x55896950aa20 2024-04-23T19:12:15.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55896950e670 con 0x55896950aa20 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.096+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55896950eec0 con 0x55896950aa20 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.097+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.097+0000 7f749a525800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.097+0000 7f749a525800 20 searching for the correct realm 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.097+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558969510c30 0x558969513020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.097+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.097+0000 7f7496ab8640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558969510c30 0x558969513020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.097+0000 7f7496ab8640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558969510c30 0x558969513020 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f7496ab8640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7490045780 0x7f7490046860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f7490045780 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f74962b7640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7490045780 0x7f7490046860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f74962b7640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7490045780 0x7f7490046860 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f74962b7640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f748802c060 0x7f748804b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.098+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.099+0000 7f7495ab6640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f748802c060 0x7f748804b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.099+0000 7f7495ab6640 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f748802c060 0x7f748804b5d0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.099+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.099+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.099+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.099+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f7490045780 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.100+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.101+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f7490045780 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.102+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.102+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.102+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.102+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.102+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.102+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.103+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.104+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.104+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.104+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.104+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f7490045780 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f7490045780 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.105+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f7490045780 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x558969510c30 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f748802c060 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.106+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x55896950aa20 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5589695135b0 con 0x7f7490045780 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:15.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55896950f810 con 0x55896950aa20 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.107+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5589695100b0 con 0x55896950aa20 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558969513a30 con 0x7f748802c060 2024-04-23T19:12:15.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 zone default found 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 4 Realm: () 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 zonegroup default 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558969513ee0 con 0x7f7490045780 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.108+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558969515c60 con 0x558969510c30 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f749a525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5589695160f0 con 0x558969510c30 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f749a525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f749a525800 20 started sync module instance, tier type = 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f749a525800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.109+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969515100 con 0x7f7490045780 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.110+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.110+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94048665818352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969516a00 con 0x7f7490045780 2024-04-23T19:12:15.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.110+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969517250 con 0x55896950aa20 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.111+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94048665818352] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.111+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.112+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94048665822544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969517310 con 0x55896950aa20 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.112+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969517e30 con 0x558969510c30 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.113+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.113+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94048665827584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969517df0 con 0x558969510c30 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.113+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969518e10 con 0x7f7490045780 2024-04-23T19:12:15.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.113+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94048665822544] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.114+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94048665827584] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.114+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.114+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94048665829920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969516da0 con 0x7f7490045780 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.114+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5589695196f0 con 0x55896950aa20 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.115+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94048665829920] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.115+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.115+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94048665831712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5589695196b0 con 0x55896950aa20 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.115+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951a140 con 0x7f748802c060 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.116+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.116+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94048665836624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951a0b0 con 0x7f748802c060 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.116+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951b110 con 0x7f748802c060 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.116+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94048665831712] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.117+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94048665836624] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.117+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.117+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94048665838896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969519200 con 0x7f748802c060 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.117+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951b580 con 0x7f748802c060 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.118+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94048665838896] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.118+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94048665840640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951b580 con 0x7f748802c060 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=0 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=1 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=2 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=3 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=4 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=5 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=6 2024-04-23T19:12:15.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94048665840640] v32'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 20 add_watcher() i=7 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.119+0000 7f749a525800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.120+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55896951e9a0 con 0x55896950aa20 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.120+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.120+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94048665842384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951be10 con 0x55896950aa20 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.121+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94048665842384] v32'267 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.121+0000 7f734cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.121+0000 7f749a525800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.121+0000 7f749a525800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.121+0000 7f749a525800 5 note: GC not initialized 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.121+0000 7f72c77fe640 20 reqs_thread_entry: start 2024-04-23T19:12:15.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f749a525800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f749a525800 20 Filter name: none 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f749a525800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f72c57fa640 20 reqs_thread_entry: start 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f749a525800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558969551e10 con 0x7f748802c060 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.122+0000 7f749a525800 20 rados_obj.operate() r=0 bl.length=490 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.123+0000 7f749a525800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.123+0000 7f749a525800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.123+0000 7f749a525800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.123+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5589695584b0 con 0x7f748802c060 2024-04-23T19:12:15.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v32'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f749a525800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f749a525800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f749a525800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94048666091296 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558969557000 con 0x55896950aa20 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94048665831712 notify 137438953505 ret 0) ==== 678+0+0 (crc 0 0 0) 0x558969296b50 con 0x55896950aa20 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f7379ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953505 cookie 94048665831712 notifier 4905 bl.length()=636 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f7379ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f7379ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94048666091296 out=8b] v0'0 uv273 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.124+0000 7f7379ffb640 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f735c003ea0 con 0x55896950aa20 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.125+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.125+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 94048666091296 notify 137438953505 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f748c08f540 con 0x55896950aa20 2024-04-23T19:12:15.141 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:15.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.126+0000 7f734cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:15.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.126+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94048665842384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951e470 con 0x55896950aa20 2024-04-23T19:12:15.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.127+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94048665842384] v32'268 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.127+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94048665818352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x55896951b2a0 con 0x7f7490045780 2024-04-23T19:12:15.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.128+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94048665818352] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.128+0000 7f749a525800 20 remove_watcher() i=0 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.128+0000 7f749a525800 2 removed watcher, disabling cache 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.128+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94048665822544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5589695154f0 con 0x55896950aa20 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.129+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94048665822544] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.129+0000 7f749a525800 20 remove_watcher() i=1 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.129+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94048665827584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5589695154f0 con 0x558969510c30 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.130+0000 7f7496ab8640 1 -- 172.21.15.63:0/3420038712 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94048665827584] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f749000a4a0 con 0x558969510c30 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.130+0000 7f749a525800 20 remove_watcher() i=2 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.130+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94048665829920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5589695154f0 con 0x7f7490045780 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.131+0000 7f74962b7640 1 -- 172.21.15.63:0/3420038712 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94048665829920] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748803e070 con 0x7f7490045780 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.131+0000 7f749a525800 20 remove_watcher() i=3 2024-04-23T19:12:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.131+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94048665831712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5589695154f0 con 0x55896950aa20 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.132+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94048665831712] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x55896950aa20 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.132+0000 7f749a525800 20 remove_watcher() i=4 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.132+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94048665836624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969518920 con 0x7f748802c060 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.133+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94048665836624] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.133+0000 7f749a525800 20 remove_watcher() i=5 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.133+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94048665838896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969519c50 con 0x7f748802c060 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.134+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94048665838896] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.134+0000 7f749a525800 20 remove_watcher() i=6 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.134+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94048665840640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x558969518920 con 0x7f748802c060 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94048665840640] v32'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f748c05a040 con 0x7f748802c060 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 20 remove_watcher() i=7 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f748802c060 msgr2=0x7f748804b5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f748802c060 0x7f748804b5d0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55896950aa20 msgr2=0x55896950cee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x55896950aa20 0x55896950cee0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7490045780 msgr2=0x7f7490046860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f7490045780 0x7f7490046860 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558969510c30 msgr2=0x558969513020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x558969510c30 0x558969513020 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7360036eb0 msgr2=0x7f7360039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7360036eb0 0x7f7360039370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7488017a60 tx=0x7f7488042000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 msgr2=0x5589694fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/3420038712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694f7960 0x5589694fee70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f749002aff0 tx=0x7f749004cc80 comp rx=0 tx=0).stop 2024-04-23T19:12:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 reap_dead start 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f7495ab6640 1 -- 172.21.15.63:0/3420038712 reap_dead start 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 shutdown_connections 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 >> 172.21.15.63:0/3420038712 conn(0x5589694f3c30 msgr2=0x5589694f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 shutdown_connections 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/3420038712 wait complete. 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7370036eb0 msgr2=0x7f7370039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7370036eb0 0x7f7370039370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f748800c520 tx=0x7f748800ef30 comp rx=0 tx=0).stop 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694ea630 msgr2=0x5589694eaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694ea630 0x5589694eaa50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x5589694d6b40 tx=0x7f74900178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 shutdown_connections 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f7370036eb0 0x7f7370039370 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 --2- 172.21.15.63:0/4048690648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589694ea630 0x5589694eaa50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.136+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 >> 172.21.15.63:0/4048690648 conn(0x5589694de6d0 msgr2=0x5589694dead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 shutdown_connections 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/4048690648 wait complete. 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5589694d9930 msgr2=0x5589694dbd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5589694d9930 0x5589694dbd20 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5589693da620 msgr2=0x5589694d6190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5589693da620 0x5589694d6190 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f737c036e60 msgr2=0x7f737c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f737c036e60 0x7f737c039320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f748c054000 tx=0x7f748c04e000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 msgr2=0x5589694cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694cefa0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x5589694c8120 tx=0x7f7488000bc0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 shutdown_connections 2024-04-23T19:12:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5589694d9930 0x5589694dbd20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5589693da620 0x5589694d6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f737c036e60 0x7f737c039320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 --2- 172.21.15.63:0/3534609814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589693d4400 0x5589694cefa0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 >> 172.21.15.63:0/3534609814 conn(0x5589693cfdb0 msgr2=0x5589694c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 shutdown_connections 2024-04-23T19:12:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.137+0000 7f749a525800 1 -- 172.21.15.63:0/3534609814 wait complete. 2024-04-23T19:12:15.153 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.153 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.154 ERROR:root:0 2024-04-23T19:12:15.154 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:15.154 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:15.154 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-04-23T19:12:15.200 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:15.200 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.200+0000 7fced060a800 1 Processor -- start 2024-04-23T19:12:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.200+0000 7fced060a800 1 -- start start 2024-04-23T19:12:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.200+0000 7fced060a800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5627df03a870 0x5627df0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.200+0000 7fced060a800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5627def787f0 con 0x5627df03bc00 2024-04-23T19:12:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.200+0000 7fced060a800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5627def67e20 con 0x5627df03a870 2024-04-23T19:12:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.201+0000 7fcecd908640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5627df03a870 0x5627df0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.201+0000 7fcecd908640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5627df03a870 0x5627df0cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41082/0 (socket says 172.21.15.63:41082) 2024-04-23T19:12:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.201+0000 7fcecd908640 1 -- 172.21.15.63:0/3944851887 learned_addr learned my addr 172.21.15.63:0/3944851887 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.201+0000 7fcecd908640 1 -- 172.21.15.63:0/3944851887 >> v1:172.21.15.16:6789/0 conn(0x5627df03bc00 legacy=0x5627df03a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.201+0000 7fcecd908640 1 -- 172.21.15.63:0/3944851887 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627def86b50 con 0x5627df03a870 2024-04-23T19:12:15.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.203+0000 7fcecd908640 1 --2- 172.21.15.63:0/3944851887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df0cf020 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcec400a520 tx=0x7fcec401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629f28ae73d479b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.203+0000 7fcecd107640 1 -- 172.21.15.63:0/3944851887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec4057070 con 0x5627df03a870 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.203+0000 7fcecd107640 1 -- 172.21.15.63:0/3944851887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec4052570 con 0x5627df03a870 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.203+0000 7fcecd107640 1 -- 172.21.15.63:0/3944851887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec4052870 con 0x5627df03a870 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.204+0000 7fced060a800 1 -- 172.21.15.63:0/3944851887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 msgr2=0x5627df0cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.204+0000 7fced060a800 1 --2- 172.21.15.63:0/3944851887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df0cf020 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcec400a520 tx=0x7fcec401d010 comp rx=0 tx=0).stop 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.204+0000 7fced060a800 1 -- 172.21.15.63:0/3944851887 shutdown_connections 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.204+0000 7fced060a800 1 --2- 172.21.15.63:0/3944851887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df0cf020 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.204+0000 7fced060a800 1 -- 172.21.15.63:0/3944851887 >> 172.21.15.63:0/3944851887 conn(0x5627df0c17e0 msgr2=0x5627df0c3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.204+0000 7fced060a800 1 -- 172.21.15.63:0/3944851887 shutdown_connections 2024-04-23T19:12:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.204+0000 7fced060a800 1 -- 172.21.15.63:0/3944851887 wait complete. 2024-04-23T19:12:15.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.217+0000 7fced060a800 1 Processor -- start 2024-04-23T19:12:15.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.217+0000 7fced060a800 1 -- start start 2024-04-23T19:12:15.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.217+0000 7fced060a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.217+0000 7fced060a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627df0116f0 con 0x5627df03a870 2024-04-23T19:12:15.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.217+0000 7fcece109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.217+0000 7fcece109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1b98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41084/0 (socket says 172.21.15.63:41084) 2024-04-23T19:12:15.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.217+0000 7fcece109640 1 -- 172.21.15.63:0/3911649729 learned_addr learned my addr 172.21.15.63:0/3911649729 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.218+0000 7fcece109640 1 -- 172.21.15.63:0/3911649729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627df1b9e90 con 0x5627df03a870 2024-04-23T19:12:15.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.218+0000 7fcece109640 1 --2- 172.21.15.63:0/3911649729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1b98c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcec0001d50 tx=0x7fcec000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b81a3f8aa62e361 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.218+0000 7fcdbc608640 1 -- 172.21.15.63:0/3911649729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec000f400 con 0x5627df03a870 2024-04-23T19:12:15.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.218+0000 7fcdbc608640 1 -- 172.21.15.63:0/3911649729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec000f5c0 con 0x5627df03a870 2024-04-23T19:12:15.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.218+0000 7fcdbc608640 1 -- 172.21.15.63:0/3911649729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec000f7f0 con 0x5627df03a870 2024-04-23T19:12:15.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 -- 172.21.15.63:0/3911649729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 msgr2=0x5627df1b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 --2- 172.21.15.63:0/3911649729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1b98c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcec0001d50 tx=0x7fcec000b5b0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 -- 172.21.15.63:0/3911649729 shutdown_connections 2024-04-23T19:12:15.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 --2- 172.21.15.63:0/3911649729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1b98c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 -- 172.21.15.63:0/3911649729 >> 172.21.15.63:0/3911649729 conn(0x5627df0c17e0 msgr2=0x5627df1b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 -- 172.21.15.63:0/3911649729 shutdown_connections 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 -- 172.21.15.63:0/3911649729 wait complete. 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 Processor -- start 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.219+0000 7fced060a800 1 -- start start 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fced060a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1c1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fced060a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627df0d1840 con 0x5627df03a870 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcece109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcece109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41090/0 (socket says 172.21.15.63:41090) 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcece109640 1 -- 172.21.15.63:0/1514994431 learned_addr learned my addr 172.21.15.63:0/1514994431 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcece109640 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627df1c1970 con 0x5627df03a870 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcece109640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1c1430 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5627df1ba650 tx=0x7fcec0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcdbae05640 1 -- 172.21.15.63:0/1514994431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec001a690 con 0x5627df03a870 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcdbae05640 1 -- 172.21.15.63:0/1514994431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec001a830 con 0x5627df03a870 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627df1c1c00 con 0x5627df03a870 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fcdbae05640 1 -- 172.21.15.63:0/1514994431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec001ab30 con 0x5627df03a870 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.220+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627df1c48b0 con 0x5627df03a870 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.221+0000 7fcdbae05640 1 -- 172.21.15.63:0/1514994431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcec0017070 con 0x5627df03a870 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.221+0000 7fcdbae05640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcdb4036e60 0x7fcdb4039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.221+0000 7fcdbae05640 1 -- 172.21.15.63:0/1514994431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fcec0015020 con 0x5627df03a870 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.221+0000 7fcecd908640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcdb4036e60 0x7fcdb4039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.222+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df03bc00 0x5627df1c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.222+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df1c8bf0 con 0x5627df03bc00 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.222+0000 7fcece90a640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df03bc00 0x5627df1c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.222+0000 7fcecd908640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcdb4036e60 0x7fcdb4039320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcec4054000 tx=0x7fcec404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.222+0000 7fcece90a640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df03bc00 0x5627df1c8670 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.222+0000 7fcece90a640 1 -- 172.21.15.63:0/1514994431 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcec800a4a0 con 0x5627df03bc00 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.222+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df1c5c40 con 0x5627df03bc00 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fcece90a640 1 -- 172.21.15.63:0/1514994431 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcec800a4a0 con 0x5627df03bc00 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df1ca9d0 con 0x5627df03bc00 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fcece90a640 1 -- 172.21.15.63:0/1514994431 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fcec800a4a0 con 0x5627df03bc00 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df1c9d10 con 0x5627df03bc00 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fcece90a640 1 -- 172.21.15.63:0/1514994431 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcec800a4a0 con 0x5627df03bc00 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5627df1cbd80 0x5627df1ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df1ce6f0 con 0x5627df1cbd80 2024-04-23T19:12:15.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.223+0000 7fcece109640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5627df1cbd80 0x5627df1ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.224+0000 7fcece109640 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5627df1cbd80 0x5627df1ce170 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.224+0000 7fcece109640 1 -- 172.21.15.63:0/1514994431 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fcec002e070 con 0x5627df1cbd80 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.224+0000 7fced060a800 1 Processor -- start 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.224+0000 7fced060a800 1 -- start start 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fced060a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1dcb10 0x5627df1dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fced060a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627defb7e30 con 0x5627df1dcb10 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcece90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1dcb10 0x5627df1dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcece90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1dcb10 0x5627df1dcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41098/0 (socket says 172.21.15.63:41098) 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcece90a640 1 -- 172.21.15.63:0/1940865846 learned_addr learned my addr 172.21.15.63:0/1940865846 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcece90a640 1 -- 172.21.15.63:0/1940865846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627df1dd470 con 0x5627df1dcb10 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcece90a640 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1dcb10 0x5627df1dcf30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x5627df1c8f90 tx=0x7fcec80178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcdb3fff640 1 -- 172.21.15.63:0/1940865846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec8013440 con 0x5627df1dcb10 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcdb3fff640 1 -- 172.21.15.63:0/1940865846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec80135e0 con 0x5627df1dcb10 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627df1ddee0 con 0x5627df1dcb10 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.225+0000 7fcdb3fff640 1 -- 172.21.15.63:0/1940865846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec80138c0 con 0x5627df1dcb10 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.226+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627df1de3e0 con 0x5627df1dcb10 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.226+0000 7fcdb3fff640 1 -- 172.21.15.63:0/1940865846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcec8011040 con 0x5627df1dcb10 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.226+0000 7fcdb3fff640 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcda8036e60 0x7fcda8039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.226+0000 7fcdb3fff640 1 -- 172.21.15.63:0/1940865846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fcec801d070 con 0x5627df1dcb10 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.226+0000 7fcece109640 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcda8036e60 0x7fcda8039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fced060a800 1 Processor -- start 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fced060a800 1 -- start start 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fcece109640 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcda8036e60 0x7fcda8039320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcec00016b0 tx=0x7fcec0041040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fced060a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fced060a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627df1ba210 con 0x5627df1e9e10 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fcece90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fcece90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41114/0 (socket says 172.21.15.63:41114) 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fcece90a640 1 -- 172.21.15.63:0/2960151106 learned_addr learned my addr 172.21.15.63:0/2960151106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.227+0000 7fcece90a640 1 -- 172.21.15.63:0/2960151106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627df1ea800 con 0x5627df1e9e10 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fcece90a640 1 --2- 172.21.15.63:0/2960151106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1ea230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fcec80013e0 tx=0x7fcec8012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46e7579ee0ecda8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fcdb1ffb640 1 -- 172.21.15.63:0/2960151106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec8043070 con 0x5627df1e9e10 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fcdb1ffb640 1 -- 172.21.15.63:0/2960151106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec802ac50 con 0x5627df1e9e10 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fcdb1ffb640 1 -- 172.21.15.63:0/2960151106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec80014f0 con 0x5627df1e9e10 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fced060a800 1 -- 172.21.15.63:0/2960151106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 msgr2=0x5627df1ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fced060a800 1 --2- 172.21.15.63:0/2960151106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1ea230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fcec80013e0 tx=0x7fcec8012c90 comp rx=0 tx=0).stop 2024-04-23T19:12:15.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fced060a800 1 -- 172.21.15.63:0/2960151106 shutdown_connections 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fced060a800 1 --2- 172.21.15.63:0/2960151106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1ea230 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fced060a800 1 -- 172.21.15.63:0/2960151106 >> 172.21.15.63:0/2960151106 conn(0x5627df1e6170 msgr2=0x5627df1e41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fced060a800 1 -- 172.21.15.63:0/2960151106 shutdown_connections 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.228+0000 7fced060a800 1 -- 172.21.15.63:0/2960151106 wait complete. 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fced060a800 1 Processor -- start 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fced060a800 1 -- start start 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fced060a800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fced060a800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627df1eaf70 con 0x5627df1e9e10 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fcece90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fcece90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1f13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41128/0 (socket says 172.21.15.63:41128) 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 learned_addr learned my addr 172.21.15.63:0/4044689702 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.229+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627df1f1920 con 0x5627df1e9e10 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.230+0000 7fcece90a640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1f13e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcec804cb80 tx=0x7fcec804cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.230+0000 7fcd9ffff640 1 -- 172.21.15.63:0/4044689702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec8043030 con 0x5627df1e9e10 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.230+0000 7fcd9ffff640 1 -- 172.21.15.63:0/4044689702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec802ae00 con 0x5627df1e9e10 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.230+0000 7fcd9ffff640 1 -- 172.21.15.63:0/4044689702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec80126e0 con 0x5627df1e9e10 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.230+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627df1f1bb0 con 0x5627df1e9e10 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.230+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627df1f4860 con 0x5627df1e9e10 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.230+0000 7fcd9ffff640 1 -- 172.21.15.63:0/4044689702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcec804e110 con 0x5627df1e9e10 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.231+0000 7fcd9ffff640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd98036eb0 0x7fcd98039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.231+0000 7fcd9ffff640 1 -- 172.21.15.63:0/4044689702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fcec8044070 con 0x5627df1e9e10 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.231+0000 7fcece109640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd98036eb0 0x7fcd98039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.231+0000 7fcece109640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd98036eb0 0x7fcd98039370 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5627df1cead0 tx=0x7fcec0032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.231+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.231+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df1fd0b0 0x5627df1ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.231+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df1ffcc0 con 0x5627df1fd0b0 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fcecd908640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df1fd0b0 0x5627df1ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fcecd908640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df1fd0b0 0x5627df1ff570 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 20 realm 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df200090 con 0x5627df1fd0b0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.232+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df200960 con 0x5627df1fd0b0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df200d00 con 0x5627df1fd0b0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df201550 con 0x5627df1fd0b0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 20 searching for the correct realm 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5627df2032c0 0x5627df2056b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.233+0000 7fcece90a640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5627df2032c0 0x5627df2056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.234+0000 7fcece90a640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5627df2032c0 0x5627df2056b0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.234+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.234+0000 7fcece90a640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcec8022420 0x7fcec803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.234+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec8022420 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.234+0000 7fcece109640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcec8022420 0x7fcec803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.234+0000 7fcece109640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcec8022420 0x7fcec803b660 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.235+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.235+0000 7fcece109640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcec004d770 0x7fcec001b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.235+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.235+0000 7fcecd908640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcec004d770 0x7fcec001b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.236+0000 7fcecd908640 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcec004d770 0x7fcec001b3d0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.236+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.236+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.236+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.236+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.236+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.236+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec8022420 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.237+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec8022420 2024-04-23T19:12:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.238+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.239+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.240+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.240+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.240+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.240+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.240+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.240+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.241+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.241+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.241+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.241+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec8022420 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.241+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.241+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec8022420 2024-04-23T19:12:15.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.242+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec8022420 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.243+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.243+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df2032c0 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.243+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.243+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec004d770 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.243+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.243+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x5627df1fd0b0 2024-04-23T19:12:15.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.243+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5627df205c40 con 0x7fcec8022420 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:15.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:15.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df201ed0 con 0x5627df1fd0b0 2024-04-23T19:12:15.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df202770 con 0x5627df1fd0b0 2024-04-23T19:12:15.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:15.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.244+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df2060c0 con 0x7fcec004d770 2024-04-23T19:12:15.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:15.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 zone default found 2024-04-23T19:12:15.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 4 Realm: () 2024-04-23T19:12:15.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:15.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:15.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 zonegroup default 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df206570 con 0x7fcec8022420 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df2082f0 con 0x5627df2032c0 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.245+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df208800 con 0x5627df2032c0 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.246+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.246+0000 7fced060a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.246+0000 7fced060a800 20 started sync module instance, tier type = 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.246+0000 7fced060a800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.246+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df207790 con 0x7fcec8022420 2024-04-23T19:12:15.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.247+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.247+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94729247161216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df2073f0 con 0x7fcec8022420 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.247+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df209730 con 0x5627df1fd0b0 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.248+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94729247161216] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.248+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.248+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94729247167104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df209b50 con 0x5627df1fd0b0 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.248+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20a510 con 0x5627df2032c0 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.249+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94729247167104] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.249+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.249+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94729247168160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df209ac0 con 0x5627df2032c0 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.249+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20b3d0 con 0x7fcec8022420 2024-04-23T19:12:15.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.250+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94729247168160] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.251+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.251+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94729247172864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20b3d0 con 0x7fcec8022420 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.251+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20bdd0 con 0x5627df1fd0b0 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.252+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.252+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94729247176928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20c2b0 con 0x5627df1fd0b0 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.252+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20cdd0 con 0x7fcec004d770 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.252+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94729247172864] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.253+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.253+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94729247180960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20cd90 con 0x7fcec004d770 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.253+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20d7e0 con 0x7fcec004d770 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.253+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94729247176928] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.254+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94729247180960] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.254+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.254+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94729247174432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20d330 con 0x7fcec004d770 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.254+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20dcf0 con 0x7fcec004d770 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94729247184960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20e110 con 0x7fcec004d770 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 20 add_watcher() i=0 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 20 add_watcher() i=1 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 20 add_watcher() i=2 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 20 add_watcher() i=3 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 20 add_watcher() i=4 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 20 add_watcher() i=5 2024-04-23T19:12:15.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94729247174432] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.255+0000 7fced060a800 20 add_watcher() i=6 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.256+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94729247184960] v32'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.256+0000 7fced060a800 20 add_watcher() i=7 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.256+0000 7fced060a800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.256+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df20e110 con 0x5627df1fd0b0 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.257+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.257+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94729247186016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20e080 con 0x5627df1fd0b0 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94729247186016] v32'269 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fcd84fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 5 note: GC not initialized 2024-04-23T19:12:15.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fccff7fe640 20 reqs_thread_entry: start 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 20 Filter name: none 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fccfd7fa640 20 reqs_thread_entry: start 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.258+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df244550 con 0x7fcec004d770 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.259+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.259+0000 7fced060a800 20 rados_obj.operate() r=0 bl.length=460 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.259+0000 7fced060a800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.259+0000 7fced060a800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.259+0000 7fced060a800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:15.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.259+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df24a800 con 0x7fcec004d770 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v32'12 uv12 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fced060a800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fced060a800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fced060a800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94729247433328 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5627df246010 con 0x5627df1fd0b0 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94729247176928 notify 137438953506 ret 0) ==== 691+0+0 (crc 0 0 0) 0x5627def86b50 con 0x5627df1fd0b0 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcdb1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953506 cookie 94729247176928 notifier 4912 bl.length()=649 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcdb1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcdb1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94729247433328 out=8b] v0'0 uv276 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcdb1ffb640 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7fcd94003ea0 con 0x5627df1fd0b0 2024-04-23T19:12:15.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.261+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.262+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 94729247433328 notify 137438953506 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fcec408f520 con 0x5627df1fd0b0 2024-04-23T19:12:15.283 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.263+0000 7fcd84fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.263+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94729247186016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df249b10 con 0x5627df1fd0b0 2024-04-23T19:12:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.264+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94729247186016] v32'270 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.264+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94729247161216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df249b10 con 0x7fcec8022420 2024-04-23T19:12:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.265+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94729247161216] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.265+0000 7fced060a800 20 remove_watcher() i=0 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.265+0000 7fced060a800 2 removed watcher, disabling cache 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.265+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94729247167104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df249b10 con 0x5627df1fd0b0 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.266+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94729247167104] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.266+0000 7fced060a800 20 remove_watcher() i=1 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.266+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94729247168160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df249b10 con 0x5627df2032c0 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.267+0000 7fcece90a640 1 -- 172.21.15.63:0/4044689702 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94729247168160] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec8029070 con 0x5627df2032c0 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.267+0000 7fced060a800 20 remove_watcher() i=2 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.267+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94729247172864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df207b80 con 0x7fcec8022420 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.268+0000 7fcece109640 1 -- 172.21.15.63:0/4044689702 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94729247172864] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec002e070 con 0x7fcec8022420 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.268+0000 7fced060a800 20 remove_watcher() i=3 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.268+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94729247176928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20a900 con 0x5627df1fd0b0 2024-04-23T19:12:15.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.269+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94729247176928] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x5627df1fd0b0 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.269+0000 7fced060a800 20 remove_watcher() i=4 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.269+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94729247180960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df207b80 con 0x7fcec004d770 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.270+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94729247180960] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.270+0000 7fced060a800 20 remove_watcher() i=5 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.270+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94729247174432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df20c8a0 con 0x7fcec004d770 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.271+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94729247174432] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.271+0000 7fced060a800 20 remove_watcher() i=6 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.271+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94729247184960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5627df207b80 con 0x7fcec004d770 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94729247184960] v32'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcec405a040 con 0x7fcec004d770 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 20 remove_watcher() i=7 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcec004d770 msgr2=0x7fcec001b3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fcec004d770 0x7fcec001b3d0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df1fd0b0 msgr2=0x5627df1ff570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df1fd0b0 0x5627df1ff570 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcec8022420 msgr2=0x7fcec803b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fcec8022420 0x7fcec803b660 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5627df2032c0 msgr2=0x5627df2056b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5627df2032c0 0x5627df2056b0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd98036eb0 msgr2=0x7fcd98039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd98036eb0 0x7fcd98039370 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5627df1cead0 tx=0x7fcec0032000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.272+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 msgr2=0x5627df1f13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fcecd908640 1 -- 172.21.15.63:0/4044689702 reap_dead start 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1f13e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcec804cb80 tx=0x7fcec804cbb0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 shutdown_connections 2024-04-23T19:12:15.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/4044689702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1e9e10 0x5627df1f13e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 >> 172.21.15.63:0/4044689702 conn(0x5627df1e6170 msgr2=0x5627df1e37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 shutdown_connections 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/4044689702 wait complete. 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcda8036e60 msgr2=0x7fcda8039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcda8036e60 0x7fcda8039320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcec00016b0 tx=0x7fcec0041040 comp rx=0 tx=0).stop 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1dcb10 msgr2=0x5627df1dcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1dcb10 0x5627df1dcf30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x5627df1c8f90 tx=0x7fcec80178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 shutdown_connections 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcda8036e60 0x7fcda8039320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1940865846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df1dcb10 0x5627df1dcf30 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 >> 172.21.15.63:0/1940865846 conn(0x5627df1d0bb0 msgr2=0x5627df1d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 shutdown_connections 2024-04-23T19:12:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1940865846 wait complete. 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5627df1cbd80 msgr2=0x5627df1ce170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5627df1cbd80 0x5627df1ce170 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df03bc00 msgr2=0x5627df1c8670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df03bc00 0x5627df1c8670 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcdb4036e60 msgr2=0x7fcdb4039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcdb4036e60 0x7fcdb4039320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcec4054000 tx=0x7fcec404e000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 msgr2=0x5627df1c1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.273+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1c1430 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5627df1ba650 tx=0x7fcec0000ed0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 shutdown_connections 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5627df1cbd80 0x5627df1ce170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5627df03bc00 0x5627df1c8670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcdb4036e60 0x7fcdb4039320 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 --2- 172.21.15.63:0/1514994431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627df03a870 0x5627df1c1430 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 >> 172.21.15.63:0/1514994431 conn(0x5627df0c17e0 msgr2=0x5627df1b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 shutdown_connections 2024-04-23T19:12:15.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.274+0000 7fced060a800 1 -- 172.21.15.63:0/1514994431 wait complete. 2024-04-23T19:12:15.295 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.295 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.295 ERROR:root:0 2024-04-23T19:12:15.295 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:15.296 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-23T19:12:15.296 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-04-23T19:12:15.344 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:12:15.344 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:12:15.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.342+0000 7f3538c13800 1 Processor -- start 2024-04-23T19:12:15.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.342+0000 7f3538c13800 1 -- start start 2024-04-23T19:12:15.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.342+0000 7f3538c13800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563c033c5a20 0x563c033c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.342+0000 7f3538c13800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563c032797f0 con 0x563c033c6380 2024-04-23T19:12:15.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.342+0000 7f3538c13800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x563c03268e20 con 0x563c033c5a20 2024-04-23T19:12:15.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.343+0000 7f353426e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563c033c5a20 0x563c033c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.343+0000 7f3533a6d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x563c033c6380 0x563c033c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:56808/0 (socket says 172.21.15.63:56808) 2024-04-23T19:12:15.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.343+0000 7f353426e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x563c033c5a20 0x563c033c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41144/0 (socket says 172.21.15.63:41144) 2024-04-23T19:12:15.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.343+0000 7f3533a6d640 1 -- 172.21.15.63:0/1774894613 learned_addr learned my addr 172.21.15.63:0/1774894613 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:12:15.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.343+0000 7f353426e640 1 -- 172.21.15.63:0/1774894613 >> v1:172.21.15.16:6789/0 conn(0x563c033c6380 legacy=0x563c033c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.343+0000 7f353426e640 1 -- 172.21.15.63:0/1774894613 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c03287b50 con 0x563c033c5a20 2024-04-23T19:12:15.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.345+0000 7f353426e640 1 --2- 172.21.15.63:0/1774894613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c033c5e40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f352400a470 tx=0x7f352401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d73fe1ebe936cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.345+0000 7f353326c640 1 -- 172.21.15.63:0/1774894613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524057070 con 0x563c033c5a20 2024-04-23T19:12:15.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.345+0000 7f353326c640 1 -- 172.21.15.63:0/1774894613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352404adc0 con 0x563c033c5a20 2024-04-23T19:12:15.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.345+0000 7f353326c640 1 -- 172.21.15.63:0/1774894613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524052590 con 0x563c033c5a20 2024-04-23T19:12:15.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.346+0000 7f3538c13800 1 -- 172.21.15.63:0/1774894613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 msgr2=0x563c033c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.346+0000 7f3538c13800 1 --2- 172.21.15.63:0/1774894613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c033c5e40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f352400a470 tx=0x7f352401ce90 comp rx=0 tx=0).stop 2024-04-23T19:12:15.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.346+0000 7f3538c13800 1 -- 172.21.15.63:0/1774894613 shutdown_connections 2024-04-23T19:12:15.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.346+0000 7f3538c13800 1 --2- 172.21.15.63:0/1774894613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c033c5e40 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.346+0000 7f3538c13800 1 -- 172.21.15.63:0/1774894613 >> 172.21.15.63:0/1774894613 conn(0x563c033c1600 msgr2=0x563c033c3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.346+0000 7f3538c13800 1 -- 172.21.15.63:0/1774894613 shutdown_connections 2024-04-23T19:12:15.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.346+0000 7f3538c13800 1 -- 172.21.15.63:0/1774894613 wait complete. 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.358+0000 7f3538c13800 1 Processor -- start 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.358+0000 7f3538c13800 1 -- start start 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.359+0000 7f3538c13800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.359+0000 7f3538c13800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c033126f0 con 0x563c033c5a20 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.359+0000 7f353426e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034ba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.359+0000 7f353426e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034ba6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41154/0 (socket says 172.21.15.63:41154) 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.359+0000 7f353426e640 1 -- 172.21.15.63:0/348941385 learned_addr learned my addr 172.21.15.63:0/348941385 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.359+0000 7f353426e640 1 -- 172.21.15.63:0/348941385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c034bac90 con 0x563c033c5a20 2024-04-23T19:12:15.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.359+0000 7f353426e640 1 --2- 172.21.15.63:0/348941385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034ba6c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3524055040 tx=0x7f352404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdb0fd96670a8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f342260c640 1 -- 172.21.15.63:0/348941385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524057040 con 0x563c033c5a20 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f342260c640 1 -- 172.21.15.63:0/348941385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3524063070 con 0x563c033c5a20 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f342260c640 1 -- 172.21.15.63:0/348941385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352405d790 con 0x563c033c5a20 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f3538c13800 1 -- 172.21.15.63:0/348941385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 msgr2=0x563c034ba6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f3538c13800 1 --2- 172.21.15.63:0/348941385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034ba6c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3524055040 tx=0x7f352404c000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f3538c13800 1 -- 172.21.15.63:0/348941385 shutdown_connections 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f3538c13800 1 --2- 172.21.15.63:0/348941385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034ba6c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f3538c13800 1 -- 172.21.15.63:0/348941385 >> 172.21.15.63:0/348941385 conn(0x563c033c1600 msgr2=0x563c034b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f3538c13800 1 -- 172.21.15.63:0/348941385 shutdown_connections 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.360+0000 7f3538c13800 1 -- 172.21.15.63:0/348941385 wait complete. 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f3538c13800 1 Processor -- start 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f3538c13800 1 -- start start 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f3538c13800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f3538c13800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c033c71e0 con 0x563c033c5a20 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f353426e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f353426e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41170/0 (socket says 172.21.15.63:41170) 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f353426e640 1 -- 172.21.15.63:0/2279836561 learned_addr learned my addr 172.21.15.63:0/2279836561 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.361+0000 7f353426e640 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c034c2680 con 0x563c033c5a20 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.362+0000 7f353426e640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034c2140 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3524060040 tx=0x7f3524002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.362+0000 7f3420e09640 1 -- 172.21.15.63:0/2279836561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524057050 con 0x563c033c5a20 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.362+0000 7f3420e09640 1 -- 172.21.15.63:0/2279836561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352405de00 con 0x563c033c5a20 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.362+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c034c2910 con 0x563c033c5a20 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.362+0000 7f3420e09640 1 -- 172.21.15.63:0/2279836561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524052590 con 0x563c033c5a20 2024-04-23T19:12:15.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.362+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c034c5570 con 0x563c033c5a20 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3420e09640 1 -- 172.21.15.63:0/2279836561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f3524052730 con 0x563c033c5a20 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3420e09640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f341c036e60 0x7f341c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3420e09640 1 -- 172.21.15.63:0/2279836561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f352408e450 con 0x563c033c5a20 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3533a6d640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f341c036e60 0x7f341c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c033c6380 0x563c034c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c034c9910 con 0x563c033c6380 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3534a6f640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c033c6380 0x563c034c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.363+0000 7f3533a6d640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f341c036e60 0x7f341c039320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3528000f60 tx=0x7f35280012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.364+0000 7f3534a6f640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c033c6380 0x563c034c9390 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.364+0000 7f3534a6f640 1 -- 172.21.15.63:0/2279836561 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f352c00a4a0 con 0x563c033c6380 2024-04-23T19:12:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.364+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c034c6960 con 0x563c033c6380 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.364+0000 7f3534a6f640 1 -- 172.21.15.63:0/2279836561 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f352c00a4a0 con 0x563c033c6380 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.364+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c034cb710 con 0x563c033c6380 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.364+0000 7f3534a6f640 1 -- 172.21.15.63:0/2279836561 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f352c00a4a0 con 0x563c033c6380 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.364+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c034caa50 con 0x563c033c6380 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.365+0000 7f3534a6f640 1 -- 172.21.15.63:0/2279836561 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f352c00a4a0 con 0x563c033c6380 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.365+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563c034ccac0 0x563c034ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.365+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c034cf430 con 0x563c034ccac0 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.365+0000 7f353426e640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563c034ccac0 0x563c034ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.365+0000 7f353426e640 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563c034ccac0 0x563c034ceeb0 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.365+0000 7f353426e640 1 -- 172.21.15.63:0/2279836561 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3524078070 con 0x563c034ccac0 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3538c13800 1 Processor -- start 2024-04-23T19:12:15.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3538c13800 1 -- start start 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3538c13800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034dd850 0x563c034ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3538c13800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c032b8e30 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3534a6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034dd850 0x563c034ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3534a6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034dd850 0x563c034ddc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41172/0 (socket says 172.21.15.63:41172) 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3534a6f640 1 -- 172.21.15.63:0/1439604968 learned_addr learned my addr 172.21.15.63:0/1439604968 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.366+0000 7f3534a6f640 1 -- 172.21.15.63:0/1439604968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c034de1b0 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.367+0000 7f3534a6f640 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034dd850 0x563c034ddc70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x563c034c34f0 tx=0x7f352c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.367+0000 7f3419ffb640 1 -- 172.21.15.63:0/1439604968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352c013440 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.367+0000 7f3419ffb640 1 -- 172.21.15.63:0/1439604968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c0135e0 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.367+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c034dec20 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.367+0000 7f3419ffb640 1 -- 172.21.15.63:0/1439604968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352c0138c0 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.367+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c034df120 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.367+0000 7f3419ffb640 1 -- 172.21.15.63:0/1439604968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f352c011040 con 0x563c034dd850 2024-04-23T19:12:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3419ffb640 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f340c036e60 0x7f340c039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3419ffb640 1 -- 172.21.15.63:0/1439604968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f352c01d070 con 0x563c034dd850 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f353426e640 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f340c036e60 0x7f340c039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3538c13800 1 Processor -- start 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3538c13800 1 -- start start 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f353426e640 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f340c036e60 0x7f340c039320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563c034cf810 tx=0x7f352408c760 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3538c13800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3538c13800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c034baf50 con 0x563c034eabb0 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3534a6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3534a6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034eafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41174/0 (socket says 172.21.15.63:41174) 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.368+0000 7f3534a6f640 1 -- 172.21.15.63:0/1504655463 learned_addr learned my addr 172.21.15.63:0/1504655463 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3534a6f640 1 -- 172.21.15.63:0/1504655463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c034eb5a0 con 0x563c034eabb0 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3534a6f640 1 --2- 172.21.15.63:0/1504655463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034eafd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f352c04ccb0 tx=0x7f352c001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ce42cf91d1cf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f34137fe640 1 -- 172.21.15.63:0/1504655463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352c012df0 con 0x563c034eabb0 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f34137fe640 1 -- 172.21.15.63:0/1504655463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c001310 con 0x563c034eabb0 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f34137fe640 1 -- 172.21.15.63:0/1504655463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352c046aa0 con 0x563c034eabb0 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3538c13800 1 -- 172.21.15.63:0/1504655463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 msgr2=0x563c034eafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3538c13800 1 --2- 172.21.15.63:0/1504655463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034eafd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f352c04ccb0 tx=0x7f352c001f90 comp rx=0 tx=0).stop 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3538c13800 1 -- 172.21.15.63:0/1504655463 shutdown_connections 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3538c13800 1 --2- 172.21.15.63:0/1504655463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034eafd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3538c13800 1 -- 172.21.15.63:0/1504655463 >> 172.21.15.63:0/1504655463 conn(0x563c034e6ed0 msgr2=0x563c034e4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3538c13800 1 -- 172.21.15.63:0/1504655463 shutdown_connections 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.369+0000 7f3538c13800 1 -- 172.21.15.63:0/1504655463 wait complete. 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3538c13800 1 Processor -- start 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3538c13800 1 -- start start 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3538c13800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3538c13800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c034ebd20 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3534a6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3534a6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:41180/0 (socket says 172.21.15.63:41180) 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 learned_addr learned my addr 172.21.15.63:0/3534362799 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.370+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c034f26e0 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.371+0000 7f3534a6f640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034f21a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f352c04bb80 tx=0x7f352c04cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.371+0000 7f3411ffb640 1 -- 172.21.15.63:0/3534362799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352c021c60 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.371+0000 7f3411ffb640 1 -- 172.21.15.63:0/3534362799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c021e00 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.371+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c034f2970 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.371+0000 7f3411ffb640 1 -- 172.21.15.63:0/3534362799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352c02a350 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.371+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c034f5620 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.371+0000 7f3411ffb640 1 -- 172.21.15.63:0/3534362799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f352c046040 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.372+0000 7f3411ffb640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3404036eb0 0x7f3404039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.372+0000 7f3411ffb640 1 -- 172.21.15.63:0/3534362799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f352c00e040 con 0x563c034eabb0 2024-04-23T19:12:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.372+0000 7f353426e640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3404036eb0 0x7f3404039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.372+0000 7f353426e640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3404036eb0 0x7f3404039370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f352408c890 tx=0x7f3524089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.372+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.372+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c034fddd0 0x563c03500290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.372+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c035009e0 con 0x563c034fddd0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3533a6d640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c034fddd0 0x563c03500290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3533a6d640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c034fddd0 0x563c03500290 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3528009050 con 0x563c034fddd0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 20 realm 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c034fd220 con 0x563c034fddd0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3528009050 con 0x563c034fddd0 2024-04-23T19:12:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.373+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c035014d0 con 0x563c034fddd0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3528009050 con 0x563c034fddd0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c03501da0 con 0x563c034fddd0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3528009050 con 0x563c034fddd0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c03502140 con 0x563c034fddd0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3528009050 con 0x563c034fddd0 2024-04-23T19:12:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 20 searching for the correct realm 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563c03504020 0x563c03506410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.374+0000 7f3534a6f640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563c03504020 0x563c03506410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.375+0000 7f3534a6f640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563c03504020 0x563c03506410 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.375+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.375+0000 7f3534a6f640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f352c059040 0x7f352c01c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.375+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352c059040 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.375+0000 7f353426e640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f352c059040 0x7f352c01c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.375+0000 7f353426e640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f352c059040 0x7f352c01c1d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.376+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.376+0000 7f353426e640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f352409b040 0x7f3524066380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.376+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.376+0000 7f3533a6d640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f352409b040 0x7f3524066380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.376+0000 7f3533a6d640 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f352409b040 0x7f3524066380 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3528009050 con 0x7f352409b040 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3528009050 con 0x563c034fddd0 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.377+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352c059040 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.378+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352c059040 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.379+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.380+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.381+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352c059040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352c059040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.382+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.383+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.383+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352c059040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.383+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.383+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c03504020 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.383+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352409b040 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x563c034fddd0 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x563c03506ac0 con 0x7f352c059040 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:12:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.384+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c03502ac0 con 0x563c034fddd0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c03503360 con 0x563c034fddd0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c03506f40 con 0x7f352409b040 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 zone default found 2024-04-23T19:12:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 4 Realm: () 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 zonegroup default 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c035073f0 con 0x7f352c059040 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.385+0000 7f3538c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c035091a0 con 0x563c03504020 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c035096e0 con 0x563c03504020 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 20 started sync module instance, tier type = 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.386+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508610 con 0x7f352c059040 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.387+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.387+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94815753636352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508270 con 0x7f352c059040 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.387+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350a4e0 con 0x563c034fddd0 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.388+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94815753636352] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.388+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.388+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94815753640768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350a8d0 con 0x563c034fddd0 2024-04-23T19:12:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.388+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350b3a0 con 0x563c03504020 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.389+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.389+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94815753645712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350bc40 con 0x563c03504020 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.389+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350c740 con 0x7f352c059040 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.390+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94815753640768] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.390+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94815753645712] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.390+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.391+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94815753641824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350bc40 con 0x7f352c059040 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.391+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350d040 con 0x563c034fddd0 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.392+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.392+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94815753653040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350d000 con 0x563c034fddd0 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.392+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350e020 con 0x7f352409b040 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.392+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94815753641824] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.393+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.393+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94815753657136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350e020 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.393+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350ea30 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.393+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94815753653040] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.394+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94815753657136] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.394+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.394+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94815753648016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350e580 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.394+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350ef40 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(58 notify.7 [create] v32'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94815753648016] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94815753661072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350eeb0 con 0x7f352409b040 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 20 add_watcher() i=0 2024-04-23T19:12:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 20 add_watcher() i=1 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 20 add_watcher() i=2 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 20 add_watcher() i=3 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 20 add_watcher() i=4 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 20 add_watcher() i=5 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.395+0000 7f3538c13800 20 add_watcher() i=6 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.396+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 94815753661072] v32'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.396+0000 7f3538c13800 20 add_watcher() i=7 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.396+0000 7f3538c13800 2 all 8 watchers are set, enabling cache 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.396+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c035122e0 con 0x563c034fddd0 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.396+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.396+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94815753655200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350f800 con 0x563c034fddd0 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.397+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94815753655200] v32'271 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f33ea7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 5 note: GC not initialized 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f33e97e2640 20 reqs_thread_entry: start 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 20 init_complete bucket index max shards: 11 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 20 Filter name: none 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f335affd640 20 reqs_thread_entry: start 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 20 rados->read ofs=0 len=0 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.398+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c03545820 con 0x7f352409b040 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.399+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv12 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.399+0000 7f3538c13800 20 rados_obj.operate() r=0 bl.length=473 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.399+0000 7f3538c13800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-23T19:12:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.399+0000 7f3538c13800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.399+0000 7f3538c13800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.399+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0354be90 con 0x7f352409b040 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v32'13 uv13 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3538c13800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3538c13800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3538c13800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94815753910528 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563c0354a9e0 con 0x563c034fddd0 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== watch-notify(notify (1) cookie 94815753653040 notify 137438953507 ret 0) ==== 691+0+0 (crc 0 0 0) 0x563c032797f0 con 0x563c034fddd0 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f34137fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 137438953507 cookie 94815753653040 notifier 4919 bl.length()=649 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f34137fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f34137fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(64 notify.4 [notify cookie 94815753910528 out=8b] v0'0 uv279 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f34137fe640 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x7f33f8003ea0 con 0x563c034fddd0 2024-04-23T19:12:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.401+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== watch-notify(notify_complete (2) cookie 94815753910528 notify 137438953507 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f352803f730 con 0x563c034fddd0 2024-04-23T19:12:15.437 INFO:teuthology.orchestra.run.smithi063.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-23T19:12:13.484773Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-23T19:12:13.484756Z","tags":[],"group_ids":[]} 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.402+0000 7f33ea7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.402+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94815753655200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03511ea0 con 0x563c034fddd0 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.403+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94815753655200] v32'272 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.403+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94815753636352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c0350d3a0 con 0x7f352c059040 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.404+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94815753636352] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.404+0000 7f3538c13800 20 remove_watcher() i=0 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.404+0000 7f3538c13800 2 removed watcher, disabling cache 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.404+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94815753640768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508a00 con 0x563c034fddd0 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.406+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94815753640768] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.406+0000 7f3538c13800 20 remove_watcher() i=1 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.406+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94815753645712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508a00 con 0x563c03504020 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.407+0000 7f3534a6f640 1 -- 172.21.15.63:0/3534362799 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94815753645712] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352c03b020 con 0x563c03504020 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.407+0000 7f3538c13800 20 remove_watcher() i=2 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.407+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94815753641824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508a00 con 0x7f352c059040 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.408+0000 7f353426e640 1 -- 172.21.15.63:0/3534362799 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94815753641824] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3524078070 con 0x7f352c059040 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.408+0000 7f3538c13800 20 remove_watcher() i=3 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.408+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94815753653040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03509ff0 con 0x563c034fddd0 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.409+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94815753653040] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x563c034fddd0 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.409+0000 7f3538c13800 20 remove_watcher() i=4 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.409+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94815753657136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508a00 con 0x7f352409b040 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.410+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94815753657136] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.410+0000 7f3538c13800 20 remove_watcher() i=5 2024-04-23T19:12:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.410+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94815753648016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508a00 con 0x7f352409b040 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.411+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94815753648016] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.411+0000 7f3538c13800 20 remove_watcher() i=6 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.411+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94815753661072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x563c03508a00 con 0x7f352409b040 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94815753661072] v32'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f352802a040 con 0x7f352409b040 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 20 remove_watcher() i=7 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f352409b040 msgr2=0x7f3524066380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f352409b040 0x7f3524066380 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c034fddd0 msgr2=0x563c03500290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c034fddd0 0x563c03500290 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f352c059040 msgr2=0x7f352c01c1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f352c059040 0x7f352c01c1d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563c03504020 msgr2=0x563c03506410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x563c03504020 0x563c03506410 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3404036eb0 msgr2=0x7f3404039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f3404036eb0 0x7f3404039370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f352408c890 tx=0x7f3524089000 comp rx=0 tx=0).stop 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3533a6d640 1 -- 172.21.15.63:0/3534362799 reap_dead start 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 msgr2=0x563c034f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.412+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034f21a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f352c04bb80 tx=0x7f352c04cc40 comp rx=0 tx=0).stop 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 shutdown_connections 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/3534362799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034eabb0 0x563c034f21a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 >> 172.21.15.63:0/3534362799 conn(0x563c034e6ed0 msgr2=0x563c034e4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 shutdown_connections 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/3534362799 wait complete. 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f340c036e60 msgr2=0x7f340c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f340c036e60 0x7f340c039320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563c034cf810 tx=0x7f352408c760 comp rx=0 tx=0).stop 2024-04-23T19:12:15.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034dd850 msgr2=0x563c034ddc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034dd850 0x563c034ddc70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x563c034c34f0 tx=0x7f352c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 shutdown_connections 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f340c036e60 0x7f340c039320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/1439604968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c034dd850 0x563c034ddc70 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 >> 172.21.15.63:0/1439604968 conn(0x563c034d18f0 msgr2=0x563c034d1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 shutdown_connections 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/1439604968 wait complete. 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563c034ccac0 msgr2=0x563c034ceeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563c034ccac0 0x563c034ceeb0 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c033c6380 msgr2=0x563c034c9390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c033c6380 0x563c034c9390 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f341c036e60 msgr2=0x7f341c039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f341c036e60 0x7f341c039320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3528000f60 tx=0x7f35280012b0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 msgr2=0x563c034c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034c2140 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3524060040 tx=0x7f3524002fb0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 shutdown_connections 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x563c034ccac0 0x563c034ceeb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x563c033c6380 0x563c034c9390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f341c036e60 0x7f341c039320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 --2- 172.21.15.63:0/2279836561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c033c5a20 0x563c034c2140 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.413+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 >> 172.21.15.63:0/2279836561 conn(0x563c033c1600 msgr2=0x563c034b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.414+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 shutdown_connections 2024-04-23T19:12:15.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:12:15.414+0000 7f3538c13800 1 -- 172.21.15.63:0/2279836561 wait complete. 2024-04-23T19:12:15.448 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.449 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-23T19:12:13.484773Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-23T19:12:13.484756Z', 'tags': [], 'group_ids': []} 2024-04-23T19:12:15.449 ERROR:root:0 2024-04-23T19:12:15.449 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-23T19:12:15.456 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.456 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.460 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.460 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-23T19:12:15.466 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.466 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.470 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.470 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2024-04-23T19:12:15.478 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}] 2024-04-23T19:12:15.479 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.483 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.483 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2024-04-23T19:12:15.491 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.491 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.495 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.495 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2024-04-23T19:12:15.502 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2024-04-23T19:12:15.503 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.506 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.507 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2024-04-23T19:12:15.515 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2024-04-23T19:12:15.515 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.520 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.520 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2024-04-23T19:12:15.527 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.527 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.531 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.531 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2024-04-23T19:12:15.536 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.536 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.540 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.540 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2024-04-23T19:12:15.546 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.547 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.550 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.551 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.554 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-23T19:12:15.555 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-23T19:12:15.559 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-23T19:12:15.573 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-23T19:12:15.577 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2024-04-23T19:12:15.577 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-23T19:12:15.583 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:12:15.568289Z', 'creation_time': '2024-04-23T19:12:15.563452Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-23T19:12:15.583 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-23T19:12:15.588 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:12:15.568289Z', 'creation_time': '2024-04-23T19:12:15.563452Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-23T19:12:15.593 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-23T19:12:15.598 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:12:15.568289Z', 'creation_time': '2024-04-23T19:12:15.563452Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-23T19:12:15.599 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2024-04-23T19:12:15.603 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'Message': '', 'RequestId': 'tx0000066c62148184f2879-006628080f-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:12:15.607 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2024-04-23T19:12:15.614 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.614 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2024-04-23T19:12:15.623 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'fud', 'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:12:15.675 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11'} 2024-04-23T19:12:15.685 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.688 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11'} 2024-04-23T19:12:15.699 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.699 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2024-04-23T19:12:15.708 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.714 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2024-04-23T19:12:15.720 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:12:15.721 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-23T19:12:15.725 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '8815d3af-8342-460f-8220-31ec38570cc3', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'marker': 'bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-23T19:12:15.693966Z', 'creation_time': '2024-04-23T19:12:15.563452Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-23T19:13:15.775 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2024-04-23T19:13:15.795 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'anonymous', 'categories': [], 's3select': {}}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'bar', 'categories': [], 's3select': {}}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'bud', 'categories': [], 's3select': {}}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'fud', 'categories': [], 's3select': {}}, {'bucket': 'myfoo7', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'fud', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:13:15.795 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2024-04-23T19:13:15.812 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'useless-bucket', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo7', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 242, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:13:15.813 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-23T19:13:15.817 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'useless-bucket', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:13:15.818 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2024-04-23T19:13:15.822 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:13:15.823 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2024-04-23T19:13:15.827 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:13:15.827 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2024-04-23T19:13:15.831 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:13:15.832 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2024-04-23T19:13:15.836 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-04-23T19:00:00.000000Z', 'epoch': 1713898800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-04-23T19:13:15.836 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2024-04-23T19:13:15.843 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:13:15.844 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-23T19:13:15.848 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2024-04-23T19:13:15.849 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-23T19:13:15.857 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:13:15.862 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-23T19:13:15.871 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:13:16.125 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-23T19:13:16.129 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx0000088a70e947cc28c23-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.239 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-23T19:13:16.244 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-23T19:13:16.249 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-23T19:13:16.253 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'group': 1, 'permission': {'flags': 1}}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-23T19:13:16.371 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2024-04-23T19:13:16.411 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:13:16.411 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-23T19:13:16.417 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2024-04-23T19:13:16.418 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-23T19:13:16.423 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-23T19:13:16.438 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-23T19:13:16.442 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000b500af402be32f59-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.469 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2024-04-23T19:13:16.499 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:13:16.499 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-23T19:13:16.503 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx00000660ac99c044acc01-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.504 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2024-04-23T19:13:16.507 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': 'f80ae34c-e526-409e-90c3-45f2fcaa83f6'}]}} 2024-04-23T19:13:16.507 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2024-04-23T19:13:16.519 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'ratelimit_user', 'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-23T19:13:16.519 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-23T19:13:16.523 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-23T19:13:16.523 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2024-04-23T19:13:16.528 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx0000037dc29d7cf0e5d08-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.528 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2024-04-23T19:13:16.531 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx0000038e365674ea00411-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.542 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-23T19:13:16.546 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-23T19:13:16.547 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2024-04-23T19:13:16.551 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000007f2669ed6950f1d1-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.551 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2024-04-23T19:13:16.555 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx0000041895c5c3bf91069-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.556 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2024-04-23T19:13:16.560 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-23T19:13:16.560 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2024-04-23T19:13:16.563 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000003aabb5f0d3bdaa24-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.564 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2024-04-23T19:13:16.567 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000bd00e03e9503de45-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.567 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2024-04-23T19:13:16.571 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000bbe841c2aacc994d-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.571 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2024-04-23T19:13:16.578 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000ac6d9335ea03edaa-006628084c-4226-default', 'HostId': '4226-default-default'} 2024-04-23T19:13:16.578 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2024-04-23T19:13:16.588 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:13:16.588 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-23T19:13:16.592 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2024-04-23T19:13:16.593 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2024-04-23T19:13:16.598 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:13:16.598 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-23T19:13:16.602 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2024-04-23T19:13:16.602 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2024-04-23T19:13:16.608 INFO:tasks.radosgw_admin_rest: empty result 2024-04-23T19:13:16.614 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-04-23T19:13:16.624 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-04-23T19:13:16.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T19:13:16.625 INFO:tasks.rgw.client.0.smithi063.stdout:2024-04-23T19:13:16.623+0000 7eff57b84640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 34978) UID: 0 2024-04-23T19:13:16.625 INFO:tasks.rgw.client.0.smithi063.stdout:2024-04-23T19:13:16.623+0000 7eff5b3dd880 -1 shutting down 2024-04-23T19:13:19.590 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-04-23T19:13:22.725 INFO:tasks.rgw.client.0:Stopped 2024-04-23T19:13:22.726 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-04-23T19:13:22.745 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/vault-root-token 2024-04-23T19:13:22.815 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-04-23T19:13:22.882 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-04-23T19:13:22.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-04-23T19:13:22.882 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-04-23T19:13:22.969 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-04-23T19:13:22.969 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.968+0000 7fce35b36800 1 Processor -- start 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.968+0000 7fce35b36800 1 -- start start 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.968+0000 7fce35b36800 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5559a69f2ba0 0x5559a69f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.968+0000 7fce35b36800 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5559a68a47f0 con 0x5559a69f54d0 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.968+0000 7fce35b36800 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x5559a6893e20 con 0x5559a69f2ba0 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.969+0000 7fce318cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x5559a69f2ba0 0x5559a69f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.969+0000 7fce310cb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x5559a69f54d0 0x5559a69f78c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.63:32784/0 (socket says 172.21.15.63:32784) 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.969+0000 7fce310cb640 1 -- 172.21.15.63:0/902542918 learned_addr learned my addr 172.21.15.63:0/902542918 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-04-23T19:13:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.969+0000 7fce318cc640 1 -- 172.21.15.63:0/902542918 >> v1:172.21.15.16:6789/0 conn(0x5559a69f54d0 legacy=0x5559a69f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.969+0000 7fce318cc640 1 -- 172.21.15.63:0/902542918 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a68b2b50 con 0x5559a69f2ba0 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.971+0000 7fce318cc640 1 --2- 172.21.15.63:0/902542918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f4f90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fce2400a470 tx=0x7fce2401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3991808a1c40fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.971+0000 7fce308ca640 1 -- 172.21.15.63:0/902542918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce24057070 con 0x5559a69f2ba0 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.971+0000 7fce308ca640 1 -- 172.21.15.63:0/902542918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce24052570 con 0x5559a69f2ba0 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.971+0000 7fce308ca640 1 -- 172.21.15.63:0/902542918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce24052870 con 0x5559a69f2ba0 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.972+0000 7fce35b36800 1 -- 172.21.15.63:0/902542918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 msgr2=0x5559a69f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.972+0000 7fce35b36800 1 --2- 172.21.15.63:0/902542918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f4f90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fce2400a470 tx=0x7fce2401ce90 comp rx=0 tx=0).stop 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.972+0000 7fce35b36800 1 -- 172.21.15.63:0/902542918 shutdown_connections 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.972+0000 7fce35b36800 1 --2- 172.21.15.63:0/902542918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f4f90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:22.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.972+0000 7fce35b36800 1 -- 172.21.15.63:0/902542918 >> 172.21.15.63:0/902542918 conn(0x5559a69ec630 msgr2=0x5559a69eea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:22.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.972+0000 7fce35b36800 1 -- 172.21.15.63:0/902542918 shutdown_connections 2024-04-23T19:13:22.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.972+0000 7fce35b36800 1 -- 172.21.15.63:0/902542918 wait complete. 2024-04-23T19:13:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.986+0000 7fce35b36800 1 Processor -- start 2024-04-23T19:13:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.986+0000 7fce35b36800 1 -- start start 2024-04-23T19:13:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.986+0000 7fce35b36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.986+0000 7fce35b36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559a693d6f0 con 0x5559a69f2ba0 2024-04-23T19:13:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fce318cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fce318cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:44226/0 (socket says 172.21.15.63:44226) 2024-04-23T19:13:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fce318cc640 1 -- 172.21.15.63:0/34693938 learned_addr learned my addr 172.21.15.63:0/34693938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:13:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fce318cc640 1 -- 172.21.15.63:0/34693938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a69f0aa0 con 0x5559a69f2ba0 2024-04-23T19:13:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fce318cc640 1 --2- 172.21.15.63:0/34693938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f24e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fce2401ced0 tx=0x7fce2404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ccbdc06849c923 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fcd1fe07640 1 -- 172.21.15.63:0/34693938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce24057040 con 0x5559a69f2ba0 2024-04-23T19:13:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fcd1fe07640 1 -- 172.21.15.63:0/34693938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce240529a0 con 0x5559a69f2ba0 2024-04-23T19:13:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.987+0000 7fcd1fe07640 1 -- 172.21.15.63:0/34693938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce24052cc0 con 0x5559a69f2ba0 2024-04-23T19:13:22.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 -- 172.21.15.63:0/34693938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 msgr2=0x5559a69f24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:22.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 --2- 172.21.15.63:0/34693938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f24e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fce2401ced0 tx=0x7fce2404c000 comp rx=0 tx=0).stop 2024-04-23T19:13:22.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 -- 172.21.15.63:0/34693938 shutdown_connections 2024-04-23T19:13:22.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 --2- 172.21.15.63:0/34693938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a69f24e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:22.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 -- 172.21.15.63:0/34693938 >> 172.21.15.63:0/34693938 conn(0x5559a69ec630 msgr2=0x5559a6adec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:22.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 -- 172.21.15.63:0/34693938 shutdown_connections 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 -- 172.21.15.63:0/34693938 wait complete. 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.988+0000 7fce35b36800 1 Processor -- start 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce35b36800 1 -- start start 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce35b36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a6aecf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce35b36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559a69f9280 con 0x5559a69f2ba0 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce318cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a6aecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce318cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a6aecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:44234/0 (socket says 172.21.15.63:44234) 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce318cc640 1 -- 172.21.15.63:0/658430635 learned_addr learned my addr 172.21.15.63:0/658430635 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce318cc640 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a6aed440 con 0x5559a69f2ba0 2024-04-23T19:13:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.989+0000 7fce318cc640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a6aecf00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fce24052740 tx=0x7fce24003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.990+0000 7fcd1e604640 1 -- 172.21.15.63:0/658430635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce24073070 con 0x5559a69f2ba0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.990+0000 7fcd1e604640 1 -- 172.21.15.63:0/658430635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2405d860 con 0x5559a69f2ba0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.990+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5559a6aed6d0 con 0x5559a69f2ba0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.990+0000 7fcd1e604640 1 -- 172.21.15.63:0/658430635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce24057070 con 0x5559a69f2ba0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.990+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a6af0380 con 0x5559a69f2ba0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.990+0000 7fcd1e604640 1 -- 172.21.15.63:0/658430635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fce24076070 con 0x5559a69f2ba0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fcd1e604640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd18036e60 0x7fcd18039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fcd1e604640 1 -- 172.21.15.63:0/658430635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fce2408e920 con 0x5559a69f2ba0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fce310cb640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd18036e60 0x7fcd18039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a69f54d0 0x5559a6af4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6af4710 con 0x5559a69f54d0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fce320cd640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a69f54d0 0x5559a6af4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fce310cb640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd18036e60 0x7fcd18039320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce28000f60 tx=0x7fce280012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.991+0000 7fce320cd640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a69f54d0 0x5559a6af4140 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.992+0000 7fce320cd640 1 -- 172.21.15.63:0/658430635 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce2c00a4a0 con 0x5559a69f54d0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.992+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6af1710 con 0x5559a69f54d0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.992+0000 7fce320cd640 1 -- 172.21.15.63:0/658430635 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fce2c00a4a0 con 0x5559a69f54d0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.992+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6af6550 con 0x5559a69f54d0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.992+0000 7fce320cd640 1 -- 172.21.15.63:0/658430635 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fce2c00a4a0 con 0x5559a69f54d0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.992+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6af5890 con 0x5559a69f54d0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.993+0000 7fce320cd640 1 -- 172.21.15.63:0/658430635 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fce2c00a4a0 con 0x5559a69f54d0 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.993+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559a6af7900 0x5559a6af9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.993+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:5 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6afa270 con 0x5559a6af7900 2024-04-23T19:13:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.993+0000 7fce318cc640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559a6af7900 0x5559a6af9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.994+0000 7fce318cc640 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559a6af7900 0x5559a6af9cf0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.994+0000 7fce318cc640 1 -- 172.21.15.63:0/658430635 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(5 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fce24075030 con 0x5559a6af7900 2024-04-23T19:13:23.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.994+0000 7fce35b36800 1 Processor -- start 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.994+0000 7fce35b36800 1 -- start start 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.994+0000 7fce35b36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b086b0 0x5559a6b08ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.994+0000 7fce35b36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559a68e3e30 con 0x5559a6b086b0 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fce320cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b086b0 0x5559a6b08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fce320cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b086b0 0x5559a6b08ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:44236/0 (socket says 172.21.15.63:44236) 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fce320cd640 1 -- 172.21.15.63:0/696275859 learned_addr learned my addr 172.21.15.63:0/696275859 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fce320cd640 1 -- 172.21.15.63:0/696275859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a6b09010 con 0x5559a6b086b0 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fce320cd640 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b086b0 0x5559a6b08ad0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x5559a6aee300 tx=0x7fce2c0178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fcd177fe640 1 -- 172.21.15.63:0/696275859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce2c013440 con 0x5559a6b086b0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fcd177fe640 1 -- 172.21.15.63:0/696275859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c0135e0 con 0x5559a6b086b0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5559a6b09a80 con 0x5559a6b086b0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.995+0000 7fcd177fe640 1 -- 172.21.15.63:0/696275859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce2c0138c0 con 0x5559a6b086b0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.996+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a6b09f80 con 0x5559a6b086b0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.996+0000 7fcd177fe640 1 -- 172.21.15.63:0/696275859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fce2c011040 con 0x5559a6b086b0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.996+0000 7fcd177fe640 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd0c036eb0 0x7fcd0c039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fcd177fe640 1 -- 172.21.15.63:0/696275859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fce2c01d070 con 0x5559a6b086b0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce318cc640 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd0c036eb0 0x7fcd0c039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce35b36800 1 Processor -- start 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce318cc640 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd0c036eb0 0x7fcd0c039370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5559a6af5c30 tx=0x7fce2408cb70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce35b36800 1 -- start start 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce35b36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce35b36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559a69f0d60 con 0x5559a6b159a0 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce320cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce320cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b15dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:44240/0 (socket says 172.21.15.63:44240) 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.997+0000 7fce320cd640 1 -- 172.21.15.63:0/2143815568 learned_addr learned my addr 172.21.15.63:0/2143815568 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.998+0000 7fce320cd640 1 -- 172.21.15.63:0/2143815568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a6b16390 con 0x5559a6b159a0 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.998+0000 7fce320cd640 1 --2- 172.21.15.63:0/2143815568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b15dc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fce2c04cc70 tx=0x7fce2c04cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35557662264ba8e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.998+0000 7fcd157fa640 1 -- 172.21.15.63:0/2143815568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce2c012cb0 con 0x5559a6b159a0 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.998+0000 7fcd157fa640 1 -- 172.21.15.63:0/2143815568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c012e50 con 0x5559a6b159a0 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.998+0000 7fcd157fa640 1 -- 172.21.15.63:0/2143815568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce2c001460 con 0x5559a6b159a0 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.998+0000 7fce35b36800 1 -- 172.21.15.63:0/2143815568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 msgr2=0x5559a6b15dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.998+0000 7fce35b36800 1 --2- 172.21.15.63:0/2143815568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b15dc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fce2c04cc70 tx=0x7fce2c04cd80 comp rx=0 tx=0).stop 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 -- 172.21.15.63:0/2143815568 shutdown_connections 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 --2- 172.21.15.63:0/2143815568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b15dc0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 -- 172.21.15.63:0/2143815568 >> 172.21.15.63:0/2143815568 conn(0x5559a6b11d50 msgr2=0x5559a6b0fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 -- 172.21.15.63:0/2143815568 shutdown_connections 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 -- 172.21.15.63:0/2143815568 wait complete. 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 Processor -- start 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 -- start start 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b1ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce35b36800 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559a6b16aa0 con 0x5559a6b159a0 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce320cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b1ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce320cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b1ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.63:44254/0 (socket says 172.21.15.63:44254) 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:22.999+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 learned_addr learned my addr 172.21.15.63:0/970096184 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-04-23T19:13:23.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.000+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a6b1d3f0 con 0x5559a6b159a0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.000+0000 7fce320cd640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b1ceb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fce2c04cba0 tx=0x7fce2c04cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.000+0000 7fcd037fe640 1 -- 172.21.15.63:0/970096184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce2c02aa60 con 0x5559a6b159a0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.000+0000 7fcd037fe640 1 -- 172.21.15.63:0/970096184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c02ac00 con 0x5559a6b159a0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.000+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5559a6b1d680 con 0x5559a6b159a0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.000+0000 7fcd037fe640 1 -- 172.21.15.63:0/970096184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce2c021a50 con 0x5559a6b159a0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.000+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a6b20330 con 0x5559a6b159a0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.001+0000 7fcd037fe640 1 -- 172.21.15.63:0/970096184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fce2c041040 con 0x5559a6b159a0 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.001+0000 7fcd037fe640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fccfc036eb0 0x7fccfc039370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.001+0000 7fcd037fe640 1 -- 172.21.15.63:0/970096184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fce2c00e040 con 0x5559a6b159a0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.001+0000 7fce318cc640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fccfc036eb0 0x7fccfc039370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.002+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.002+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a6b28af0 0x5559a6b2afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.002+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b2b700 con 0x5559a6b28af0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.002+0000 7fce318cc640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fccfc036eb0 0x7fccfc039370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce2408e010 tx=0x7fce24088040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.002+0000 7fce310cb640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a6b28af0 0x5559a6b2afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.002+0000 7fce310cb640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a6b28af0 0x5559a6b2afb0 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce28009050 con 0x5559a6b28af0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:13:23.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 realm 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b27f40 con 0x5559a6b28af0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce28009050 con 0x5559a6b28af0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b2c1f0 con 0x5559a6b28af0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce28009050 con 0x5559a6b28af0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:13:23.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b2cac0 con 0x5559a6b28af0 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fce28009050 con 0x5559a6b28af0 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.003+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:5 5.1d 5:bfc21ac0:::zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b2ce60 con 0x5559a6b28af0 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 5 ==== osd_op_reply(5 zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fce28009050 con 0x5559a6b28af0 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce35b36800 20 rados_obj.operate() r=0 bl.length=993 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce35b36800 20 searching for the correct realm 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559a6b2ed40 0x5559a6b31150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x5559a6b2ed40 2024-04-23T19:13:23.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce320cd640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559a6b2ed40 0x5559a6b31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce320cd640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559a6b2ed40 0x5559a6b31150 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.004+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== osd_op_reply(6 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce320cd640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fce2c021bf0 0x7fce2c02b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x7fce2c021bf0 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce318cc640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fce2c021bf0 0x7fce2c02b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce318cc640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fce2c021bf0 0x7fce2c02b860 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== osd_op_reply(7 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce318cc640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fce2409b040 0x7fce24076360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x7fce2409b040 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.005+0000 7fce310cb640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fce2409b040 0x7fce24076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.006+0000 7fce310cb640 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fce2409b040 0x7fce24076360 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.006+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== osd_op_reply(8 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce28009050 con 0x7fce2409b040 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.006+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x5559a6b28af0 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.006+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 6 ==== osd_op_reply(9 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce28009050 con 0x5559a6b28af0 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.006+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x5559a6b28af0 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.007+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 7 ==== osd_op_reply(10 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.007+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x7fce2409b040 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.007+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== osd_op_reply(11 [pgnls start_epoch 32 out=112b] v23'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.007+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x7fce2409b040 2024-04-23T19:13:23.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.007+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 3 ==== osd_op_reply(12 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.007+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x7fce2c021bf0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== osd_op_reply(13 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x5559a6b2ed40 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== osd_op_reply(14 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x5559a6b28af0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 8 ==== osd_op_reply(15 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x7fce2409b040 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 4 ==== osd_op_reply(16 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.008+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x5559a6b28af0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 9 ==== osd_op_reply(17 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31820 con 0x7fce2c021bf0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 3 ==== osd_op_reply(18 [pgnls start_epoch 32 out=79b] v23'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 3 ==== osd_op_reply(19 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 4 ==== osd_op_reply(20 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.009+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2409b040 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.010+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 5 ==== osd_op_reply(21 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.010+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.010+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 5 ==== osd_op_reply(22 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.010+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.010+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 6 ==== osd_op_reply(23 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.010+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b28af0 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.011+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 10 ==== osd_op_reply(24 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.011+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b28af0 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.011+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 11 ==== osd_op_reply(25 [pgnls start_epoch 32 out=74b] v32'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.011+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2409b040 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.011+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 6 ==== osd_op_reply(26 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.011+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b2ed40 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 7 ==== osd_op_reply(27 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2c021bf0 2024-04-23T19:13:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 4 ==== osd_op_reply(28 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b28af0 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 12 ==== osd_op_reply(29 [pgnls start_epoch 32 out=107b] v32'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2c021bf0 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 5 ==== osd_op_reply(30 [pgnls start_epoch 32 out=82b] v32'1 uv1 ondisk = 1) ==== 144+0+82 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.012+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b2ed40 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.013+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 8 ==== osd_op_reply(31 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.013+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2409b040 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.013+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 7 ==== osd_op_reply(32 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.013+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2c021bf0 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.013+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 6 ==== osd_op_reply(33 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.013+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b2ed40 2024-04-23T19:13:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.014+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 9 ==== osd_op_reply(34 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.014+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2409b040 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.014+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 8 ==== osd_op_reply(35 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.014+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x5559a6b28af0 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.014+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 13 ==== osd_op_reply(36 [pgnls start_epoch 32 out=115b] v32'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.014+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 32 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e32) -- 0x7fce240a9060 con 0x7fce2c021bf0 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 7 ==== osd_op_reply(37 [pgnls start_epoch 32 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 RGWRados::pool_iterate: got zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 RGWRados::pool_iterate: got default.zone. 2024-04-23T19:13:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 RGWRados::pool_iterate: got zone_info.bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 RGWRados::pool_iterate: got period_config.default 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b2d7e0 con 0x5559a6b28af0 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b2e080 con 0x5559a6b28af0 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-23T19:13:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.015+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:40 5.14 5:28161b5e:::zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b31ca0 con 0x7fce2409b040 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 9 ==== osd_op_reply(40 zonegroup_info.8815d3af-8342-460f-8220-31ec38570cc3 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 rados_obj.operate() r=0 bl.length=436 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 zone default found 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 4 Realm: () 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 4 ZoneGroup: default (8815d3af-8342-460f-8220-31ec38570cc3) 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 4 Zone: default (bc28bc2a-1a44-4b88-9ab6-dfcb32cea038) 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 zonegroup default 2024-04-23T19:13:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b32150 con 0x7fce2c021bf0 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 8 ==== osd_op_reply(41 period_config.default [read 0~187 out=187b] v0'0 uv1 ondisk = 0) ==== 165+0+187 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 rados_obj.operate() r=0 bl.length=187 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b33f60 con 0x5559a6b2ed40 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 20 rados->read ofs=0 len=0 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.016+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b34470 con 0x5559a6b2ed40 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.017+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.017+0000 7fce35b36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-23T19:13:23.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.017+0000 7fce35b36800 20 started sync module instance, tier type = 2024-04-23T19:13:23.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.017+0000 7fce35b36800 20 started zone id=bc28bc2a-1a44-4b88-9ab6-dfcb32cea038 (name=default) with tier type = 2024-04-23T19:13:23.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.017+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b33370 con 0x7fce2c021bf0 2024-04-23T19:13:23.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.018+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 9 ==== osd_op_reply(44 notify.0 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.018+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93843537213280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b33b80 con 0x7fce2c021bf0 2024-04-23T19:13:23.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.018+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b353a0 con 0x5559a6b28af0 2024-04-23T19:13:23.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.019+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93843537213280] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.019+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 16 ==== osd_op_reply(46 notify.1 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.019+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93843537219312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b357c0 con 0x5559a6b28af0 2024-04-23T19:13:23.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.019+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b36180 con 0x5559a6b2ed40 2024-04-23T19:13:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.020+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93843537219312] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.021+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 12 ==== osd_op_reply(48 notify.2 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.021+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93843537220368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b35730 con 0x5559a6b2ed40 2024-04-23T19:13:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.021+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b36ff0 con 0x7fce2c021bf0 2024-04-23T19:13:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.022+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 11 ==== osd_op_reply(50 notify.3 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.022+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93843537226560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b37840 con 0x7fce2c021bf0 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.022+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b382f0 con 0x5559a6b28af0 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.022+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93843537220368] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.023+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 18 ==== osd_op_reply(52 notify.4 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.023+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93843537224992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b386e0 con 0x5559a6b28af0 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.023+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b39120 con 0x7fce2409b040 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.023+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93843537226560] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.024+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93843537224992] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.024+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 10 ==== osd_op_reply(54 notify.5 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.024+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93843537234928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b390e0 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.024+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b399c0 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.025+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93843537234928] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.026+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 12 ==== osd_op_reply(56 notify.6 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.026+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93843537228688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b390e0 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.026+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b39fe0 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93843537228688] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 14 ==== osd_op_reply(58 notify.7 [create] v32'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93843537237184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b39fe0 con 0x7fce2409b040 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 20 add_watcher() i=0 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 20 add_watcher() i=1 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 20 add_watcher() i=2 2024-04-23T19:13:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 20 add_watcher() i=3 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 20 add_watcher() i=4 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 20 add_watcher() i=5 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.027+0000 7fce35b36800 20 add_watcher() i=6 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.028+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 15 ==== osd_op_reply(59 notify.7 [watch watch cookie 93843537237184] v32'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.028+0000 7fce35b36800 20 add_watcher() i=7 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.028+0000 7fce35b36800 2 all 8 watchers are set, enabling cache 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.028+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b3d400 con 0x5559a6b28af0 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.029+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.029+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93843537239296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b3a860 con 0x5559a6b28af0 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.030+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93843537239296] v32'274 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.030+0000 7fcce37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:13:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.030+0000 7fce35b36800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.030+0000 7fce35b36800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.030+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b45c40 con 0x5559a6b28af0 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.031+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v32'38 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.031+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46290 con 0x7fce2409b040 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.032+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 16 ==== osd_op_reply(63 gc.1 [create,call,call,call] v32'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.032+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b466d0 con 0x5559a6b2ed40 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.033+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v32'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.033+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46720 con 0x5559a6b28af0 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.034+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v32'275 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.034+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46cb0 con 0x7fce2c021bf0 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.035+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v32'84 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.036+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b47160 con 0x5559a6b2ed40 2024-04-23T19:13:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.037+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v32'55 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.037+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b47160 con 0x5559a6b28af0 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.038+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v32'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.038+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b471f0 con 0x7fce2409b040 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.039+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 17 ==== osd_op_reply(69 gc.7 [create,call,call,call] v32'55 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.039+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b471f0 con 0x7fce2c021bf0 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.040+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v32'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.040+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b2ed40 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.041+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v32'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.041+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2c021bf0 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.042+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v32'85 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.042+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2409b040 2024-04-23T19:13:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.043+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 18 ==== osd_op_reply(73 gc.11 [create,call,call,call] v32'41 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.043+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2c021bf0 2024-04-23T19:13:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.044+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v32'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.045+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2409b040 2024-04-23T19:13:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.046+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 19 ==== osd_op_reply(75 gc.13 [create,call,call,call] v32'64 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.046+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b28af0 2024-04-23T19:13:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.047+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v32'276 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.047+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b2ed40 2024-04-23T19:13:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.048+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v32'78 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.048+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2c021bf0 2024-04-23T19:13:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.049+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v32'86 uv83 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.049+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b2ed40 2024-04-23T19:13:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.050+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v32'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.051+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b28af0 2024-04-23T19:13:23.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.052+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v32'39 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.052+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b28af0 2024-04-23T19:13:23.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.053+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v32'277 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.053+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b2ed40 2024-04-23T19:13:23.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.054+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v32'79 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.054+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2c021bf0 2024-04-23T19:13:23.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.055+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v32'41 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.055+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b2ed40 2024-04-23T19:13:23.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.056+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v32'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.056+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b2ed40 2024-04-23T19:13:23.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.057+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v32'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.057+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46cb0 con 0x5559a6b2ed40 2024-04-23T19:13:23.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.058+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v32'27 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.058+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46cb0 con 0x5559a6b2ed40 2024-04-23T19:13:23.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.059+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v32'56 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.059+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46cb0 con 0x7fce2409b040 2024-04-23T19:13:23.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.061+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 20 ==== osd_op_reply(88 gc.26 [create,call,call,call] v32'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.061+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46cb0 con 0x5559a6b2ed40 2024-04-23T19:13:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.062+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v32'33 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.062+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b46cb0 con 0x7fce2409b040 2024-04-23T19:13:23.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.063+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 21 ==== osd_op_reply(90 gc.28 [create,call,call,call] v32'56 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.063+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x5559a6b2ed40 2024-04-23T19:13:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.064+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v32'80 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.064+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2409b040 2024-04-23T19:13:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.066+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 22 ==== osd_op_reply(92 gc.30 [create,call,call,call] v32'65 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.066+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b477e0 con 0x7fce2c021bf0 2024-04-23T19:13:23.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.067+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v32'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.067+0000 7fcce27fc640 20 reqs_thread_entry: start 2024-04-23T19:13:23.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.067+0000 7fce35b36800 20 init_complete bucket index max shards: 11 2024-04-23T19:13:23.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.067+0000 7fce35b36800 20 Filter name: none 2024-04-23T19:13:23.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.067+0000 7fcc53fff640 20 reqs_thread_entry: start 2024-04-23T19:13:23.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.067+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-23T19:13:23.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.067+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:94 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.069+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 23 ==== osd_op_reply(94 gc.7 [call] v32'57 uv57 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.069+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:95 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.069+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 24 ==== osd_op_reply(95 gc.7 [call out=15b] v0'0 uv57 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.069+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.069+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:96 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.069+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 25 ==== osd_op_reply(96 gc.7 [call out=24b] v0'0 uv57 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.069+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:97 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.070+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 26 ==== osd_op_reply(97 gc.7 [call out=15b] v0'0 uv57 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.070+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.070+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:98 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.070+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 27 ==== osd_op_reply(98 gc.7 [call out=15b] v0'0 uv57 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.070+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.070+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:99 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.071+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 28 ==== osd_op_reply(99 gc.7 [call] v32'58 uv58 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.071+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-23T19:13:23.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.071+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:100 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.072+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 20 ==== osd_op_reply(100 gc.8 [call] v32'34 uv34 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.072+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:101 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.072+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 21 ==== osd_op_reply(101 gc.8 [call out=15b] v0'0 uv34 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:102 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 22 ==== osd_op_reply(102 gc.8 [call out=24b] v0'0 uv34 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:103 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 23 ==== osd_op_reply(103 gc.8 [call out=15b] v0'0 uv34 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:104 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 24 ==== osd_op_reply(104 gc.8 [call out=15b] v0'0 uv34 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.073+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:105 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.075+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 25 ==== osd_op_reply(105 gc.8 [call] v32'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.075+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.075+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:106 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.076+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 26 ==== osd_op_reply(106 gc.9 [call] v32'38 uv38 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.076+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:107 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.076+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 27 ==== osd_op_reply(107 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.076+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.076+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:108 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.076+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 28 ==== osd_op_reply(108 gc.9 [call out=24b] v0'0 uv38 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.076+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:109 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.077+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 29 ==== osd_op_reply(109 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.077+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.077+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:110 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.077+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 30 ==== osd_op_reply(110 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.077+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.077+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:111 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.078+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 31 ==== osd_op_reply(111 gc.9 [call] v32'39 uv39 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.078+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-23T19:13:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.078+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:112 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 26 ==== osd_op_reply(112 gc.10 [call] v32'87 uv87 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:113 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 27 ==== osd_op_reply(113 gc.10 [call out=15b] v0'0 uv87 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:114 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x7fce2c021bf0 2024-04-23T19:13:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 28 ==== osd_op_reply(114 gc.10 [call out=24b] v0'0 uv87 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:115 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.079+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 29 ==== osd_op_reply(115 gc.10 [call out=15b] v0'0 uv87 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.080+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.080+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:116 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.080+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 30 ==== osd_op_reply(116 gc.10 [call out=15b] v0'0 uv87 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.080+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.080+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:117 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.081+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 31 ==== osd_op_reply(117 gc.10 [call] v32'88 uv88 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.081+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-23T19:13:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.081+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:118 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.082+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 29 ==== osd_op_reply(118 gc.11 [call] v32'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.082+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:119 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.082+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 30 ==== osd_op_reply(119 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.082+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.082+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:120 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x7fce2409b040 2024-04-23T19:13:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.083+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 31 ==== osd_op_reply(120 gc.11 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.083+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:121 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.083+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 32 ==== osd_op_reply(121 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.083+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.083+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:122 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.083+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 33 ==== osd_op_reply(122 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.084+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.084+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:123 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.085+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 34 ==== osd_op_reply(123 gc.11 [call] v32'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.085+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-23T19:13:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.085+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:124 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.086+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 32 ==== osd_op_reply(124 gc.12 [call] v32'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.086+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:125 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.086+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 33 ==== osd_op_reply(125 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.086+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.086+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:126 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x7fce2c021bf0 2024-04-23T19:13:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.086+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 34 ==== osd_op_reply(126 gc.12 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.086+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:127 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.087+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 35 ==== osd_op_reply(127 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.087+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.087+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:128 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.087+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 36 ==== osd_op_reply(128 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.087+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.087+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:129 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.088+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 37 ==== osd_op_reply(129 gc.12 [call] v32'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.088+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-23T19:13:23.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.088+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:130 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.090+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 35 ==== osd_op_reply(130 gc.13 [call] v32'66 uv66 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.090+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:131 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.090+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 36 ==== osd_op_reply(131 gc.13 [call out=15b] v0'0 uv66 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.090+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.090+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:132 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x7fce2409b040 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.090+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 37 ==== osd_op_reply(132 gc.13 [call out=24b] v0'0 uv66 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.090+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.091+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 38 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv66 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.091+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.091+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.091+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 39 ==== osd_op_reply(134 gc.13 [call out=15b] v0'0 uv66 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.091+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.091+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2409b040 2024-04-23T19:13:23.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.092+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 40 ==== osd_op_reply(135 gc.13 [call] v32'67 uv67 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.092+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-23T19:13:23.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.092+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.093+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 28 ==== osd_op_reply(136 gc.14 [call] v32'278 uv278 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.093+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 29 ==== osd_op_reply(137 gc.14 [call out=15b] v0'0 uv278 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 30 ==== osd_op_reply(138 gc.14 [call out=24b] v0'0 uv278 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:139 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 31 ==== osd_op_reply(139 gc.14 [call out=15b] v0'0 uv278 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.094+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:140 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.095+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 32 ==== osd_op_reply(140 gc.14 [call out=15b] v0'0 uv278 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.095+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.095+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:141 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b28af0 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.096+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 33 ==== osd_op_reply(141 gc.14 [call] v32'279 uv279 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.096+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.096+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:142 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.097+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 32 ==== osd_op_reply(142 gc.15 [call] v32'81 uv81 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.097+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:143 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.097+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 33 ==== osd_op_reply(143 gc.15 [call out=15b] v0'0 uv81 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.097+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.097+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:144 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x5559a6b2ed40 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.097+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 34 ==== osd_op_reply(144 gc.15 [call out=24b] v0'0 uv81 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.097+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:145 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.098+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 35 ==== osd_op_reply(145 gc.15 [call out=15b] v0'0 uv81 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.098+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.098+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:146 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.098+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 36 ==== osd_op_reply(146 gc.15 [call out=15b] v0'0 uv81 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.098+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.098+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:147 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x5559a6b2ed40 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.099+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 37 ==== osd_op_reply(147 gc.15 [call] v32'82 uv82 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.099+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-23T19:13:23.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.099+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:148 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.100+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 38 ==== osd_op_reply(148 gc.16 [call] v32'89 uv89 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.100+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:149 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.100+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 39 ==== osd_op_reply(149 gc.16 [call out=15b] v0'0 uv89 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.100+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.100+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:150 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72b20 con 0x7fce2c021bf0 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.100+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 40 ==== osd_op_reply(150 gc.16 [call out=24b] v0'0 uv89 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.100+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:151 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 41 ==== osd_op_reply(151 gc.16 [call out=15b] v0'0 uv89 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:152 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b73b30 con 0x7fce2c021bf0 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 42 ==== osd_op_reply(152 gc.16 [call out=1715b] v0'0 uv89 ondisk = 0) ==== 149+0+1715 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 20 garbage collection: RGWGC::process iterating over entry tag='bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.2334725585701816996', time=2024-04-23T19:13:31.121601+0000, chain.objs.size()=7 2024-04-23T19:13:23.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_1 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:153 3.3as0 3:5f032cb1:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_1:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b71dd0 con 0x7fce2c021bf0 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_2 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:154 3.23s0 3:c763b83f:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_2:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b76bf0 con 0x7fce2409b040 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_3 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:155 3.21s0 3:851d17bf:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_3:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b79510 con 0x7fce2409b040 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_4 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:156 3.35s0 3:aec504a6:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_4:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b7bd80 con 0x5559a6b28af0 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_5 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:157 3.19s0 3:994e312b:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_5:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b79510 con 0x5559a6b28af0 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_6 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:158 3.38s0 3:1ca666ec:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_6:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b79510 con 0x7fce2409b040 2024-04-23T19:13:23.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_7 2024-04-23T19:13:23.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.101+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:159 3.1fs0 3:f9816701:::bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_7:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.103+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 41 ==== osd_op_reply(155 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_3 [call] v32'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.103+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 34 ==== osd_op_reply(159 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_7 [call] v32'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.103+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 43 ==== osd_op_reply(153 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_1 [call] v32'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.103+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 35 ==== osd_op_reply(157 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_5 [call] v32'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.104+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 42 ==== osd_op_reply(158 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_6 [call] v32'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.105+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 36 ==== osd_op_reply(156 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_4 [call] v32'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.105+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 43 ==== osd_op_reply(154 bc28bc2a-1a44-4b88-9ab6-dfcb32cea038.4226.11__shadow_.YoJIgXfWEEngY53MOIhGBotFF7Aer6i_2 [call] v32'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.105+0000 7fce35b36800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-23T19:13:23.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.105+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:160 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x7fce2c021bf0 2024-04-23T19:13:23.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.111+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 44 ==== osd_op_reply(160 gc.16 [call] v32'90 uv90 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.111+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:161 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x7fce2c021bf0 2024-04-23T19:13:23.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.114+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 45 ==== osd_op_reply(161 gc.16 [call] v32'91 uv91 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.114+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-23T19:13:23.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.114+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:162 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.115+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 38 ==== osd_op_reply(162 gc.17 [call] v32'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.115+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:163 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.115+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 39 ==== osd_op_reply(163 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.115+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.115+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:164 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 40 ==== osd_op_reply(164 gc.17 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:165 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 41 ==== osd_op_reply(165 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:166 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b2ed40 2024-04-23T19:13:23.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 42 ==== osd_op_reply(166 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.116+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:167 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b2ed40 2024-04-23T19:13:23.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.117+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 43 ==== osd_op_reply(167 gc.17 [call] v32'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.117+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-23T19:13:23.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.117+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:168 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.118+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 37 ==== osd_op_reply(168 gc.18 [call] v32'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.118+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:169 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 38 ==== osd_op_reply(169 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:170 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 39 ==== osd_op_reply(170 gc.18 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:171 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 40 ==== osd_op_reply(171 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.119+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:172 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.120+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 41 ==== osd_op_reply(172 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.120+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.120+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:173 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.121+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 42 ==== osd_op_reply(173 gc.18 [call] v32'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.121+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-23T19:13:23.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.121+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:174 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.122+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 43 ==== osd_op_reply(174 gc.19 [call] v32'280 uv280 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.122+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:175 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.122+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 44 ==== osd_op_reply(175 gc.19 [call out=15b] v0'0 uv280 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.122+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.122+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:176 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b831b0 con 0x5559a6b28af0 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 45 ==== osd_op_reply(176 gc.19 [call out=24b] v0'0 uv280 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:177 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74670 con 0x5559a6b28af0 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 46 ==== osd_op_reply(177 gc.19 [call out=15b] v0'0 uv280 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:178 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74670 con 0x5559a6b28af0 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 47 ==== osd_op_reply(178 gc.19 [call out=15b] v0'0 uv280 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.123+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:179 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74670 con 0x5559a6b28af0 2024-04-23T19:13:23.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.124+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 48 ==== osd_op_reply(179 gc.19 [call] v32'281 uv281 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.124+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.125+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:180 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74670 con 0x5559a6b2ed40 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.125+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 44 ==== osd_op_reply(180 gc.20 [call] v32'83 uv83 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:181 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 45 ==== osd_op_reply(181 gc.20 [call out=15b] v0'0 uv83 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:182 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 46 ==== osd_op_reply(182 gc.20 [call out=24b] v0'0 uv83 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:183 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 47 ==== osd_op_reply(183 gc.20 [call out=15b] v0'0 uv83 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:184 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 48 ==== osd_op_reply(184 gc.20 [call out=15b] v0'0 uv83 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.126+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:185 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.127+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 49 ==== osd_op_reply(185 gc.20 [call] v32'84 uv84 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.127+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.127+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:186 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.128+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 46 ==== osd_op_reply(186 gc.21 [call] v32'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.128+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:187 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 47 ==== osd_op_reply(187 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:188 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 48 ==== osd_op_reply(188 gc.21 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:189 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 49 ==== osd_op_reply(189 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.129+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:190 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.130+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 50 ==== osd_op_reply(190 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.130+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.130+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:191 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.131+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 51 ==== osd_op_reply(191 gc.21 [call] v32'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.131+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.131+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:192 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 50 ==== osd_op_reply(192 gc.22 [call] v32'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:193 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 51 ==== osd_op_reply(193 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:194 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x5559a6b2ed40 2024-04-23T19:13:23.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 52 ==== osd_op_reply(194 gc.22 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:195 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 53 ==== osd_op_reply(195 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.132+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:196 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.133+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 54 ==== osd_op_reply(196 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.133+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.133+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:197 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.134+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 55 ==== osd_op_reply(197 gc.22 [call] v32'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.134+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.134+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:198 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.134+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 56 ==== osd_op_reply(198 gc.23 [call] v32'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.135+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:199 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.135+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 57 ==== osd_op_reply(199 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.135+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.135+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:200 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75950 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.135+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 58 ==== osd_op_reply(200 gc.23 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.135+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:201 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.135+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 59 ==== osd_op_reply(201 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.136+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.136+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:202 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.136+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 60 ==== osd_op_reply(202 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.136+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.136+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:203 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.137+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 61 ==== osd_op_reply(203 gc.23 [call] v32'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.137+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-23T19:13:23.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.137+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:204 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.138+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 62 ==== osd_op_reply(204 gc.24 [call] v32'28 uv28 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.138+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:205 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.138+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 63 ==== osd_op_reply(205 gc.24 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.138+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.138+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:206 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75950 con 0x5559a6b2ed40 2024-04-23T19:13:23.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.138+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 64 ==== osd_op_reply(206 gc.24 [call out=24b] v0'0 uv28 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.138+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:207 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.139+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 65 ==== osd_op_reply(207 gc.24 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.139+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.139+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:208 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.139+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 66 ==== osd_op_reply(208 gc.24 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.139+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.139+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:209 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.140+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 67 ==== osd_op_reply(209 gc.24 [call] v32'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.140+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.140+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:210 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 68 ==== osd_op_reply(210 gc.25 [call] v32'57 uv57 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:211 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 69 ==== osd_op_reply(211 gc.25 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:212 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75950 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 70 ==== osd_op_reply(212 gc.25 [call out=24b] v0'0 uv57 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:213 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.141+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 71 ==== osd_op_reply(213 gc.25 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.142+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.142+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:214 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.142+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 72 ==== osd_op_reply(214 gc.25 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.142+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.142+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:215 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.143+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 73 ==== osd_op_reply(215 gc.25 [call] v32'58 uv58 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.143+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.143+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:216 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.144+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 44 ==== osd_op_reply(216 gc.26 [call] v32'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.144+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:217 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.144+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 45 ==== osd_op_reply(217 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.144+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.144+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:218 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.144+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 46 ==== osd_op_reply(218 gc.26 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.144+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:219 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.145+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 47 ==== osd_op_reply(219 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.145+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.145+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:220 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.145+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 48 ==== osd_op_reply(220 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.145+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.145+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:221 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.146+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 49 ==== osd_op_reply(221 gc.26 [call] v32'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.146+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.146+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:222 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 74 ==== osd_op_reply(222 gc.27 [call] v32'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:223 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 75 ==== osd_op_reply(223 gc.27 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:224 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x5559a6b2ed40 2024-04-23T19:13:23.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 76 ==== osd_op_reply(224 gc.27 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:225 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.147+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 77 ==== osd_op_reply(225 gc.27 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.148+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.148+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:226 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.148+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 78 ==== osd_op_reply(226 gc.27 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.148+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.148+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:227 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.149+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 79 ==== osd_op_reply(227 gc.27 [call] v32'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.149+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.149+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:228 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.150+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 50 ==== osd_op_reply(228 gc.28 [call] v32'59 uv59 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.150+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:229 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 51 ==== osd_op_reply(229 gc.28 [call out=15b] v0'0 uv59 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:230 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 52 ==== osd_op_reply(230 gc.28 [call out=24b] v0'0 uv59 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:231 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 53 ==== osd_op_reply(231 gc.28 [call out=15b] v0'0 uv59 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.151+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:232 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.152+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 54 ==== osd_op_reply(232 gc.28 [call out=15b] v0'0 uv59 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.152+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.152+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:233 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.153+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 55 ==== osd_op_reply(233 gc.28 [call] v32'60 uv60 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.153+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.153+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:234 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.154+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 80 ==== osd_op_reply(234 gc.29 [call] v32'85 uv85 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.154+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:235 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.154+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 81 ==== osd_op_reply(235 gc.29 [call out=15b] v0'0 uv85 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.154+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.154+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:236 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 82 ==== osd_op_reply(236 gc.29 [call out=24b] v0'0 uv85 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:237 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 83 ==== osd_op_reply(237 gc.29 [call out=15b] v0'0 uv85 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:238 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 84 ==== osd_op_reply(238 gc.29 [call out=15b] v0'0 uv85 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.155+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:239 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.156+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 85 ==== osd_op_reply(239 gc.29 [call] v32'86 uv86 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.156+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.156+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:240 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.157+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 56 ==== osd_op_reply(240 gc.30 [call] v32'68 uv68 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.157+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:241 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.158+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 57 ==== osd_op_reply(241 gc.30 [call out=15b] v0'0 uv68 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.158+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.158+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:242 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.158+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 58 ==== osd_op_reply(242 gc.30 [call out=24b] v0'0 uv68 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.158+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:243 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.159+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 59 ==== osd_op_reply(243 gc.30 [call out=15b] v0'0 uv68 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.159+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.159+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:244 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.159+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 60 ==== osd_op_reply(244 gc.30 [call out=15b] v0'0 uv68 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.159+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.159+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:245 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.160+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 61 ==== osd_op_reply(245 gc.30 [call] v32'69 uv69 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.160+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-23T19:13:23.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.160+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:246 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2c021bf0 2024-04-23T19:13:23.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.161+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 52 ==== osd_op_reply(246 gc.31 [call] v32'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.161+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:247 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2c021bf0 2024-04-23T19:13:23.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.161+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 53 ==== osd_op_reply(247 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.161+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.161+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:248 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x7fce2c021bf0 2024-04-23T19:13:23.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 54 ==== osd_op_reply(248 gc.31 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:249 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2c021bf0 2024-04-23T19:13:23.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 55 ==== osd_op_reply(249 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:250 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 56 ==== osd_op_reply(250 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.162+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:251 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.164+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 57 ==== osd_op_reply(251 gc.31 [call] v32'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.164+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.164+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:252 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b28af0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 49 ==== osd_op_reply(252 gc.0 [call] v32'39 uv39 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:253 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b28af0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 50 ==== osd_op_reply(253 gc.0 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:254 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x5559a6b28af0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 51 ==== osd_op_reply(254 gc.0 [call out=24b] v0'0 uv39 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:255 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 52 ==== osd_op_reply(255 gc.0 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:256 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 53 ==== osd_op_reply(256 gc.0 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.165+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:257 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.166+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 54 ==== osd_op_reply(257 gc.0 [call] v32'40 uv40 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.166+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.166+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:258 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.168+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 62 ==== osd_op_reply(258 gc.1 [call] v32'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.168+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:259 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.168+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 63 ==== osd_op_reply(259 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.168+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.168+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:260 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 64 ==== osd_op_reply(260 gc.1 [call out=24b] v0'0 uv33 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:261 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 65 ==== osd_op_reply(261 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:262 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 66 ==== osd_op_reply(262 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.169+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:263 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x7fce2409b040 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.170+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 67 ==== osd_op_reply(263 gc.1 [call] v32'34 uv34 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.170+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.170+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:264 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.171+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 86 ==== osd_op_reply(264 gc.2 [call] v32'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.171+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:265 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 87 ==== osd_op_reply(265 gc.2 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:266 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x5559a6b2ed40 2024-04-23T19:13:23.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 88 ==== osd_op_reply(266 gc.2 [call out=24b] v0'0 uv33 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:267 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 89 ==== osd_op_reply(267 gc.2 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:268 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 90 ==== osd_op_reply(268 gc.2 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.172+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:269 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b2ed40 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.173+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 91 ==== osd_op_reply(269 gc.2 [call] v32'34 uv34 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.173+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.173+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:270 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b28af0 2024-04-23T19:13:23.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.174+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 55 ==== osd_op_reply(270 gc.3 [call] v32'282 uv282 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.174+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:271 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b74560 con 0x5559a6b28af0 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 56 ==== osd_op_reply(271 gc.3 [call out=15b] v0'0 uv282 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:272 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b75970 con 0x5559a6b28af0 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 57 ==== osd_op_reply(272 gc.3 [call out=24b] v0'0 uv282 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:273 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 58 ==== osd_op_reply(273 gc.3 [call out=15b] v0'0 uv282 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:274 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 59 ==== osd_op_reply(274 gc.3 [call out=15b] v0'0 uv282 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.175+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:275 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.176+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 60 ==== osd_op_reply(275 gc.3 [call] v32'283 uv283 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.176+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.176+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:276 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.177+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 58 ==== osd_op_reply(276 gc.4 [call] v32'92 uv92 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.177+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:277 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 59 ==== osd_op_reply(277 gc.4 [call out=15b] v0'0 uv92 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:278 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 60 ==== osd_op_reply(278 gc.4 [call out=24b] v0'0 uv92 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:279 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 61 ==== osd_op_reply(279 gc.4 [call out=15b] v0'0 uv92 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.178+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:280 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.179+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 62 ==== osd_op_reply(280 gc.4 [call out=15b] v0'0 uv92 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.179+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.179+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:281 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.180+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 63 ==== osd_op_reply(281 gc.4 [call] v32'93 uv93 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.180+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-23T19:13:23.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.180+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:282 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 92 ==== osd_op_reply(282 gc.5 [call] v32'59 uv59 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:283 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 93 ==== osd_op_reply(283 gc.5 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:284 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 94 ==== osd_op_reply(284 gc.5 [call out=24b] v0'0 uv59 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:285 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 95 ==== osd_op_reply(285 gc.5 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.181+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:286 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.182+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 96 ==== osd_op_reply(286 gc.5 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.182+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.182+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:287 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.183+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 97 ==== osd_op_reply(287 gc.5 [call] v32'60 uv60 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.183+0000 7fce35b36800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.183+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:288 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.184+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 61 ==== osd_op_reply(288 gc.6 [call] v32'34 uv34 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.184+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:289 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.184+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 62 ==== osd_op_reply(289 gc.6 [call out=15b] v0'0 uv34 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.184+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.184+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:290 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.184+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 63 ==== osd_op_reply(290 gc.6 [call out=24b] v0'0 uv34 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.184+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:291 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.185+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 64 ==== osd_op_reply(291 gc.6 [call out=15b] v0'0 uv34 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.185+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.185+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:292 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.185+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 65 ==== osd_op_reply(292 gc.6 [call out=15b] v0'0 uv34 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.185+0000 7fce35b36800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.185+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:293 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.186+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 66 ==== osd_op_reply(293 gc.6 [call] v32'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.187+0000 7fcce37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-23T19:13:23.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.187+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93843537239296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.188+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 67 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 93843537239296] v32'284 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.188+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93843537213280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.189+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 64 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 93843537213280] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.189+0000 7fce35b36800 20 remove_watcher() i=0 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.189+0000 7fce35b36800 2 removed watcher, disabling cache 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.189+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93843537219312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.190+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 68 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 93843537219312] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.190+0000 7fce35b36800 20 remove_watcher() i=1 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.190+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93843537220368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b2ed40 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.191+0000 7fce320cd640 1 -- 172.21.15.63:0/970096184 <== osd.3 v2:172.21.15.63:6816/1289558434 98 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 93843537220368] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2c029070 con 0x5559a6b2ed40 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.191+0000 7fce35b36800 20 remove_watcher() i=2 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.191+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93843537226560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2c021bf0 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.192+0000 7fce318cc640 1 -- 172.21.15.63:0/970096184 <== osd.2 v2:172.21.15.63:6808/753864785 65 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 93843537226560] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce24097030 con 0x7fce2c021bf0 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.192+0000 7fce35b36800 20 remove_watcher() i=3 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.192+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93843537224992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x5559a6b28af0 2024-04-23T19:13:23.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.194+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.1 v2:172.21.15.63:6800/3216879695 69 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 93843537224992] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x5559a6b28af0 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.194+0000 7fce35b36800 20 remove_watcher() i=4 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.194+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93843537234928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.195+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 68 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 93843537234928] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.195+0000 7fce35b36800 20 remove_watcher() i=5 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.195+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93843537228688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.196+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 69 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 93843537228688] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.196+0000 7fce35b36800 20 remove_watcher() i=6 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.196+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93843537237184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32) -- 0x5559a6b72540 con 0x7fce2409b040 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 <== osd.0 v2:172.21.15.16:6800/2262294208 70 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 93843537237184] v32'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce2802a040 con 0x7fce2409b040 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 20 remove_watcher() i=7 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fce2409b040 msgr2=0x7fce24076360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7fce2409b040 0x7fce24076360 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a6b28af0 msgr2=0x5559a6b2afb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a6b28af0 0x5559a6b2afb0 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fce2c021bf0 msgr2=0x7fce2c02b860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7fce2c021bf0 0x7fce2c02b860 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559a6b2ed40 msgr2=0x5559a6b31150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x5559a6b2ed40 0x5559a6b31150 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fccfc036eb0 msgr2=0x7fccfc039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fccfc036eb0 0x7fccfc039370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce2408e010 tx=0x7fce24088040 comp rx=0 tx=0).stop 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 msgr2=0x5559a6b1ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce310cb640 1 -- 172.21.15.63:0/970096184 reap_dead start 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b1ceb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fce2c04cba0 tx=0x7fce2c04cbd0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 shutdown_connections 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 --2- 172.21.15.63:0/970096184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b159a0 0x5559a6b1ceb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 >> 172.21.15.63:0/970096184 conn(0x5559a6b11d50 msgr2=0x5559a6b171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 shutdown_connections 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.197+0000 7fce35b36800 1 -- 172.21.15.63:0/970096184 wait complete. 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd0c036eb0 msgr2=0x7fcd0c039370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd0c036eb0 0x7fcd0c039370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5559a6af5c30 tx=0x7fce2408cb70 comp rx=0 tx=0).stop 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b086b0 msgr2=0x5559a6b08ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b086b0 0x5559a6b08ad0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x5559a6aee300 tx=0x7fce2c0178e0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 shutdown_connections 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd0c036eb0 0x7fcd0c039370 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/696275859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a6b086b0 0x5559a6b08ad0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 >> 172.21.15.63:0/696275859 conn(0x5559a6afc750 msgr2=0x5559a6afcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 shutdown_connections 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/696275859 wait complete. 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559a6af7900 msgr2=0x5559a6af9cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559a6af7900 0x5559a6af9cf0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a69f54d0 msgr2=0x5559a6af4140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a69f54d0 0x5559a6af4140 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd18036e60 msgr2=0x7fcd18039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd18036e60 0x7fcd18039320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce28000f60 tx=0x7fce280012b0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 msgr2=0x5559a6aecf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a6aecf00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fce24052740 tx=0x7fce24003020 comp rx=0 tx=0).stop 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 shutdown_connections 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x5559a6af7900 0x5559a6af9cf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x5559a69f54d0 0x5559a6af4140 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd18036e60 0x7fcd18039320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 --2- 172.21.15.63:0/658430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559a69f2ba0 0x5559a6aecf00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 >> 172.21.15.63:0/658430635 conn(0x5559a69ec630 msgr2=0x5559a6ade820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 shutdown_connections 2024-04-23T19:13:23.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-04-23T19:13:23.198+0000 7fce35b36800 1 -- 172.21.15.63:0/658430635 wait complete. 2024-04-23T19:13:23.210 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-23T19:13:23.221 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T19:13:23.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:13:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.322+0000 7f78dc35f640 1 Processor -- start 2024-04-23T19:13:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.322+0000 7f78dc35f640 1 -- start start 2024-04-23T19:13:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.322+0000 7f78dc35f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78d412ec80 0x7f78d4131070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.323+0000 7f78dc35f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d4128410 con 0x7f78d41288d0 2024-04-23T19:13:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.323+0000 7f78dc35f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f78d41286b0 con 0x7f78d412ec80 2024-04-23T19:13:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.323+0000 7f78d98fa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78d412ec80 0x7f78d4131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.323+0000 7f78da0fb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78d41288d0 0x7f78d4128cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38522/0 (socket says 172.21.15.16:38522) 2024-04-23T19:13:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.323+0000 7f78da0fb640 1 -- 172.21.15.16:0/3979852 learned_addr learned my addr 172.21.15.16:0/3979852 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.323+0000 7f78d98fa640 1 -- 172.21.15.16:0/3979852 >> v1:172.21.15.16:6789/0 conn(0x7f78d41288d0 legacy=0x7f78d4128cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.323+0000 7f78d98fa640 1 -- 172.21.15.16:0/3979852 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d406de40 con 0x7f78d412ec80 2024-04-23T19:13:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78d98fa640 1 --2- 172.21.15.16:0/3979852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d412ec80 0x7f78d4131070 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009720 tx=0x7f78c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e5deb77d6ca089 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78d90f9640 1 -- 172.21.15.16:0/3979852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78c80316d0 con 0x7f78d412ec80 2024-04-23T19:13:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78d90f9640 1 -- 172.21.15.16:0/3979852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c8037070 con 0x7f78d412ec80 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78d90f9640 1 -- 172.21.15.16:0/3979852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78c80319c0 con 0x7f78d412ec80 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78dc35f640 1 -- 172.21.15.16:0/3979852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d412ec80 msgr2=0x7f78d4131070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78dc35f640 1 --2- 172.21.15.16:0/3979852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d412ec80 0x7f78d4131070 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009720 tx=0x7f78c802b840 comp rx=0 tx=0).stop 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78dc35f640 1 -- 172.21.15.16:0/3979852 shutdown_connections 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78dc35f640 1 --2- 172.21.15.16:0/3979852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d412ec80 0x7f78d4131070 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.324+0000 7f78dc35f640 1 -- 172.21.15.16:0/3979852 >> 172.21.15.16:0/3979852 conn(0x7f78d409a200 msgr2=0x7f78d4097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.325+0000 7f78dc35f640 1 -- 172.21.15.16:0/3979852 shutdown_connections 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.325+0000 7f78dc35f640 1 -- 172.21.15.16:0/3979852 wait complete. 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.325+0000 7f78dc35f640 1 Processor -- start 2024-04-23T19:13:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.325+0000 7f78dc35f640 1 -- start start 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78dc35f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d41288d0 0x7f78d4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78dc35f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78d40945f0 con 0x7f78d41288d0 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78da0fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d41288d0 0x7f78d4070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78da0fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d41288d0 0x7f78d4070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38964/0 (socket says 172.21.15.16:38964) 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78da0fb640 1 -- 172.21.15.16:0/1282355324 learned_addr learned my addr 172.21.15.16:0/1282355324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78da0fb640 1 -- 172.21.15.16:0/1282355324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d4129df0 con 0x7f78d41288d0 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78da0fb640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d41288d0 0x7f78d4070f00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c4a0 tx=0x7f78cc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78d2ffd640 1 -- 172.21.15.16:0/1282355324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78cc017070 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78d2ffd640 1 -- 172.21.15.16:0/1282355324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78cc005c10 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d41298c0 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78d2ffd640 1 -- 172.21.15.16:0/1282355324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78cc007070 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.326+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d4071620 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.327+0000 7f78d2ffd640 1 -- 172.21.15.16:0/1282355324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f78cc01a070 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.327+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d412a0d0 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.327+0000 7f78d2ffd640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f78c00385c0 0x7f78c003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.328+0000 7f78d2ffd640 1 -- 172.21.15.16:0/1282355324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f78cc04b440 con 0x7f78d41288d0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.328+0000 7f78d98fa640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f78c00385c0 0x7f78c003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.328+0000 7f78d98fa640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f78c00385c0 0x7f78c003aa80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f78c8036000 tx=0x7f78c8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.332+0000 7f78d2ffd640 1 -- 172.21.15.16:0/1282355324 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78d412a0d0 con 0x7f78d41288d0 2024-04-23T19:13:23.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.514+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78d4072ab0 con 0x7f78c00385c0 2024-04-23T19:13:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.530+0000 7f78d2ffd640 1 -- 172.21.15.16:0/1282355324 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665664 (secure 0 0 0) 0x7f78d4072ab0 con 0x7f78c00385c0 2024-04-23T19:13:23.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:13:23.551 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:13:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f78c00385c0 msgr2=0x7f78c003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f78c00385c0 0x7f78c003aa80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f78c8036000 tx=0x7f78c8008030 comp rx=0 tx=0).stop 2024-04-23T19:13:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d41288d0 msgr2=0x7f78d4070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d41288d0 0x7f78d4070f00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c4a0 tx=0x7f78cc00c960 comp rx=0 tx=0).stop 2024-04-23T19:13:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 shutdown_connections 2024-04-23T19:13:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f78c00385c0 0x7f78c003aa80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 --2- 172.21.15.16:0/1282355324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d41288d0 0x7f78d4070f00 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 >> 172.21.15.16:0/1282355324 conn(0x7f78d409a200 msgr2=0x7f78d4126c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 shutdown_connections 2024-04-23T19:13:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.535+0000 7f78dc35f640 1 -- 172.21.15.16:0/1282355324 wait complete. 2024-04-23T19:13:23.557 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-04-23T19:13:22.700633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412789,"num_objects":227,"num_object_clones":0,"num_object_copies":461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":4876,"num_read_kb":4453,"num_write":2104,"num_write_kb":61372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4476,"ondisk_log_size":4476,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":177844,"kb_used_data":40660,"kb_used_omap":6,"kb_used_meta":137145,"kb_avail":377309516,"statfs":{"total":386547056640,"available":386364944384,"internally_reserved":0,"allocated":41635840,"data_stored":40478942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":140437271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811687,"num_objects":9,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003607"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722031+0000","last_change":"2024-04-23T19:10:07.492726+0000","last_active":"2024-04-23T19:10:47.722031+0000","last_peered":"2024-04-23T19:10:47.722031+0000","last_clean":"2024-04-23T19:10:47.722031+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:10:47.722031+0000","last_undegraded":"2024-04-23T19:10:47.722031+0000","last_fullsized":"2024-04-23T19:10:47.722031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771923+0000","last_change":"2024-04-23T19:10:09.485943+0000","last_active":"2024-04-23T19:12:15.771923+0000","last_peered":"2024-04-23T19:12:15.771923+0000","last_clean":"2024-04-23T19:12:15.771923+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:12:15.771923+0000","last_undegraded":"2024-04-23T19:12:15.771923+0000","last_fullsized":"2024-04-23T19:12:15.771923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729774+0000","last_change":"2024-04-23T19:10:07.490929+0000","last_active":"2024-04-23T19:10:47.729774+0000","last_peered":"2024-04-23T19:10:47.729774+0000","last_clean":"2024-04-23T19:10:47.729774+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:10:47.729774+0000","last_undegraded":"2024-04-23T19:10:47.729774+0000","last_fullsized":"2024-04-23T19:10:47.729774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:43.016228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.498665+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:54.763728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729513+0000","last_change":"2024-04-23T19:10:07.508341+0000","last_active":"2024-04-23T19:10:47.729513+0000","last_peered":"2024-04-23T19:10:47.729513+0000","last_clean":"2024-04-23T19:10:47.729513+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:10:47.729513+0000","last_undegraded":"2024-04-23T19:10:47.729513+0000","last_fullsized":"2024-04-23T19:10:47.729513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:50.865140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771899+0000","last_change":"2024-04-23T19:10:09.498971+0000","last_active":"2024-04-23T19:12:15.771899+0000","last_peered":"2024-04-23T19:12:15.771899+0000","last_clean":"2024-04-23T19:12:15.771899+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:12:15.771899+0000","last_undegraded":"2024-04-23T19:12:15.771899+0000","last_fullsized":"2024-04-23T19:12:15.771899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:58:56.677184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719199+0000","last_change":"2024-04-23T19:10:07.490859+0000","last_active":"2024-04-23T19:10:47.719199+0000","last_peered":"2024-04-23T19:10:47.719199+0000","last_clean":"2024-04-23T19:10:47.719199+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:10:47.719199+0000","last_undegraded":"2024-04-23T19:10:47.719199+0000","last_fullsized":"2024-04-23T19:10:47.719199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447391+0000","last_change":"2024-04-23T19:10:09.489556+0000","last_active":"2024-04-23T19:13:16.447391+0000","last_peered":"2024-04-23T19:13:16.447391+0000","last_clean":"2024-04-23T19:13:16.447391+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:13:16.447391+0000","last_undegraded":"2024-04-23T19:13:16.447391+0000","last_fullsized":"2024-04-23T19:13:16.447391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:55.937097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.675532+0000","last_change":"2024-04-23T19:10:07.504507+0000","last_active":"2024-04-23T19:12:15.675532+0000","last_peered":"2024-04-23T19:12:15.675532+0000","last_clean":"2024-04-23T19:12:15.675532+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:12:15.675532+0000","last_undegraded":"2024-04-23T19:12:15.675532+0000","last_fullsized":"2024-04-23T19:12:15.675532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:05.672303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.764033+0000","last_change":"2024-04-23T19:10:09.482365+0000","last_active":"2024-04-23T19:12:15.764033+0000","last_peered":"2024-04-23T19:12:15.764033+0000","last_clean":"2024-04-23T19:12:15.764033+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:12:15.764033+0000","last_undegraded":"2024-04-23T19:12:15.764033+0000","last_fullsized":"2024-04-23T19:12:15.764033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719720+0000","last_change":"2024-04-23T19:10:07.490892+0000","last_active":"2024-04-23T19:12:40.719720+0000","last_peered":"2024-04-23T19:12:40.719720+0000","last_clean":"2024-04-23T19:12:40.719720+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:12:40.719720+0000","last_undegraded":"2024-04-23T19:12:40.719720+0000","last_fullsized":"2024-04-23T19:12:40.719720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258616+0000","last_change":"2024-04-23T19:10:09.499345+0000","last_active":"2024-04-23T19:13:16.258616+0000","last_peered":"2024-04-23T19:13:16.258616+0000","last_clean":"2024-04-23T19:13:16.258616+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:16.258616+0000","last_undegraded":"2024-04-23T19:13:16.258616+0000","last_fullsized":"2024-04-23T19:13:16.258616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:28.869634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724671+0000","last_change":"2024-04-23T19:10:07.489309+0000","last_active":"2024-04-23T19:10:47.724671+0000","last_peered":"2024-04-23T19:10:47.724671+0000","last_clean":"2024-04-23T19:10:47.724671+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:10:47.724671+0000","last_undegraded":"2024-04-23T19:10:47.724671+0000","last_fullsized":"2024-04-23T19:10:47.724671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:18.611845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'23","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458768+0000","last_change":"2024-04-23T19:10:09.492130+0000","last_active":"2024-04-23T19:13:16.458768+0000","last_peered":"2024-04-23T19:13:16.458768+0000","last_clean":"2024-04-23T19:13:16.458768+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:16.458768+0000","last_undegraded":"2024-04-23T19:13:16.458768+0000","last_fullsized":"2024-04-23T19:13:16.458768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.714552+0000","last_change":"2024-04-23T19:10:07.487170+0000","last_active":"2024-04-23T19:12:40.714552+0000","last_peered":"2024-04-23T19:12:40.714552+0000","last_clean":"2024-04-23T19:12:40.714552+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:12:40.714552+0000","last_undegraded":"2024-04-23T19:12:40.714552+0000","last_fullsized":"2024-04-23T19:12:40.714552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'17","reported_seq":72,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458912+0000","last_change":"2024-04-23T19:10:09.493592+0000","last_active":"2024-04-23T19:13:16.458912+0000","last_peered":"2024-04-23T19:13:16.458912+0000","last_clean":"2024-04-23T19:13:16.458912+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:13:16.458912+0000","last_undegraded":"2024-04-23T19:13:16.458912+0000","last_fullsized":"2024-04-23T19:13:16.458912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727109+0000","last_change":"2024-04-23T19:10:09.499521+0000","last_active":"2024-04-23T19:10:47.727109+0000","last_peered":"2024-04-23T19:10:47.727109+0000","last_clean":"2024-04-23T19:10:47.727109+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:10:47.727109+0000","last_undegraded":"2024-04-23T19:10:47.727109+0000","last_fullsized":"2024-04-23T19:10:47.727109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:05.408478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722404+0000","last_change":"2024-04-23T19:10:07.491914+0000","last_active":"2024-04-23T19:10:47.722404+0000","last_peered":"2024-04-23T19:10:47.722404+0000","last_clean":"2024-04-23T19:10:47.722404+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:10:47.722404+0000","last_undegraded":"2024-04-23T19:10:47.722404+0000","last_fullsized":"2024-04-23T19:10:47.722404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449380+0000","last_change":"2024-04-23T19:10:09.482719+0000","last_active":"2024-04-23T19:13:16.449380+0000","last_peered":"2024-04-23T19:13:16.449380+0000","last_clean":"2024-04-23T19:13:16.449380+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:13:16.449380+0000","last_undegraded":"2024-04-23T19:13:16.449380+0000","last_fullsized":"2024-04-23T19:13:16.449380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725054+0000","last_change":"2024-04-23T19:10:07.489933+0000","last_active":"2024-04-23T19:10:47.725054+0000","last_peered":"2024-04-23T19:10:47.725054+0000","last_clean":"2024-04-23T19:10:47.725054+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:10:47.725054+0000","last_undegraded":"2024-04-23T19:10:47.725054+0000","last_fullsized":"2024-04-23T19:10:47.725054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720219+0000","last_change":"2024-04-23T19:10:09.484959+0000","last_active":"2024-04-23T19:10:47.720219+0000","last_peered":"2024-04-23T19:10:47.720219+0000","last_clean":"2024-04-23T19:10:47.720219+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:10:47.720219+0000","last_undegraded":"2024-04-23T19:10:47.720219+0000","last_fullsized":"2024-04-23T19:10:47.720219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:54.767051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720449+0000","last_change":"2024-04-23T19:10:07.493585+0000","last_active":"2024-04-23T19:12:31.720449+0000","last_peered":"2024-04-23T19:12:31.720449+0000","last_clean":"2024-04-23T19:12:31.720449+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:12:31.720449+0000","last_undegraded":"2024-04-23T19:12:31.720449+0000","last_fullsized":"2024-04-23T19:12:31.720449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408189+0000","last_change":"2024-04-23T19:10:09.491221+0000","last_active":"2024-04-23T19:13:16.408189+0000","last_peered":"2024-04-23T19:13:16.408189+0000","last_clean":"2024-04-23T19:13:16.408189+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:16.408189+0000","last_undegraded":"2024-04-23T19:13:16.408189+0000","last_fullsized":"2024-04-23T19:13:16.408189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718838+0000","last_change":"2024-04-23T19:10:07.501878+0000","last_active":"2024-04-23T19:10:47.718838+0000","last_peered":"2024-04-23T19:10:47.718838+0000","last_clean":"2024-04-23T19:10:47.718838+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:10:47.718838+0000","last_undegraded":"2024-04-23T19:10:47.718838+0000","last_fullsized":"2024-04-23T19:10:47.718838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771581+0000","last_change":"2024-04-23T19:10:09.492916+0000","last_active":"2024-04-23T19:12:15.771581+0000","last_peered":"2024-04-23T19:12:15.771581+0000","last_clean":"2024-04-23T19:12:15.771581+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:12:15.771581+0000","last_undegraded":"2024-04-23T19:12:15.771581+0000","last_fullsized":"2024-04-23T19:12:15.771581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728570+0000","last_change":"2024-04-23T19:10:07.490910+0000","last_active":"2024-04-23T19:10:47.728570+0000","last_peered":"2024-04-23T19:10:47.728570+0000","last_clean":"2024-04-23T19:10:47.728570+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:10:47.728570+0000","last_undegraded":"2024-04-23T19:10:47.728570+0000","last_fullsized":"2024-04-23T19:10:47.728570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482419+0000","last_change":"2024-04-23T19:10:09.482527+0000","last_active":"2024-04-23T19:13:16.482419+0000","last_peered":"2024-04-23T19:13:16.482419+0000","last_clean":"2024-04-23T19:13:16.482419+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:16.482419+0000","last_undegraded":"2024-04-23T19:13:16.482419+0000","last_fullsized":"2024-04-23T19:13:16.482419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:46.899076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727215+0000","last_change":"2024-04-23T19:10:07.493956+0000","last_active":"2024-04-23T19:10:47.727215+0000","last_peered":"2024-04-23T19:10:47.727215+0000","last_clean":"2024-04-23T19:10:47.727215+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:10:47.727215+0000","last_undegraded":"2024-04-23T19:10:47.727215+0000","last_fullsized":"2024-04-23T19:10:47.727215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:20.896254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'35","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.422839+0000","last_change":"2024-04-23T19:10:17.514587+0000","last_active":"2024-04-23T19:13:16.422839+0000","last_peered":"2024-04-23T19:13:16.422839+0000","last_clean":"2024-04-23T19:13:16.422839+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:16.422839+0000","last_undegraded":"2024-04-23T19:13:16.422839+0000","last_fullsized":"2024-04-23T19:13:16.422839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:18.256132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.432503+0000","last_change":"2024-04-23T19:10:09.487433+0000","last_active":"2024-04-23T19:10:49.432503+0000","last_peered":"2024-04-23T19:10:49.432503+0000","last_clean":"2024-04-23T19:10:49.432503+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:10:49.432503+0000","last_undegraded":"2024-04-23T19:10:49.432503+0000","last_fullsized":"2024-04-23T19:10:49.432503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:36.758021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728416+0000","last_change":"2024-04-23T19:10:07.507346+0000","last_active":"2024-04-23T19:10:47.728416+0000","last_peered":"2024-04-23T19:10:47.728416+0000","last_clean":"2024-04-23T19:10:47.728416+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:10:47.728416+0000","last_undegraded":"2024-04-23T19:10:47.728416+0000","last_fullsized":"2024-04-23T19:10:47.728416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:42.849025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728325+0000","last_change":"2024-04-23T19:10:11.488429+0000","last_active":"2024-04-23T19:10:47.728325+0000","last_peered":"2024-04-23T19:10:47.728325+0000","last_clean":"2024-04-23T19:10:47.728325+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:10:47.728325+0000","last_undegraded":"2024-04-23T19:10:47.728325+0000","last_fullsized":"2024-04-23T19:10:47.728325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:50.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722656+0000","last_change":"2024-04-23T19:10:13.501588+0000","last_active":"2024-04-23T19:12:41.722656+0000","last_peered":"2024-04-23T19:12:41.722656+0000","last_clean":"2024-04-23T19:12:41.722656+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:12:41.722656+0000","last_undegraded":"2024-04-23T19:12:41.722656+0000","last_fullsized":"2024-04-23T19:12:41.722656+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:34.948969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722273+0000","last_change":"2024-04-23T19:10:15.509911+0000","last_active":"2024-04-23T19:10:47.722273+0000","last_peered":"2024-04-23T19:10:47.722273+0000","last_clean":"2024-04-23T19:10:47.722273+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:10:47.722273+0000","last_undegraded":"2024-04-23T19:10:47.722273+0000","last_fullsized":"2024-04-23T19:10:47.722273+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'9","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.539127+0000","last_change":"2024-04-23T19:10:17.514347+0000","last_active":"2024-04-23T19:13:16.539127+0000","last_peered":"2024-04-23T19:13:16.539127+0000","last_clean":"2024-04-23T19:13:16.539127+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:16.539127+0000","last_undegraded":"2024-04-23T19:13:16.539127+0000","last_fullsized":"2024-04-23T19:13:16.539127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:53.299426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399880+0000","last_change":"2024-04-23T19:10:09.496014+0000","last_active":"2024-04-23T19:13:16.399880+0000","last_peered":"2024-04-23T19:13:16.399880+0000","last_clean":"2024-04-23T19:13:16.399880+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:13:16.399880+0000","last_undegraded":"2024-04-23T19:13:16.399880+0000","last_fullsized":"2024-04-23T19:13:16.399880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:32.575973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720263+0000","last_change":"2024-04-23T19:10:07.492730+0000","last_active":"2024-04-23T19:10:47.720263+0000","last_peered":"2024-04-23T19:10:47.720263+0000","last_clean":"2024-04-23T19:10:47.720263+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:10:47.720263+0000","last_undegraded":"2024-04-23T19:10:47.720263+0000","last_fullsized":"2024-04-23T19:10:47.720263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"23'1","reported_seq":218,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.386033+0000","last_change":"2024-04-23T19:10:11.489225+0000","last_active":"2024-04-23T19:12:15.386033+0000","last_peered":"2024-04-23T19:12:15.386033+0000","last_clean":"2024-04-23T19:12:15.386033+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:12:15.386033+0000","last_undegraded":"2024-04-23T19:12:15.386033+0000","last_fullsized":"2024-04-23T19:12:15.386033+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:01.595709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"30'6","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721580+0000","last_change":"2024-04-23T19:10:13.491509+0000","last_active":"2024-04-23T19:10:47.721580+0000","last_peered":"2024-04-23T19:10:47.721580+0000","last_clean":"2024-04-23T19:10:47.721580+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:10:47.721580+0000","last_undegraded":"2024-04-23T19:10:47.721580+0000","last_fullsized":"2024-04-23T19:10:47.721580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:32.880512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720747+0000","last_change":"2024-04-23T19:10:15.506746+0000","last_active":"2024-04-23T19:10:47.720747+0000","last_peered":"2024-04-23T19:10:47.720747+0000","last_clean":"2024-04-23T19:10:47.720747+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:10:47.720747+0000","last_undegraded":"2024-04-23T19:10:47.720747+0000","last_fullsized":"2024-04-23T19:10:47.720747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726990+0000","last_change":"2024-04-23T19:10:17.518249+0000","last_active":"2024-04-23T19:10:47.726990+0000","last_peered":"2024-04-23T19:10:47.726990+0000","last_clean":"2024-04-23T19:10:47.726990+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:10:47.726990+0000","last_undegraded":"2024-04-23T19:10:47.726990+0000","last_fullsized":"2024-04-23T19:10:47.726990+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:58.051610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447281+0000","last_change":"2024-04-23T19:10:09.490234+0000","last_active":"2024-04-23T19:13:16.447281+0000","last_peered":"2024-04-23T19:13:16.447281+0000","last_clean":"2024-04-23T19:13:16.447281+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:13:16.447281+0000","last_undegraded":"2024-04-23T19:13:16.447281+0000","last_fullsized":"2024-04-23T19:13:16.447281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:54.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.389417+0000","last_change":"2024-04-23T19:10:07.504260+0000","last_active":"2024-04-23T19:13:16.389417+0000","last_peered":"2024-04-23T19:13:16.389417+0000","last_clean":"2024-04-23T19:13:16.389417+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:13:16.389417+0000","last_undegraded":"2024-04-23T19:13:16.389417+0000","last_fullsized":"2024-04-23T19:13:16.389417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:46.596307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730636+0000","last_change":"2024-04-23T19:10:11.493566+0000","last_active":"2024-04-23T19:10:47.730636+0000","last_peered":"2024-04-23T19:10:47.730636+0000","last_clean":"2024-04-23T19:10:47.730636+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:10:47.730636+0000","last_undegraded":"2024-04-23T19:10:47.730636+0000","last_fullsized":"2024-04-23T19:10:47.730636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718899+0000","last_change":"2024-04-23T19:10:13.502669+0000","last_active":"2024-04-23T19:12:35.718899+0000","last_peered":"2024-04-23T19:12:35.718899+0000","last_clean":"2024-04-23T19:12:35.718899+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:12:35.718899+0000","last_undegraded":"2024-04-23T19:12:35.718899+0000","last_fullsized":"2024-04-23T19:12:35.718899+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:02.466667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'282","reported_seq":385,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.598276+0000","last_change":"2024-04-23T19:10:15.515398+0000","last_active":"2024-04-23T19:13:16.598276+0000","last_peered":"2024-04-23T19:13:16.598276+0000","last_clean":"2024-04-23T19:13:16.598276+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:13:16.598276+0000","last_undegraded":"2024-04-23T19:13:16.598276+0000","last_fullsized":"2024-04-23T19:13:16.598276+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.433967+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:10:49.433967+0000","last_peered":"2024-04-23T19:10:49.433967+0000","last_clean":"2024-04-23T19:10:49.433967+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:10:49.433967+0000","last_undegraded":"2024-04-23T19:10:49.433967+0000","last_fullsized":"2024-04-23T19:10:49.433967+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:55.634563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482380+0000","last_change":"2024-04-23T19:10:09.487095+0000","last_active":"2024-04-23T19:13:16.482380+0000","last_peered":"2024-04-23T19:13:16.482380+0000","last_clean":"2024-04-23T19:13:16.482380+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:13:16.482380+0000","last_undegraded":"2024-04-23T19:13:16.482380+0000","last_fullsized":"2024-04-23T19:13:16.482380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716719+0000","last_change":"2024-04-23T19:10:07.504154+0000","last_active":"2024-04-23T19:10:47.716719+0000","last_peered":"2024-04-23T19:10:47.716719+0000","last_clean":"2024-04-23T19:10:47.716719+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:10:47.716719+0000","last_undegraded":"2024-04-23T19:10:47.716719+0000","last_fullsized":"2024-04-23T19:10:47.716719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:24.070928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728932+0000","last_change":"2024-04-23T19:10:11.488228+0000","last_active":"2024-04-23T19:10:47.728932+0000","last_peered":"2024-04-23T19:10:47.728932+0000","last_clean":"2024-04-23T19:10:47.728932+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:10:47.728932+0000","last_undegraded":"2024-04-23T19:10:47.728932+0000","last_fullsized":"2024-04-23T19:10:47.728932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:01.154683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719829+0000","last_change":"2024-04-23T19:10:13.505813+0000","last_active":"2024-04-23T19:12:40.719829+0000","last_peered":"2024-04-23T19:12:40.719829+0000","last_clean":"2024-04-23T19:12:40.719829+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:12:40.719829+0000","last_undegraded":"2024-04-23T19:12:40.719829+0000","last_fullsized":"2024-04-23T19:12:40.719829+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:39.996179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'282","reported_seq":401,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575901+0000","last_change":"2024-04-23T19:10:15.500511+0000","last_active":"2024-04-23T19:13:16.575901+0000","last_peered":"2024-04-23T19:13:16.575901+0000","last_clean":"2024-04-23T19:13:16.575901+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:16.575901+0000","last_undegraded":"2024-04-23T19:13:16.575901+0000","last_fullsized":"2024-04-23T19:13:16.575901+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:23.813091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449319+0000","last_change":"2024-04-23T19:10:47.730482+0000","last_active":"2024-04-23T19:13:16.449319+0000","last_peered":"2024-04-23T19:13:16.449319+0000","last_clean":"2024-04-23T19:13:16.449319+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:13:16.449319+0000","last_undegraded":"2024-04-23T19:13:16.449319+0000","last_fullsized":"2024-04-23T19:13:16.449319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:33.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545310+0000","last_change":"2024-04-23T19:10:07.492480+0000","last_active":"2024-04-23T19:10:48.545310+0000","last_peered":"2024-04-23T19:10:48.545310+0000","last_clean":"2024-04-23T19:10:48.545310+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:10:48.545310+0000","last_undegraded":"2024-04-23T19:10:48.545310+0000","last_fullsized":"2024-04-23T19:10:48.545310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716980+0000","last_change":"2024-04-23T19:10:11.490040+0000","last_active":"2024-04-23T19:10:47.716980+0000","last_peered":"2024-04-23T19:10:47.716980+0000","last_clean":"2024-04-23T19:10:47.716980+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:10:47.716980+0000","last_undegraded":"2024-04-23T19:10:47.716980+0000","last_fullsized":"2024-04-23T19:10:47.716980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:07.412668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'40","reported_seq":87,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.316535+0000","last_change":"2024-04-23T19:10:13.494728+0000","last_active":"2024-04-23T19:12:14.316535+0000","last_peered":"2024-04-23T19:12:14.316535+0000","last_clean":"2024-04-23T19:12:14.316535+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:12:14.316535+0000","last_undegraded":"2024-04-23T19:12:14.316535+0000","last_fullsized":"2024-04-23T19:12:14.316535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:56.304108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728369+0000","last_change":"2024-04-23T19:10:15.506637+0000","last_active":"2024-04-23T19:10:47.728369+0000","last_peered":"2024-04-23T19:10:47.728369+0000","last_clean":"2024-04-23T19:10:47.728369+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:10:47.728369+0000","last_undegraded":"2024-04-23T19:10:47.728369+0000","last_fullsized":"2024-04-23T19:10:47.728369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:13.464606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.971818+0000","last_change":"2024-04-23T19:10:17.519710+0000","last_active":"2024-04-23T19:11:31.971818+0000","last_peered":"2024-04-23T19:11:31.971818+0000","last_clean":"2024-04-23T19:11:31.971818+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:11:31.971818+0000","last_undegraded":"2024-04-23T19:11:31.971818+0000","last_fullsized":"2024-04-23T19:11:31.971818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:10.239585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'7","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.488606+0000","last_change":"2024-04-23T19:10:09.482665+0000","last_active":"2024-04-23T19:13:16.488606+0000","last_peered":"2024-04-23T19:13:16.488606+0000","last_clean":"2024-04-23T19:13:16.488606+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:16.488606+0000","last_undegraded":"2024-04-23T19:13:16.488606+0000","last_fullsized":"2024-04-23T19:13:16.488606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724246+0000","last_change":"2024-04-23T19:10:07.500938+0000","last_active":"2024-04-23T19:10:47.724246+0000","last_peered":"2024-04-23T19:10:47.724246+0000","last_clean":"2024-04-23T19:10:47.724246+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:10:47.724246+0000","last_undegraded":"2024-04-23T19:10:47.724246+0000","last_fullsized":"2024-04-23T19:10:47.724246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727117+0000","last_change":"2024-04-23T19:10:11.493617+0000","last_active":"2024-04-23T19:10:47.727117+0000","last_peered":"2024-04-23T19:10:47.727117+0000","last_clean":"2024-04-23T19:10:47.727117+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:10:47.727117+0000","last_undegraded":"2024-04-23T19:10:47.727117+0000","last_fullsized":"2024-04-23T19:10:47.727117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:32.061098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.296627+0000","last_change":"2024-04-23T19:10:13.501851+0000","last_active":"2024-04-23T19:12:14.296627+0000","last_peered":"2024-04-23T19:12:14.296627+0000","last_clean":"2024-04-23T19:12:14.296627+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:12:14.296627+0000","last_undegraded":"2024-04-23T19:12:14.296627+0000","last_fullsized":"2024-04-23T19:12:14.296627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:30.608779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727229+0000","last_change":"2024-04-23T19:10:15.502245+0000","last_active":"2024-04-23T19:10:47.727229+0000","last_peered":"2024-04-23T19:10:47.727229+0000","last_clean":"2024-04-23T19:10:47.727229+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:10:47.727229+0000","last_undegraded":"2024-04-23T19:10:47.727229+0000","last_fullsized":"2024-04-23T19:10:47.727229+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'5","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.261614+0000","last_change":"2024-04-23T19:10:17.518314+0000","last_active":"2024-04-23T19:13:16.261614+0000","last_peered":"2024-04-23T19:13:16.261614+0000","last_clean":"2024-04-23T19:13:16.261614+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:13:16.261614+0000","last_undegraded":"2024-04-23T19:13:16.261614+0000","last_fullsized":"2024-04-23T19:13:16.261614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:11.577457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.166247+0000","last_change":"2024-04-23T19:10:09.492229+0000","last_active":"2024-04-23T19:13:16.166247+0000","last_peered":"2024-04-23T19:13:16.166247+0000","last_clean":"2024-04-23T19:13:16.166247+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:13:16.166247+0000","last_undegraded":"2024-04-23T19:13:16.166247+0000","last_fullsized":"2024-04-23T19:13:16.166247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:36.861914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729281+0000","last_change":"2024-04-23T19:10:07.492637+0000","last_active":"2024-04-23T19:10:47.729281+0000","last_peered":"2024-04-23T19:10:47.729281+0000","last_clean":"2024-04-23T19:10:47.729281+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:10:47.729281+0000","last_undegraded":"2024-04-23T19:10:47.729281+0000","last_fullsized":"2024-04-23T19:10:47.729281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.154249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729542+0000","last_change":"2024-04-23T19:10:11.491503+0000","last_active":"2024-04-23T19:10:47.729542+0000","last_peered":"2024-04-23T19:10:47.729542+0000","last_clean":"2024-04-23T19:10:47.729542+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:10:47.729542+0000","last_undegraded":"2024-04-23T19:10:47.729542+0000","last_fullsized":"2024-04-23T19:10:47.729542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.318445+0000","last_change":"2024-04-23T19:10:13.508355+0000","last_active":"2024-04-23T19:12:14.318445+0000","last_peered":"2024-04-23T19:12:14.318445+0000","last_clean":"2024-04-23T19:12:14.318445+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:12:14.318445+0000","last_undegraded":"2024-04-23T19:12:14.318445+0000","last_fullsized":"2024-04-23T19:12:14.318445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:19.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'282","reported_seq":348,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.608082+0000","last_change":"2024-04-23T19:10:15.509684+0000","last_active":"2024-04-23T19:13:16.608082+0000","last_peered":"2024-04-23T19:13:16.608082+0000","last_clean":"2024-04-23T19:13:16.608082+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:16.608082+0000","last_undegraded":"2024-04-23T19:13:16.608082+0000","last_fullsized":"2024-04-23T19:13:16.608082+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728722+0000","last_change":"2024-04-23T19:10:17.520791+0000","last_active":"2024-04-23T19:10:47.728722+0000","last_peered":"2024-04-23T19:10:47.728722+0000","last_clean":"2024-04-23T19:10:47.728722+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:10:47.728722+0000","last_undegraded":"2024-04-23T19:10:47.728722+0000","last_fullsized":"2024-04-23T19:10:47.728722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:03.997750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399785+0000","last_change":"2024-04-23T19:10:09.476300+0000","last_active":"2024-04-23T19:13:16.399785+0000","last_peered":"2024-04-23T19:13:16.399785+0000","last_clean":"2024-04-23T19:13:16.399785+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:16.399785+0000","last_undegraded":"2024-04-23T19:13:16.399785+0000","last_fullsized":"2024-04-23T19:13:16.399785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:59.576302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726343+0000","last_change":"2024-04-23T19:10:07.501083+0000","last_active":"2024-04-23T19:10:47.726343+0000","last_peered":"2024-04-23T19:10:47.726343+0000","last_clean":"2024-04-23T19:10:47.726343+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:10:47.726343+0000","last_undegraded":"2024-04-23T19:10:47.726343+0000","last_fullsized":"2024-04-23T19:10:47.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:08.925701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727624+0000","last_change":"2024-04-23T19:10:11.493665+0000","last_active":"2024-04-23T19:10:47.727624+0000","last_peered":"2024-04-23T19:10:47.727624+0000","last_clean":"2024-04-23T19:10:47.727624+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:10:47.727624+0000","last_undegraded":"2024-04-23T19:10:47.727624+0000","last_fullsized":"2024-04-23T19:10:47.727624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:29.762026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'54","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.320262+0000","last_change":"2024-04-23T19:10:13.499017+0000","last_active":"2024-04-23T19:12:14.320262+0000","last_peered":"2024-04-23T19:12:14.320262+0000","last_clean":"2024-04-23T19:12:14.320262+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:12:14.320262+0000","last_undegraded":"2024-04-23T19:12:14.320262+0000","last_fullsized":"2024-04-23T19:12:14.320262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:42.419535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726896+0000","last_change":"2024-04-23T19:10:15.503053+0000","last_active":"2024-04-23T19:10:47.726896+0000","last_peered":"2024-04-23T19:10:47.726896+0000","last_clean":"2024-04-23T19:10:47.726896+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:10:47.726896+0000","last_undegraded":"2024-04-23T19:10:47.726896+0000","last_fullsized":"2024-04-23T19:10:47.726896+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'18","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.583903+0000","last_change":"2024-04-23T19:10:17.513990+0000","last_active":"2024-04-23T19:13:16.583903+0000","last_peered":"2024-04-23T19:13:16.583903+0000","last_clean":"2024-04-23T19:13:16.583903+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:13:16.583903+0000","last_undegraded":"2024-04-23T19:13:16.583903+0000","last_fullsized":"2024-04-23T19:13:16.583903+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:57.264911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258650+0000","last_change":"2024-04-23T19:10:09.490325+0000","last_active":"2024-04-23T19:13:16.258650+0000","last_peered":"2024-04-23T19:13:16.258650+0000","last_clean":"2024-04-23T19:13:16.258650+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:16.258650+0000","last_undegraded":"2024-04-23T19:13:16.258650+0000","last_fullsized":"2024-04-23T19:13:16.258650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728226+0000","last_change":"2024-04-23T19:10:07.490182+0000","last_active":"2024-04-23T19:10:47.728226+0000","last_peered":"2024-04-23T19:10:47.728226+0000","last_clean":"2024-04-23T19:10:47.728226+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:10:47.728226+0000","last_undegraded":"2024-04-23T19:10:47.728226+0000","last_fullsized":"2024-04-23T19:10:47.728226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719940+0000","last_change":"2024-04-23T19:10:11.490064+0000","last_active":"2024-04-23T19:10:47.719940+0000","last_peered":"2024-04-23T19:10:47.719940+0000","last_clean":"2024-04-23T19:10:47.719940+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:10:47.719940+0000","last_undegraded":"2024-04-23T19:10:47.719940+0000","last_fullsized":"2024-04-23T19:10:47.719940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:48.063186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.300676+0000","last_change":"2024-04-23T19:10:13.491370+0000","last_active":"2024-04-23T19:12:14.300676+0000","last_peered":"2024-04-23T19:12:14.300676+0000","last_clean":"2024-04-23T19:12:14.300676+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:12:14.300676+0000","last_undegraded":"2024-04-23T19:12:14.300676+0000","last_fullsized":"2024-04-23T19:12:14.300676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:59.208837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728086+0000","last_change":"2024-04-23T19:10:15.512494+0000","last_active":"2024-04-23T19:10:47.728086+0000","last_peered":"2024-04-23T19:10:47.728086+0000","last_clean":"2024-04-23T19:10:47.728086+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:10:47.728086+0000","last_undegraded":"2024-04-23T19:10:47.728086+0000","last_fullsized":"2024-04-23T19:10:47.728086+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:54.341417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718208+0000","last_change":"2024-04-23T19:10:17.511337+0000","last_active":"2024-04-23T19:12:16.718208+0000","last_peered":"2024-04-23T19:12:16.718208+0000","last_clean":"2024-04-23T19:12:16.718208+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:12:16.718208+0000","last_undegraded":"2024-04-23T19:12:16.718208+0000","last_fullsized":"2024-04-23T19:12:16.718208+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:31.868099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536682+0000","last_change":"2024-04-23T19:10:09.499760+0000","last_active":"2024-04-23T19:13:16.536682+0000","last_peered":"2024-04-23T19:13:16.536682+0000","last_clean":"2024-04-23T19:13:16.536682+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:16.536682+0000","last_undegraded":"2024-04-23T19:13:16.536682+0000","last_fullsized":"2024-04-23T19:13:16.536682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:43.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727856+0000","last_change":"2024-04-23T19:10:07.494969+0000","last_active":"2024-04-23T19:10:47.727856+0000","last_peered":"2024-04-23T19:10:47.727856+0000","last_clean":"2024-04-23T19:10:47.727856+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:10:47.727856+0000","last_undegraded":"2024-04-23T19:10:47.727856+0000","last_fullsized":"2024-04-23T19:10:47.727856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:47.276166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728157+0000","last_change":"2024-04-23T19:10:11.491444+0000","last_active":"2024-04-23T19:10:47.728157+0000","last_peered":"2024-04-23T19:10:47.728157+0000","last_clean":"2024-04-23T19:10:47.728157+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:10:47.728157+0000","last_undegraded":"2024-04-23T19:10:47.728157+0000","last_fullsized":"2024-04-23T19:10:47.728157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:00.459979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.327052+0000","last_change":"2024-04-23T19:10:13.493342+0000","last_active":"2024-04-23T19:12:14.327052+0000","last_peered":"2024-04-23T19:12:14.327052+0000","last_clean":"2024-04-23T19:12:14.327052+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:12:14.327052+0000","last_undegraded":"2024-04-23T19:12:14.327052+0000","last_fullsized":"2024-04-23T19:12:14.327052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:44.904145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'282","reported_seq":403,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.587980+0000","last_change":"2024-04-23T19:10:15.507087+0000","last_active":"2024-04-23T19:13:16.587980+0000","last_peered":"2024-04-23T19:13:16.587980+0000","last_clean":"2024-04-23T19:13:16.587980+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:16.587980+0000","last_undegraded":"2024-04-23T19:13:16.587980+0000","last_fullsized":"2024-04-23T19:13:16.587980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:48.999181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'12","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712278+0000","last_change":"2024-04-23T19:10:17.518152+0000","last_active":"2024-04-23T19:12:25.712278+0000","last_peered":"2024-04-23T19:12:25.712278+0000","last_clean":"2024-04-23T19:12:25.712278+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:12:25.712278+0000","last_undegraded":"2024-04-23T19:12:25.712278+0000","last_fullsized":"2024-04-23T19:12:25.712278+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399883+0000","last_change":"2024-04-23T19:10:09.493242+0000","last_active":"2024-04-23T19:13:16.399883+0000","last_peered":"2024-04-23T19:13:16.399883+0000","last_clean":"2024-04-23T19:13:16.399883+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:13:16.399883+0000","last_undegraded":"2024-04-23T19:13:16.399883+0000","last_fullsized":"2024-04-23T19:13:16.399883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.679903+0000","last_change":"2024-04-23T19:10:07.508434+0000","last_active":"2024-04-23T19:10:49.679903+0000","last_peered":"2024-04-23T19:10:49.679903+0000","last_clean":"2024-04-23T19:10:49.679903+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:10:49.679903+0000","last_undegraded":"2024-04-23T19:10:49.679903+0000","last_fullsized":"2024-04-23T19:10:49.679903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:44.765151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'3","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.385678+0000","last_change":"2024-04-23T19:10:11.491234+0000","last_active":"2024-04-23T19:12:15.385678+0000","last_peered":"2024-04-23T19:12:15.385678+0000","last_clean":"2024-04-23T19:12:15.385678+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:12:15.385678+0000","last_undegraded":"2024-04-23T19:12:15.385678+0000","last_fullsized":"2024-04-23T19:12:15.385678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:03.467829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727806+0000","last_change":"2024-04-23T19:10:13.501643+0000","last_active":"2024-04-23T19:10:47.727806+0000","last_peered":"2024-04-23T19:10:47.727806+0000","last_clean":"2024-04-23T19:10:47.727806+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:10:47.727806+0000","last_undegraded":"2024-04-23T19:10:47.727806+0000","last_fullsized":"2024-04-23T19:10:47.727806+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:28.282546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'282","reported_seq":465,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575725+0000","last_change":"2024-04-23T19:10:15.501932+0000","last_active":"2024-04-23T19:13:16.575725+0000","last_peered":"2024-04-23T19:13:16.575725+0000","last_clean":"2024-04-23T19:13:16.575725+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:16.575725+0000","last_undegraded":"2024-04-23T19:13:16.575725+0000","last_fullsized":"2024-04-23T19:13:16.575725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.546370+0000","last_change":"2024-04-23T19:10:17.515498+0000","last_active":"2024-04-23T19:12:15.546370+0000","last_peered":"2024-04-23T19:12:15.546370+0000","last_clean":"2024-04-23T19:12:15.546370+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:12:15.546370+0000","last_undegraded":"2024-04-23T19:12:15.546370+0000","last_fullsized":"2024-04-23T19:12:15.546370+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:42.717317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.699282+0000","last_change":"2024-04-23T19:10:09.494353+0000","last_active":"2024-04-23T19:10:49.699282+0000","last_peered":"2024-04-23T19:10:49.699282+0000","last_clean":"2024-04-23T19:10:49.699282+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:10:49.699282+0000","last_undegraded":"2024-04-23T19:10:49.699282+0000","last_fullsized":"2024-04-23T19:10:49.699282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.419635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.386367+0000","last_change":"2024-04-23T19:10:07.507496+0000","last_active":"2024-04-23T19:13:16.386367+0000","last_peered":"2024-04-23T19:13:16.386367+0000","last_clean":"2024-04-23T19:13:16.386367+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:13:16.386367+0000","last_undegraded":"2024-04-23T19:13:16.386367+0000","last_fullsized":"2024-04-23T19:13:16.386367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:08.216950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728027+0000","last_change":"2024-04-23T19:10:11.490678+0000","last_active":"2024-04-23T19:10:47.728027+0000","last_peered":"2024-04-23T19:10:47.728027+0000","last_clean":"2024-04-23T19:10:47.728027+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:10:47.728027+0000","last_undegraded":"2024-04-23T19:10:47.728027+0000","last_fullsized":"2024-04-23T19:10:47.728027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:00.102054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'38","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.313282+0000","last_change":"2024-04-23T19:10:13.491356+0000","last_active":"2024-04-23T19:12:14.313282+0000","last_peered":"2024-04-23T19:12:14.313282+0000","last_clean":"2024-04-23T19:12:14.313282+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:12:14.313282+0000","last_undegraded":"2024-04-23T19:12:14.313282+0000","last_fullsized":"2024-04-23T19:12:14.313282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:58.150827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720298+0000","last_change":"2024-04-23T19:10:15.506621+0000","last_active":"2024-04-23T19:10:47.720298+0000","last_peered":"2024-04-23T19:10:47.720298+0000","last_clean":"2024-04-23T19:10:47.720298+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:10:47.720298+0000","last_undegraded":"2024-04-23T19:10:47.720298+0000","last_fullsized":"2024-04-23T19:10:47.720298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:01.215939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.516499+0000","last_change":"2024-04-23T19:10:17.506577+0000","last_active":"2024-04-23T19:13:16.516499+0000","last_peered":"2024-04-23T19:13:16.516499+0000","last_clean":"2024-04-23T19:13:16.516499+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:16.516499+0000","last_undegraded":"2024-04-23T19:13:16.516499+0000","last_fullsized":"2024-04-23T19:13:16.516499+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:44.079746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718448+0000","last_change":"2024-04-23T19:10:09.499812+0000","last_active":"2024-04-23T19:12:16.718448+0000","last_peered":"2024-04-23T19:12:16.718448+0000","last_clean":"2024-04-23T19:12:16.718448+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:12:16.718448+0000","last_undegraded":"2024-04-23T19:12:16.718448+0000","last_fullsized":"2024-04-23T19:12:16.718448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:34.586789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716709+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:10:47.716709+0000","last_peered":"2024-04-23T19:10:47.716709+0000","last_clean":"2024-04-23T19:10:47.716709+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:10:47.716709+0000","last_undegraded":"2024-04-23T19:10:47.716709+0000","last_fullsized":"2024-04-23T19:10:47.716709+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:35.747545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728488+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:10:47.728488+0000","last_peered":"2024-04-23T19:10:47.728488+0000","last_clean":"2024-04-23T19:10:47.728488+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:10:47.728488+0000","last_undegraded":"2024-04-23T19:10:47.728488+0000","last_fullsized":"2024-04-23T19:10:47.728488+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:09.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.547770+0000","last_change":"2024-04-23T19:10:07.508995+0000","last_active":"2024-04-23T19:10:48.547770+0000","last_peered":"2024-04-23T19:10:48.547770+0000","last_clean":"2024-04-23T19:10:48.547770+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:10:48.547770+0000","last_undegraded":"2024-04-23T19:10:48.547770+0000","last_fullsized":"2024-04-23T19:10:48.547770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:08.545897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718470+0000","last_change":"2024-04-23T19:10:11.486708+0000","last_active":"2024-04-23T19:10:47.718470+0000","last_peered":"2024-04-23T19:10:47.718470+0000","last_clean":"2024-04-23T19:10:47.718470+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:10:47.718470+0000","last_undegraded":"2024-04-23T19:10:47.718470+0000","last_fullsized":"2024-04-23T19:10:47.718470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'77","reported_seq":146,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.325024+0000","last_change":"2024-04-23T19:10:13.500080+0000","last_active":"2024-04-23T19:12:14.325024+0000","last_peered":"2024-04-23T19:12:14.325024+0000","last_clean":"2024-04-23T19:12:14.325024+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:12:14.325024+0000","last_undegraded":"2024-04-23T19:12:14.325024+0000","last_fullsized":"2024-04-23T19:12:14.325024+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:43.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728972+0000","last_change":"2024-04-23T19:10:15.502149+0000","last_active":"2024-04-23T19:10:47.728972+0000","last_peered":"2024-04-23T19:10:47.728972+0000","last_clean":"2024-04-23T19:10:47.728972+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:10:47.728972+0000","last_undegraded":"2024-04-23T19:10:47.728972+0000","last_fullsized":"2024-04-23T19:10:47.728972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:59.765301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":162,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.491783+0000","last_change":"2024-04-23T19:10:17.516475+0000","last_active":"2024-04-23T19:13:16.491783+0000","last_peered":"2024-04-23T19:13:16.491783+0000","last_clean":"2024-04-23T19:13:16.491783+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:16.491783+0000","last_undegraded":"2024-04-23T19:13:16.491783+0000","last_fullsized":"2024-04-23T19:13:16.491783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:20.418910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.149738+0000","last_change":"2024-04-23T19:10:09.488098+0000","last_active":"2024-04-23T19:11:32.149738+0000","last_peered":"2024-04-23T19:11:32.149738+0000","last_clean":"2024-04-23T19:11:32.149738+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:11:32.149738+0000","last_undegraded":"2024-04-23T19:11:32.149738+0000","last_fullsized":"2024-04-23T19:11:32.149738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:52.645415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725571+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:10:47.725571+0000","last_peered":"2024-04-23T19:10:47.725571+0000","last_clean":"2024-04-23T19:10:47.725571+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:10:47.725571+0000","last_undegraded":"2024-04-23T19:10:47.725571+0000","last_fullsized":"2024-04-23T19:10:47.725571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:17.746806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726921+0000","last_change":"2024-04-23T19:10:07.486483+0000","last_active":"2024-04-23T19:10:47.726921+0000","last_peered":"2024-04-23T19:10:47.726921+0000","last_clean":"2024-04-23T19:10:47.726921+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:10:47.726921+0000","last_undegraded":"2024-04-23T19:10:47.726921+0000","last_fullsized":"2024-04-23T19:10:47.726921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729492+0000","last_change":"2024-04-23T19:10:11.487708+0000","last_active":"2024-04-23T19:10:47.729492+0000","last_peered":"2024-04-23T19:10:47.729492+0000","last_clean":"2024-04-23T19:10:47.729492+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:10:47.729492+0000","last_undegraded":"2024-04-23T19:10:47.729492+0000","last_fullsized":"2024-04-23T19:10:47.729492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:07.101469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'273","reported_seq":476,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575779+0000","last_change":"2024-04-23T19:10:13.500713+0000","last_active":"2024-04-23T19:13:16.575779+0000","last_peered":"2024-04-23T19:13:16.575779+0000","last_clean":"2024-04-23T19:13:16.575779+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:16.575779+0000","last_undegraded":"2024-04-23T19:13:16.575779+0000","last_fullsized":"2024-04-23T19:13:16.575779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:46.992080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727751+0000","last_change":"2024-04-23T19:10:15.513032+0000","last_active":"2024-04-23T19:10:47.727751+0000","last_peered":"2024-04-23T19:10:47.727751+0000","last_clean":"2024-04-23T19:10:47.727751+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:10:47.727751+0000","last_undegraded":"2024-04-23T19:10:47.727751+0000","last_fullsized":"2024-04-23T19:10:47.727751+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:13.084568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.968215+0000","last_change":"2024-04-23T19:10:17.518750+0000","last_active":"2024-04-23T19:11:31.968215+0000","last_peered":"2024-04-23T19:11:31.968215+0000","last_clean":"2024-04-23T19:11:31.968215+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:11:31.968215+0000","last_undegraded":"2024-04-23T19:11:31.968215+0000","last_fullsized":"2024-04-23T19:11:31.968215+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:02.438494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482305+0000","last_change":"2024-04-23T19:10:09.486782+0000","last_active":"2024-04-23T19:13:16.482305+0000","last_peered":"2024-04-23T19:13:16.482305+0000","last_clean":"2024-04-23T19:13:16.482305+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:13:16.482305+0000","last_undegraded":"2024-04-23T19:13:16.482305+0000","last_fullsized":"2024-04-23T19:13:16.482305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718645+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:10:47.718645+0000","last_peered":"2024-04-23T19:10:47.718645+0000","last_clean":"2024-04-23T19:10:47.718645+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:10:47.718645+0000","last_undegraded":"2024-04-23T19:10:47.718645+0000","last_fullsized":"2024-04-23T19:10:47.718645+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:45.326443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726584+0000","last_change":"2024-04-23T19:10:07.491212+0000","last_active":"2024-04-23T19:10:47.726584+0000","last_peered":"2024-04-23T19:10:47.726584+0000","last_clean":"2024-04-23T19:10:47.726584+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:10:47.726584+0000","last_undegraded":"2024-04-23T19:10:47.726584+0000","last_fullsized":"2024-04-23T19:10:47.726584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"23'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.736899+0000","last_change":"2024-04-23T19:10:47.736369+0000","last_active":"2024-04-23T19:10:47.736899+0000","last_peered":"2024-04-23T19:10:47.736899+0000","last_clean":"2024-04-23T19:10:47.736899+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:10:47.736899+0000","last_undegraded":"2024-04-23T19:10:47.736899+0000","last_fullsized":"2024-04-23T19:10:47.736899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:05.556192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'16","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.576017+0000","last_change":"2024-04-23T19:10:13.495216+0000","last_active":"2024-04-23T19:13:16.576017+0000","last_peered":"2024-04-23T19:13:16.576017+0000","last_clean":"2024-04-23T19:13:16.576017+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:16.576017+0000","last_undegraded":"2024-04-23T19:13:16.576017+0000","last_fullsized":"2024-04-23T19:13:16.576017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:32.747522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728617+0000","last_change":"2024-04-23T19:10:15.502291+0000","last_active":"2024-04-23T19:10:47.728617+0000","last_peered":"2024-04-23T19:10:47.728617+0000","last_clean":"2024-04-23T19:10:47.728617+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:10:47.728617+0000","last_undegraded":"2024-04-23T19:10:47.728617+0000","last_fullsized":"2024-04-23T19:10:47.728617+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:04.115844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725858+0000","last_change":"2024-04-23T19:10:17.517093+0000","last_active":"2024-04-23T19:10:47.725858+0000","last_peered":"2024-04-23T19:10:47.725858+0000","last_clean":"2024-04-23T19:10:47.725858+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:10:47.725858+0000","last_undegraded":"2024-04-23T19:10:47.725858+0000","last_fullsized":"2024-04-23T19:10:47.725858+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:42.364548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399934+0000","last_change":"2024-04-23T19:10:09.477714+0000","last_active":"2024-04-23T19:13:16.399934+0000","last_peered":"2024-04-23T19:13:16.399934+0000","last_clean":"2024-04-23T19:13:16.399934+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:16.399934+0000","last_undegraded":"2024-04-23T19:13:16.399934+0000","last_fullsized":"2024-04-23T19:13:16.399934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726493+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:10:47.726493+0000","last_peered":"2024-04-23T19:10:47.726493+0000","last_clean":"2024-04-23T19:10:47.726493+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:10:47.726493+0000","last_undegraded":"2024-04-23T19:10:47.726493+0000","last_fullsized":"2024-04-23T19:10:47.726493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:51.612339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720012+0000","last_change":"2024-04-23T19:10:07.482292+0000","last_active":"2024-04-23T19:10:47.720012+0000","last_peered":"2024-04-23T19:10:47.720012+0000","last_clean":"2024-04-23T19:10:47.720012+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:10:47.720012+0000","last_undegraded":"2024-04-23T19:10:47.720012+0000","last_fullsized":"2024-04-23T19:10:47.720012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729072+0000","last_change":"2024-04-23T19:10:11.490475+0000","last_active":"2024-04-23T19:10:47.729072+0000","last_peered":"2024-04-23T19:10:47.729072+0000","last_clean":"2024-04-23T19:10:47.729072+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:10:47.729072+0000","last_undegraded":"2024-04-23T19:10:47.729072+0000","last_fullsized":"2024-04-23T19:10:47.729072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:37.989491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'14","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.784607+0000","last_change":"2024-04-23T19:10:13.501263+0000","last_active":"2024-04-23T19:13:15.784607+0000","last_peered":"2024-04-23T19:13:15.784607+0000","last_clean":"2024-04-23T19:13:15.784607+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:15.784607+0000","last_undegraded":"2024-04-23T19:13:15.784607+0000","last_fullsized":"2024-04-23T19:13:15.784607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:45:20.243610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728295+0000","last_change":"2024-04-23T19:10:15.512566+0000","last_active":"2024-04-23T19:10:47.728295+0000","last_peered":"2024-04-23T19:10:47.728295+0000","last_clean":"2024-04-23T19:10:47.728295+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:10:47.728295+0000","last_undegraded":"2024-04-23T19:10:47.728295+0000","last_fullsized":"2024-04-23T19:10:47.728295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:37:48.972551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.182118+0000","last_change":"2024-04-23T19:10:17.515693+0000","last_active":"2024-04-23T19:13:16.182118+0000","last_peered":"2024-04-23T19:13:16.182118+0000","last_clean":"2024-04-23T19:13:16.182118+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:13:16.182118+0000","last_undegraded":"2024-04-23T19:13:16.182118+0000","last_fullsized":"2024-04-23T19:13:16.182118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:12.374714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447462+0000","last_change":"2024-04-23T19:10:09.494884+0000","last_active":"2024-04-23T19:13:16.447462+0000","last_peered":"2024-04-23T19:13:16.447462+0000","last_clean":"2024-04-23T19:13:16.447462+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:16.447462+0000","last_undegraded":"2024-04-23T19:13:16.447462+0000","last_fullsized":"2024-04-23T19:13:16.447462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.383442+0000","last_change":"2024-04-23T19:10:07.483919+0000","last_active":"2024-04-23T19:13:16.383442+0000","last_peered":"2024-04-23T19:13:16.383442+0000","last_clean":"2024-04-23T19:13:16.383442+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:13:16.383442+0000","last_undegraded":"2024-04-23T19:13:16.383442+0000","last_fullsized":"2024-04-23T19:13:16.383442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728467+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:10:47.728467+0000","last_peered":"2024-04-23T19:10:47.728467+0000","last_clean":"2024-04-23T19:10:47.728467+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:10:47.728467+0000","last_undegraded":"2024-04-23T19:10:47.728467+0000","last_fullsized":"2024-04-23T19:10:47.728467+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:34:51.167423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:55.721760+0000","last_change":"2024-04-23T19:10:47.730021+0000","last_active":"2024-04-23T19:12:55.721760+0000","last_peered":"2024-04-23T19:12:55.721760+0000","last_clean":"2024-04-23T19:12:55.721760+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:12:55.721760+0000","last_undegraded":"2024-04-23T19:12:55.721760+0000","last_fullsized":"2024-04-23T19:12:55.721760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:56.417228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'63","reported_seq":122,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710844+0000","last_change":"2024-04-23T19:10:13.505680+0000","last_active":"2024-04-23T19:12:15.710844+0000","last_peered":"2024-04-23T19:12:15.710844+0000","last_clean":"2024-04-23T19:12:15.710844+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:12:15.710844+0000","last_undegraded":"2024-04-23T19:12:15.710844+0000","last_fullsized":"2024-04-23T19:12:15.710844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:59.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719607+0000","last_change":"2024-04-23T19:10:15.506374+0000","last_active":"2024-04-23T19:10:47.719607+0000","last_peered":"2024-04-23T19:10:47.719607+0000","last_clean":"2024-04-23T19:10:47.719607+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:10:47.719607+0000","last_undegraded":"2024-04-23T19:10:47.719607+0000","last_fullsized":"2024-04-23T19:10:47.719607+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:42.776000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.868259+0000","last_change":"2024-04-23T19:10:17.506903+0000","last_active":"2024-04-23T19:13:15.868259+0000","last_peered":"2024-04-23T19:13:15.868259+0000","last_clean":"2024-04-23T19:13:15.868259+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:15.868259+0000","last_undegraded":"2024-04-23T19:13:15.868259+0000","last_fullsized":"2024-04-23T19:13:15.868259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:24.436159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447101+0000","last_change":"2024-04-23T19:10:09.492787+0000","last_active":"2024-04-23T19:13:16.447101+0000","last_peered":"2024-04-23T19:13:16.447101+0000","last_clean":"2024-04-23T19:13:16.447101+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:16.447101+0000","last_undegraded":"2024-04-23T19:13:16.447101+0000","last_fullsized":"2024-04-23T19:13:16.447101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545103+0000","last_change":"2024-04-23T19:10:07.486351+0000","last_active":"2024-04-23T19:10:48.545103+0000","last_peered":"2024-04-23T19:10:48.545103+0000","last_clean":"2024-04-23T19:10:48.545103+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:10:48.545103+0000","last_undegraded":"2024-04-23T19:10:48.545103+0000","last_fullsized":"2024-04-23T19:10:48.545103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:48:50.668566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728626+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:10:47.728626+0000","last_peered":"2024-04-23T19:10:47.728626+0000","last_clean":"2024-04-23T19:10:47.728626+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:10:47.728626+0000","last_undegraded":"2024-04-23T19:10:47.728626+0000","last_fullsized":"2024-04-23T19:10:47.728626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:36.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726094+0000","last_change":"2024-04-23T19:10:11.488952+0000","last_active":"2024-04-23T19:10:47.726094+0000","last_peered":"2024-04-23T19:10:47.726094+0000","last_clean":"2024-04-23T19:10:47.726094+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:10:47.726094+0000","last_undegraded":"2024-04-23T19:10:47.726094+0000","last_fullsized":"2024-04-23T19:10:47.726094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:09.450816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'15","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.702102+0000","last_change":"2024-04-23T19:10:13.497890+0000","last_active":"2024-04-23T19:10:49.702102+0000","last_peered":"2024-04-23T19:10:49.702102+0000","last_clean":"2024-04-23T19:10:49.702102+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:10:49.702102+0000","last_undegraded":"2024-04-23T19:10:49.702102+0000","last_fullsized":"2024-04-23T19:10:49.702102+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:21.612895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'282","reported_seq":355,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575687+0000","last_change":"2024-04-23T19:10:15.502267+0000","last_active":"2024-04-23T19:13:16.575687+0000","last_peered":"2024-04-23T19:13:16.575687+0000","last_clean":"2024-04-23T19:13:16.575687+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:16.575687+0000","last_undegraded":"2024-04-23T19:13:16.575687+0000","last_fullsized":"2024-04-23T19:13:16.575687+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.147752+0000","last_change":"2024-04-23T19:10:17.513890+0000","last_active":"2024-04-23T19:11:32.147752+0000","last_peered":"2024-04-23T19:11:32.147752+0000","last_clean":"2024-04-23T19:11:32.147752+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:11:32.147752+0000","last_undegraded":"2024-04-23T19:11:32.147752+0000","last_fullsized":"2024-04-23T19:11:32.147752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:45.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482692+0000","last_change":"2024-04-23T19:10:09.496860+0000","last_active":"2024-04-23T19:13:16.482692+0000","last_peered":"2024-04-23T19:13:16.482692+0000","last_clean":"2024-04-23T19:13:16.482692+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:13:16.482692+0000","last_undegraded":"2024-04-23T19:13:16.482692+0000","last_fullsized":"2024-04-23T19:13:16.482692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:58.945396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729332+0000","last_change":"2024-04-23T19:10:07.493532+0000","last_active":"2024-04-23T19:10:47.729332+0000","last_peered":"2024-04-23T19:10:47.729332+0000","last_clean":"2024-04-23T19:10:47.729332+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:10:47.729332+0000","last_undegraded":"2024-04-23T19:10:47.729332+0000","last_fullsized":"2024-04-23T19:10:47.729332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:03.936999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729461+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:10:47.729461+0000","last_peered":"2024-04-23T19:10:47.729461+0000","last_clean":"2024-04-23T19:10:47.729461+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:10:47.729461+0000","last_undegraded":"2024-04-23T19:10:47.729461+0000","last_fullsized":"2024-04-23T19:10:47.729461+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:13.817485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728459+0000","last_change":"2024-04-23T19:10:11.487496+0000","last_active":"2024-04-23T19:10:47.728459+0000","last_peered":"2024-04-23T19:10:47.728459+0000","last_clean":"2024-04-23T19:10:47.728459+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:10:47.728459+0000","last_undegraded":"2024-04-23T19:10:47.728459+0000","last_fullsized":"2024-04-23T19:10:47.728459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:12.234867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"30'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728000+0000","last_change":"2024-04-23T19:10:13.503998+0000","last_active":"2024-04-23T19:10:47.728000+0000","last_peered":"2024-04-23T19:10:47.728000+0000","last_clean":"2024-04-23T19:10:47.728000+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:10:47.728000+0000","last_undegraded":"2024-04-23T19:10:47.728000+0000","last_fullsized":"2024-04-23T19:10:47.728000+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:33.974553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'282","reported_seq":429,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.584566+0000","last_change":"2024-04-23T19:10:15.509417+0000","last_active":"2024-04-23T19:13:16.584566+0000","last_peered":"2024-04-23T19:13:16.584566+0000","last_clean":"2024-04-23T19:13:16.584566+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:16.584566+0000","last_undegraded":"2024-04-23T19:13:16.584566+0000","last_fullsized":"2024-04-23T19:13:16.584566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718167+0000","last_change":"2024-04-23T19:10:17.516808+0000","last_active":"2024-04-23T19:10:47.718167+0000","last_peered":"2024-04-23T19:10:47.718167+0000","last_clean":"2024-04-23T19:10:47.718167+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:10:47.718167+0000","last_undegraded":"2024-04-23T19:10:47.718167+0000","last_fullsized":"2024-04-23T19:10:47.718167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:41.513557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449088+0000","last_change":"2024-04-23T19:10:09.493717+0000","last_active":"2024-04-23T19:13:16.449088+0000","last_peered":"2024-04-23T19:13:16.449088+0000","last_clean":"2024-04-23T19:13:16.449088+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:16.449088+0000","last_undegraded":"2024-04-23T19:13:16.449088+0000","last_fullsized":"2024-04-23T19:13:16.449088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:44.856835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.749316+0000","last_change":"2024-04-23T19:10:07.500858+0000","last_active":"2024-04-23T19:12:15.749316+0000","last_peered":"2024-04-23T19:12:15.749316+0000","last_clean":"2024-04-23T19:12:15.749316+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:12:15.749316+0000","last_undegraded":"2024-04-23T19:12:15.749316+0000","last_fullsized":"2024-04-23T19:12:15.749316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728818+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:10:47.728818+0000","last_peered":"2024-04-23T19:10:47.728818+0000","last_clean":"2024-04-23T19:10:47.728818+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:10:47.728818+0000","last_undegraded":"2024-04-23T19:10:47.728818+0000","last_fullsized":"2024-04-23T19:10:47.728818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.025384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727223+0000","last_change":"2024-04-23T19:10:11.489138+0000","last_active":"2024-04-23T19:10:47.727223+0000","last_peered":"2024-04-23T19:10:47.727223+0000","last_clean":"2024-04-23T19:10:47.727223+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:10:47.727223+0000","last_undegraded":"2024-04-23T19:10:47.727223+0000","last_fullsized":"2024-04-23T19:10:47.727223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:19.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'83","reported_seq":155,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.716534+0000","last_change":"2024-04-23T19:10:13.503368+0000","last_active":"2024-04-23T19:12:20.716534+0000","last_peered":"2024-04-23T19:12:20.716534+0000","last_clean":"2024-04-23T19:12:20.716534+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:12:20.716534+0000","last_undegraded":"2024-04-23T19:12:20.716534+0000","last_fullsized":"2024-04-23T19:12:20.716534+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:31.109448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6026,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":77,"num_read_kb":63,"num_write":30,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727890+0000","last_change":"2024-04-23T19:10:15.500592+0000","last_active":"2024-04-23T19:10:47.727890+0000","last_peered":"2024-04-23T19:10:47.727890+0000","last_clean":"2024-04-23T19:10:47.727890+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:10:47.727890+0000","last_undegraded":"2024-04-23T19:10:47.727890+0000","last_fullsized":"2024-04-23T19:10:47.727890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:11.115258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722791+0000","last_change":"2024-04-23T19:10:17.515628+0000","last_active":"2024-04-23T19:10:47.722791+0000","last_peered":"2024-04-23T19:10:47.722791+0000","last_clean":"2024-04-23T19:10:47.722791+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:10:47.722791+0000","last_undegraded":"2024-04-23T19:10:47.722791+0000","last_fullsized":"2024-04-23T19:10:47.722791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.641905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722100+0000","last_change":"2024-04-23T19:10:17.515232+0000","last_active":"2024-04-23T19:10:47.722100+0000","last_peered":"2024-04-23T19:10:47.722100+0000","last_clean":"2024-04-23T19:10:47.722100+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:10:47.722100+0000","last_undegraded":"2024-04-23T19:10:47.722100+0000","last_fullsized":"2024-04-23T19:10:47.722100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:40.692521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720221+0000","last_change":"2024-04-23T19:10:09.492213+0000","last_active":"2024-04-23T19:10:47.720221+0000","last_peered":"2024-04-23T19:10:47.720221+0000","last_clean":"2024-04-23T19:10:47.720221+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:10:47.720221+0000","last_undegraded":"2024-04-23T19:10:47.720221+0000","last_fullsized":"2024-04-23T19:10:47.720221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:54.477619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726467+0000","last_change":"2024-04-23T19:10:07.500107+0000","last_active":"2024-04-23T19:10:47.726467+0000","last_peered":"2024-04-23T19:10:47.726467+0000","last_clean":"2024-04-23T19:10:47.726467+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:10:47.726467+0000","last_undegraded":"2024-04-23T19:10:47.726467+0000","last_fullsized":"2024-04-23T19:10:47.726467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:14.278078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727809+0000","last_change":"2024-04-23T19:10:47.725041+0000","last_active":"2024-04-23T19:10:47.727809+0000","last_peered":"2024-04-23T19:10:47.727809+0000","last_clean":"2024-04-23T19:10:47.727809+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:10:47.727809+0000","last_undegraded":"2024-04-23T19:10:47.727809+0000","last_fullsized":"2024-04-23T19:10:47.727809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:17.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.317498+0000","last_change":"2024-04-23T19:10:13.497156+0000","last_active":"2024-04-23T19:12:14.317498+0000","last_peered":"2024-04-23T19:12:14.317498+0000","last_clean":"2024-04-23T19:12:14.317498+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:12:14.317498+0000","last_undegraded":"2024-04-23T19:12:14.317498+0000","last_fullsized":"2024-04-23T19:12:14.317498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:49.136472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726980+0000","last_change":"2024-04-23T19:10:15.514289+0000","last_active":"2024-04-23T19:10:47.726980+0000","last_peered":"2024-04-23T19:10:47.726980+0000","last_clean":"2024-04-23T19:10:47.726980+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:10:47.726980+0000","last_undegraded":"2024-04-23T19:10:47.726980+0000","last_fullsized":"2024-04-23T19:10:47.726980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:12.517542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.023661+0000","last_change":"2024-04-23T19:10:17.514011+0000","last_active":"2024-04-23T19:12:14.023661+0000","last_peered":"2024-04-23T19:12:14.023661+0000","last_clean":"2024-04-23T19:12:14.023661+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:12:14.023661+0000","last_undegraded":"2024-04-23T19:12:14.023661+0000","last_fullsized":"2024-04-23T19:12:14.023661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:55:22.164549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.733183+0000","last_change":"2024-04-23T19:10:47.732302+0000","last_active":"2024-04-23T19:10:47.733183+0000","last_peered":"2024-04-23T19:10:47.733183+0000","last_clean":"2024-04-23T19:10:47.733183+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:10:47.733183+0000","last_undegraded":"2024-04-23T19:10:47.733183+0000","last_fullsized":"2024-04-23T19:10:47.733183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:05.759643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.139371+0000","last_change":"2024-04-23T19:10:07.493914+0000","last_active":"2024-04-23T19:13:16.139371+0000","last_peered":"2024-04-23T19:13:16.139371+0000","last_clean":"2024-04-23T19:13:16.139371+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:13:16.139371+0000","last_undegraded":"2024-04-23T19:13:16.139371+0000","last_fullsized":"2024-04-23T19:13:16.139371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729231+0000","last_change":"2024-04-23T19:10:11.487861+0000","last_active":"2024-04-23T19:10:47.729231+0000","last_peered":"2024-04-23T19:10:47.729231+0000","last_clean":"2024-04-23T19:10:47.729231+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:10:47.729231+0000","last_undegraded":"2024-04-23T19:10:47.729231+0000","last_fullsized":"2024-04-23T19:10:47.729231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:47.278160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'5","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789104+0000","last_change":"2024-04-23T19:10:13.494591+0000","last_active":"2024-04-23T19:13:15.789104+0000","last_peered":"2024-04-23T19:13:15.789104+0000","last_clean":"2024-04-23T19:13:15.789104+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:15.789104+0000","last_undegraded":"2024-04-23T19:13:15.789104+0000","last_fullsized":"2024-04-23T19:13:15.789104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:58.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718895+0000","last_change":"2024-04-23T19:10:15.513825+0000","last_active":"2024-04-23T19:10:47.718895+0000","last_peered":"2024-04-23T19:10:47.718895+0000","last_clean":"2024-04-23T19:10:47.718895+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:10:47.718895+0000","last_undegraded":"2024-04-23T19:10:47.718895+0000","last_fullsized":"2024-04-23T19:10:47.718895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728340+0000","last_change":"2024-04-23T19:10:17.520788+0000","last_active":"2024-04-23T19:10:47.728340+0000","last_peered":"2024-04-23T19:10:47.728340+0000","last_clean":"2024-04-23T19:10:47.728340+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:10:47.728340+0000","last_undegraded":"2024-04-23T19:10:47.728340+0000","last_fullsized":"2024-04-23T19:10:47.728340+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:36.438306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399875+0000","last_change":"2024-04-23T19:10:09.491571+0000","last_active":"2024-04-23T19:13:16.399875+0000","last_peered":"2024-04-23T19:13:16.399875+0000","last_clean":"2024-04-23T19:13:16.399875+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:16.399875+0000","last_undegraded":"2024-04-23T19:13:16.399875+0000","last_fullsized":"2024-04-23T19:13:16.399875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724424+0000","last_change":"2024-04-23T19:10:07.500769+0000","last_active":"2024-04-23T19:10:47.724424+0000","last_peered":"2024-04-23T19:10:47.724424+0000","last_clean":"2024-04-23T19:10:47.724424+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:10:47.724424+0000","last_undegraded":"2024-04-23T19:10:47.724424+0000","last_fullsized":"2024-04-23T19:10:47.724424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718451+0000","last_change":"2024-04-23T19:10:11.493115+0000","last_active":"2024-04-23T19:10:47.718451+0000","last_peered":"2024-04-23T19:10:47.718451+0000","last_clean":"2024-04-23T19:10:47.718451+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:10:47.718451+0000","last_undegraded":"2024-04-23T19:10:47.718451+0000","last_fullsized":"2024-04-23T19:10:47.718451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:51.288167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'40","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.848055+0000","last_change":"2024-04-23T19:10:13.491151+0000","last_active":"2024-04-23T19:13:15.848055+0000","last_peered":"2024-04-23T19:13:15.848055+0000","last_clean":"2024-04-23T19:13:15.848055+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:15.848055+0000","last_undegraded":"2024-04-23T19:13:15.848055+0000","last_fullsized":"2024-04-23T19:13:15.848055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:56.656293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717052+0000","last_change":"2024-04-23T19:10:15.497695+0000","last_active":"2024-04-23T19:10:47.717052+0000","last_peered":"2024-04-23T19:10:47.717052+0000","last_clean":"2024-04-23T19:10:47.717052+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:10:47.717052+0000","last_undegraded":"2024-04-23T19:10:47.717052+0000","last_fullsized":"2024-04-23T19:10:47.717052+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:40.709841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.164233+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:10:50.164233+0000","last_peered":"2024-04-23T19:10:50.164233+0000","last_clean":"2024-04-23T19:10:50.164233+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:10:50.164233+0000","last_undegraded":"2024-04-23T19:10:50.164233+0000","last_fullsized":"2024-04-23T19:10:50.164233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:38:04.095526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'9","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165987+0000","last_change":"2024-04-23T19:10:09.498393+0000","last_active":"2024-04-23T19:13:16.165987+0000","last_peered":"2024-04-23T19:13:16.165987+0000","last_clean":"2024-04-23T19:13:16.165987+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:13:16.165987+0000","last_undegraded":"2024-04-23T19:13:16.165987+0000","last_fullsized":"2024-04-23T19:13:16.165987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:24.946389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722073+0000","last_change":"2024-04-23T19:10:07.496451+0000","last_active":"2024-04-23T19:10:47.722073+0000","last_peered":"2024-04-23T19:10:47.722073+0000","last_clean":"2024-04-23T19:10:47.722073+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:10:47.722073+0000","last_undegraded":"2024-04-23T19:10:47.722073+0000","last_fullsized":"2024-04-23T19:10:47.722073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724117+0000","last_change":"2024-04-23T19:10:11.490728+0000","last_active":"2024-04-23T19:10:47.724117+0000","last_peered":"2024-04-23T19:10:47.724117+0000","last_clean":"2024-04-23T19:10:47.724117+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:10:47.724117+0000","last_undegraded":"2024-04-23T19:10:47.724117+0000","last_fullsized":"2024-04-23T19:10:47.724117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:55.751511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'32","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.302845+0000","last_change":"2024-04-23T19:10:13.493330+0000","last_active":"2024-04-23T19:12:14.302845+0000","last_peered":"2024-04-23T19:12:14.302845+0000","last_clean":"2024-04-23T19:12:14.302845+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:12:14.302845+0000","last_undegraded":"2024-04-23T19:12:14.302845+0000","last_fullsized":"2024-04-23T19:12:14.302845+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:48.481253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":44,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719061+0000","last_change":"2024-04-23T19:10:15.508565+0000","last_active":"2024-04-23T19:10:47.719061+0000","last_peered":"2024-04-23T19:10:47.719061+0000","last_clean":"2024-04-23T19:10:47.719061+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:10:47.719061+0000","last_undegraded":"2024-04-23T19:10:47.719061+0000","last_fullsized":"2024-04-23T19:10:47.719061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:02.695164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703848+0000","last_change":"2024-04-23T19:10:47.728408+0000","last_active":"2024-04-23T19:10:50.703848+0000","last_peered":"2024-04-23T19:10:50.703848+0000","last_clean":"2024-04-23T19:10:50.703848+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:10:50.703848+0000","last_undegraded":"2024-04-23T19:10:50.703848+0000","last_fullsized":"2024-04-23T19:10:50.703848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:36.054291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.720730+0000","last_change":"2024-04-23T19:10:07.503907+0000","last_active":"2024-04-23T19:12:15.720730+0000","last_peered":"2024-04-23T19:12:15.720730+0000","last_clean":"2024-04-23T19:12:15.720730+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:12:15.720730+0000","last_undegraded":"2024-04-23T19:12:15.720730+0000","last_fullsized":"2024-04-23T19:12:15.720730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:52.469809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730560+0000","last_change":"2024-04-23T19:10:11.490872+0000","last_active":"2024-04-23T19:10:47.730560+0000","last_peered":"2024-04-23T19:10:47.730560+0000","last_clean":"2024-04-23T19:10:47.730560+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:10:47.730560+0000","last_undegraded":"2024-04-23T19:10:47.730560+0000","last_fullsized":"2024-04-23T19:10:47.730560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'54","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.323713+0000","last_change":"2024-04-23T19:10:13.494268+0000","last_active":"2024-04-23T19:12:14.323713+0000","last_peered":"2024-04-23T19:12:14.323713+0000","last_clean":"2024-04-23T19:12:14.323713+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:12:14.323713+0000","last_undegraded":"2024-04-23T19:12:14.323713+0000","last_fullsized":"2024-04-23T19:12:14.323713+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:00.688062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719088+0000","last_change":"2024-04-23T19:10:15.505820+0000","last_active":"2024-04-23T19:10:47.719088+0000","last_peered":"2024-04-23T19:10:47.719088+0000","last_clean":"2024-04-23T19:10:47.719088+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:10:47.719088+0000","last_undegraded":"2024-04-23T19:10:47.719088+0000","last_fullsized":"2024-04-23T19:10:47.719088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:23.619996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.704248+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:15.704248+0000","last_peered":"2024-04-23T19:12:15.704248+0000","last_clean":"2024-04-23T19:12:15.704248+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:12:15.704248+0000","last_undegraded":"2024-04-23T19:12:15.704248+0000","last_fullsized":"2024-04-23T19:12:15.704248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:58.312384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482427+0000","last_change":"2024-04-23T19:10:09.482621+0000","last_active":"2024-04-23T19:13:16.482427+0000","last_peered":"2024-04-23T19:13:16.482427+0000","last_clean":"2024-04-23T19:13:16.482427+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:13:16.482427+0000","last_undegraded":"2024-04-23T19:13:16.482427+0000","last_fullsized":"2024-04-23T19:13:16.482427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720227+0000","last_change":"2024-04-23T19:10:07.493942+0000","last_active":"2024-04-23T19:12:31.720227+0000","last_peered":"2024-04-23T19:12:31.720227+0000","last_clean":"2024-04-23T19:12:31.720227+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:12:31.720227+0000","last_undegraded":"2024-04-23T19:12:31.720227+0000","last_fullsized":"2024-04-23T19:12:31.720227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721926+0000","last_change":"2024-04-23T19:10:11.486920+0000","last_active":"2024-04-23T19:10:47.721926+0000","last_peered":"2024-04-23T19:10:47.721926+0000","last_clean":"2024-04-23T19:10:47.721926+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:10:47.721926+0000","last_undegraded":"2024-04-23T19:10:47.721926+0000","last_fullsized":"2024-04-23T19:10:47.721926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:56.355675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.303836+0000","last_change":"2024-04-23T19:10:13.502855+0000","last_active":"2024-04-23T19:12:14.303836+0000","last_peered":"2024-04-23T19:12:14.303836+0000","last_clean":"2024-04-23T19:12:14.303836+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:12:14.303836+0000","last_undegraded":"2024-04-23T19:12:14.303836+0000","last_fullsized":"2024-04-23T19:12:14.303836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:09.770521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717958+0000","last_change":"2024-04-23T19:10:15.510094+0000","last_active":"2024-04-23T19:10:47.717958+0000","last_peered":"2024-04-23T19:10:47.717958+0000","last_clean":"2024-04-23T19:10:47.717958+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:10:47.717958+0000","last_undegraded":"2024-04-23T19:10:47.717958+0000","last_fullsized":"2024-04-23T19:10:47.717958+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.609469+0000","last_change":"2024-04-23T19:10:17.517391+0000","last_active":"2024-04-23T19:12:12.609469+0000","last_peered":"2024-04-23T19:12:12.609469+0000","last_clean":"2024-04-23T19:12:12.609469+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:12:12.609469+0000","last_undegraded":"2024-04-23T19:12:12.609469+0000","last_fullsized":"2024-04-23T19:12:12.609469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:35:39.248022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703799+0000","last_change":"2024-04-23T19:10:47.733611+0000","last_active":"2024-04-23T19:10:50.703799+0000","last_peered":"2024-04-23T19:10:50.703799+0000","last_clean":"2024-04-23T19:10:50.703799+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:10:50.703799+0000","last_undegraded":"2024-04-23T19:10:50.703799+0000","last_fullsized":"2024-04-23T19:10:50.703799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:04.349258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717131+0000","last_change":"2024-04-23T19:10:07.502477+0000","last_active":"2024-04-23T19:10:47.717131+0000","last_peered":"2024-04-23T19:10:47.717131+0000","last_clean":"2024-04-23T19:10:47.717131+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:10:47.717131+0000","last_undegraded":"2024-04-23T19:10:47.717131+0000","last_fullsized":"2024-04-23T19:10:47.717131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719460+0000","last_change":"2024-04-23T19:10:11.487445+0000","last_active":"2024-04-23T19:10:47.719460+0000","last_peered":"2024-04-23T19:10:47.719460+0000","last_clean":"2024-04-23T19:10:47.719460+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:10:47.719460+0000","last_undegraded":"2024-04-23T19:10:47.719460+0000","last_fullsized":"2024-04-23T19:10:47.719460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:56.771240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.321490+0000","last_change":"2024-04-23T19:10:13.498101+0000","last_active":"2024-04-23T19:12:14.321490+0000","last_peered":"2024-04-23T19:12:14.321490+0000","last_clean":"2024-04-23T19:12:14.321490+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:12:14.321490+0000","last_undegraded":"2024-04-23T19:12:14.321490+0000","last_fullsized":"2024-04-23T19:12:14.321490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729093+0000","last_change":"2024-04-23T19:10:15.504279+0000","last_active":"2024-04-23T19:10:47.729093+0000","last_peered":"2024-04-23T19:10:47.729093+0000","last_clean":"2024-04-23T19:10:47.729093+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:10:47.729093+0000","last_undegraded":"2024-04-23T19:10:47.729093+0000","last_fullsized":"2024-04-23T19:10:47.729093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:05.168473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.451981+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.451981+0000","last_peered":"2024-04-23T19:13:16.451981+0000","last_clean":"2024-04-23T19:13:16.451981+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.451981+0000","last_undegraded":"2024-04-23T19:13:16.451981+0000","last_fullsized":"2024-04-23T19:13:16.451981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:52.410520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399955+0000","last_change":"2024-04-23T19:10:09.477657+0000","last_active":"2024-04-23T19:13:16.399955+0000","last_peered":"2024-04-23T19:13:16.399955+0000","last_clean":"2024-04-23T19:13:16.399955+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:16.399955+0000","last_undegraded":"2024-04-23T19:13:16.399955+0000","last_fullsized":"2024-04-23T19:13:16.399955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.117950+0000","last_change":"2024-04-23T19:10:07.504027+0000","last_active":"2024-04-23T19:13:16.117950+0000","last_peered":"2024-04-23T19:13:16.117950+0000","last_clean":"2024-04-23T19:13:16.117950+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:13:16.117950+0000","last_undegraded":"2024-04-23T19:13:16.117950+0000","last_fullsized":"2024-04-23T19:13:16.117950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:34.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'4","reported_seq":226,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.374959+0000","last_change":"2024-04-23T19:10:11.485789+0000","last_active":"2024-04-23T19:12:15.374959+0000","last_peered":"2024-04-23T19:12:15.374959+0000","last_clean":"2024-04-23T19:12:15.374959+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:12:15.374959+0000","last_undegraded":"2024-04-23T19:12:15.374959+0000","last_fullsized":"2024-04-23T19:12:15.374959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:54.297420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728532+0000","last_change":"2024-04-23T19:10:13.502626+0000","last_active":"2024-04-23T19:10:47.728532+0000","last_peered":"2024-04-23T19:10:47.728532+0000","last_clean":"2024-04-23T19:10:47.728532+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:10:47.728532+0000","last_undegraded":"2024-04-23T19:10:47.728532+0000","last_fullsized":"2024-04-23T19:10:47.728532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:27.703823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'282","reported_seq":417,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575740+0000","last_change":"2024-04-23T19:10:15.510611+0000","last_active":"2024-04-23T19:13:16.575740+0000","last_peered":"2024-04-23T19:13:16.575740+0000","last_clean":"2024-04-23T19:13:16.575740+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:16.575740+0000","last_undegraded":"2024-04-23T19:13:16.575740+0000","last_fullsized":"2024-04-23T19:13:16.575740+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:00.157929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725003+0000","last_change":"2024-04-23T19:10:17.513830+0000","last_active":"2024-04-23T19:10:47.725003+0000","last_peered":"2024-04-23T19:10:47.725003+0000","last_clean":"2024-04-23T19:10:47.725003+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:10:47.725003+0000","last_undegraded":"2024-04-23T19:10:47.725003+0000","last_fullsized":"2024-04-23T19:10:47.725003+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.594849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.150535+0000","last_change":"2024-04-23T19:10:09.476601+0000","last_active":"2024-04-23T19:11:32.150535+0000","last_peered":"2024-04-23T19:11:32.150535+0000","last_clean":"2024-04-23T19:11:32.150535+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:11:32.150535+0000","last_undegraded":"2024-04-23T19:11:32.150535+0000","last_fullsized":"2024-04-23T19:11:32.150535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:10.224761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718929+0000","last_change":"2024-04-23T19:10:07.497992+0000","last_active":"2024-04-23T19:10:47.718929+0000","last_peered":"2024-04-23T19:10:47.718929+0000","last_clean":"2024-04-23T19:10:47.718929+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:10:47.718929+0000","last_undegraded":"2024-04-23T19:10:47.718929+0000","last_fullsized":"2024-04-23T19:10:47.718929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720669+0000","last_change":"2024-04-23T19:10:11.489334+0000","last_active":"2024-04-23T19:10:47.720669+0000","last_peered":"2024-04-23T19:10:47.720669+0000","last_clean":"2024-04-23T19:10:47.720669+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:10:47.720669+0000","last_undegraded":"2024-04-23T19:10:47.720669+0000","last_fullsized":"2024-04-23T19:10:47.720669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'37","reported_seq":82,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.294646+0000","last_change":"2024-04-23T19:10:13.492238+0000","last_active":"2024-04-23T19:12:14.294646+0000","last_peered":"2024-04-23T19:12:14.294646+0000","last_clean":"2024-04-23T19:12:14.294646+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:12:14.294646+0000","last_undegraded":"2024-04-23T19:12:14.294646+0000","last_fullsized":"2024-04-23T19:12:14.294646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726624+0000","last_change":"2024-04-23T19:10:15.502657+0000","last_active":"2024-04-23T19:10:47.726624+0000","last_peered":"2024-04-23T19:10:47.726624+0000","last_clean":"2024-04-23T19:10:47.726624+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:10:47.726624+0000","last_undegraded":"2024-04-23T19:10:47.726624+0000","last_fullsized":"2024-04-23T19:10:47.726624+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722842+0000","last_change":"2024-04-23T19:10:17.518482+0000","last_active":"2024-04-23T19:10:47.722842+0000","last_peered":"2024-04-23T19:10:47.722842+0000","last_clean":"2024-04-23T19:10:47.722842+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:10:47.722842+0000","last_undegraded":"2024-04-23T19:10:47.722842+0000","last_fullsized":"2024-04-23T19:10:47.722842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'3","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.405706+0000","last_change":"2024-04-23T19:10:09.477644+0000","last_active":"2024-04-23T19:13:16.405706+0000","last_peered":"2024-04-23T19:13:16.405706+0000","last_clean":"2024-04-23T19:13:16.405706+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:16.405706+0000","last_undegraded":"2024-04-23T19:13:16.405706+0000","last_fullsized":"2024-04-23T19:13:16.405706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728885+0000","last_change":"2024-04-23T19:10:07.503802+0000","last_active":"2024-04-23T19:10:47.728885+0000","last_peered":"2024-04-23T19:10:47.728885+0000","last_clean":"2024-04-23T19:10:47.728885+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:10:47.728885+0000","last_undegraded":"2024-04-23T19:10:47.728885+0000","last_fullsized":"2024-04-23T19:10:47.728885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:03.446904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720608+0000","last_change":"2024-04-23T19:10:11.489940+0000","last_active":"2024-04-23T19:10:47.720608+0000","last_peered":"2024-04-23T19:10:47.720608+0000","last_clean":"2024-04-23T19:10:47.720608+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:10:47.720608+0000","last_undegraded":"2024-04-23T19:10:47.720608+0000","last_fullsized":"2024-04-23T19:10:47.720608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:35.423825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.295694+0000","last_change":"2024-04-23T19:10:13.505761+0000","last_active":"2024-04-23T19:12:14.295694+0000","last_peered":"2024-04-23T19:12:14.295694+0000","last_clean":"2024-04-23T19:12:14.295694+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:12:14.295694+0000","last_undegraded":"2024-04-23T19:12:14.295694+0000","last_fullsized":"2024-04-23T19:12:14.295694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:28.957267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.723783+0000","last_change":"2024-04-23T19:10:15.514687+0000","last_active":"2024-04-23T19:10:47.723783+0000","last_peered":"2024-04-23T19:10:47.723783+0000","last_clean":"2024-04-23T19:10:47.723783+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:10:47.723783+0000","last_undegraded":"2024-04-23T19:10:47.723783+0000","last_fullsized":"2024-04-23T19:10:47.723783+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:29.022733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.768829+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:15.768829+0000","last_peered":"2024-04-23T19:12:15.768829+0000","last_clean":"2024-04-23T19:12:15.768829+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:15.768829+0000","last_undegraded":"2024-04-23T19:12:15.768829+0000","last_fullsized":"2024-04-23T19:12:15.768829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:42.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'11","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165914+0000","last_change":"2024-04-23T19:10:09.490516+0000","last_active":"2024-04-23T19:13:16.165914+0000","last_peered":"2024-04-23T19:13:16.165914+0000","last_clean":"2024-04-23T19:13:16.165914+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:13:16.165914+0000","last_undegraded":"2024-04-23T19:13:16.165914+0000","last_fullsized":"2024-04-23T19:13:16.165914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:54.586490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729712+0000","last_change":"2024-04-23T19:10:07.486770+0000","last_active":"2024-04-23T19:10:47.729712+0000","last_peered":"2024-04-23T19:10:47.729712+0000","last_clean":"2024-04-23T19:10:47.729712+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:10:47.729712+0000","last_undegraded":"2024-04-23T19:10:47.729712+0000","last_fullsized":"2024-04-23T19:10:47.729712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:56:06.039657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717835+0000","last_change":"2024-04-23T19:10:11.491433+0000","last_active":"2024-04-23T19:10:47.717835+0000","last_peered":"2024-04-23T19:10:47.717835+0000","last_clean":"2024-04-23T19:10:47.717835+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:10:47.717835+0000","last_undegraded":"2024-04-23T19:10:47.717835+0000","last_fullsized":"2024-04-23T19:10:47.717835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'26","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.319314+0000","last_change":"2024-04-23T19:10:13.503113+0000","last_active":"2024-04-23T19:12:14.319314+0000","last_peered":"2024-04-23T19:12:14.319314+0000","last_clean":"2024-04-23T19:12:14.319314+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:12:14.319314+0000","last_undegraded":"2024-04-23T19:12:14.319314+0000","last_fullsized":"2024-04-23T19:12:14.319314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:25.616871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726203+0000","last_change":"2024-04-23T19:10:15.512937+0000","last_active":"2024-04-23T19:10:47.726203+0000","last_peered":"2024-04-23T19:10:47.726203+0000","last_clean":"2024-04-23T19:10:47.726203+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:10:47.726203+0000","last_undegraded":"2024-04-23T19:10:47.726203+0000","last_fullsized":"2024-04-23T19:10:47.726203+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:07.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.329498+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:14.329498+0000","last_peered":"2024-04-23T19:12:14.329498+0000","last_clean":"2024-04-23T19:12:14.329498+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:14.329498+0000","last_undegraded":"2024-04-23T19:12:14.329498+0000","last_fullsized":"2024-04-23T19:12:14.329498+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:30.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482420+0000","last_change":"2024-04-23T19:10:09.476153+0000","last_active":"2024-04-23T19:13:16.482420+0000","last_peered":"2024-04-23T19:13:16.482420+0000","last_clean":"2024-04-23T19:13:16.482420+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:16.482420+0000","last_undegraded":"2024-04-23T19:13:16.482420+0000","last_fullsized":"2024-04-23T19:13:16.482420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:07.680114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.716957+0000","last_change":"2024-04-23T19:10:07.501211+0000","last_active":"2024-04-23T19:12:11.716957+0000","last_peered":"2024-04-23T19:12:11.716957+0000","last_clean":"2024-04-23T19:12:11.716957+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:12:11.716957+0000","last_undegraded":"2024-04-23T19:12:11.716957+0000","last_fullsized":"2024-04-23T19:12:11.716957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'2","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.979180+0000","last_change":"2024-04-23T19:10:11.491220+0000","last_active":"2024-04-23T19:12:12.979180+0000","last_peered":"2024-04-23T19:12:12.979180+0000","last_clean":"2024-04-23T19:12:12.979180+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:12:12.979180+0000","last_undegraded":"2024-04-23T19:12:12.979180+0000","last_fullsized":"2024-04-23T19:12:12.979180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:56.767918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'32","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789943+0000","last_change":"2024-04-23T19:10:13.502743+0000","last_active":"2024-04-23T19:13:15.789943+0000","last_peered":"2024-04-23T19:13:15.789943+0000","last_clean":"2024-04-23T19:13:15.789943+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:15.789943+0000","last_undegraded":"2024-04-23T19:13:15.789943+0000","last_fullsized":"2024-04-23T19:13:15.789943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:13.230238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727977+0000","last_change":"2024-04-23T19:10:15.502314+0000","last_active":"2024-04-23T19:10:47.727977+0000","last_peered":"2024-04-23T19:10:47.727977+0000","last_clean":"2024-04-23T19:10:47.727977+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:10:47.727977+0000","last_undegraded":"2024-04-23T19:10:47.727977+0000","last_fullsized":"2024-04-23T19:10:47.727977+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:26.867666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.434484+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.434484+0000","last_peered":"2024-04-23T19:13:16.434484+0000","last_clean":"2024-04-23T19:13:16.434484+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.434484+0000","last_undegraded":"2024-04-23T19:13:16.434484+0000","last_fullsized":"2024-04-23T19:13:16.434484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:47.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489162+0000","last_change":"2024-04-23T19:10:09.494269+0000","last_active":"2024-04-23T19:13:16.489162+0000","last_peered":"2024-04-23T19:13:16.489162+0000","last_clean":"2024-04-23T19:13:16.489162+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:16.489162+0000","last_undegraded":"2024-04-23T19:13:16.489162+0000","last_fullsized":"2024-04-23T19:13:16.489162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:25.844962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.477833+0000","last_change":"2024-04-23T19:10:07.492825+0000","last_active":"2024-04-23T19:13:16.477833+0000","last_peered":"2024-04-23T19:13:16.477833+0000","last_clean":"2024-04-23T19:13:16.477833+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:13:16.477833+0000","last_undegraded":"2024-04-23T19:13:16.477833+0000","last_fullsized":"2024-04-23T19:13:16.477833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771507+0000","last_change":"2024-04-23T19:10:09.478781+0000","last_active":"2024-04-23T19:12:15.771507+0000","last_peered":"2024-04-23T19:12:15.771507+0000","last_clean":"2024-04-23T19:12:15.771507+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:12:15.771507+0000","last_undegraded":"2024-04-23T19:12:15.771507+0000","last_fullsized":"2024-04-23T19:12:15.771507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710865+0000","last_change":"2024-04-23T19:10:07.489213+0000","last_active":"2024-04-23T19:12:15.710865+0000","last_peered":"2024-04-23T19:12:15.710865+0000","last_clean":"2024-04-23T19:12:15.710865+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:12:15.710865+0000","last_undegraded":"2024-04-23T19:12:15.710865+0000","last_fullsized":"2024-04-23T19:12:15.710865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.495393+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:17.602017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726729+0000","last_change":"2024-04-23T19:10:07.506697+0000","last_active":"2024-04-23T19:10:47.726729+0000","last_peered":"2024-04-23T19:10:47.726729+0000","last_clean":"2024-04-23T19:10:47.726729+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:10:47.726729+0000","last_undegraded":"2024-04-23T19:10:47.726729+0000","last_fullsized":"2024-04-23T19:10:47.726729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:36.637883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.158375+0000","last_change":"2024-04-23T19:10:09.478697+0000","last_active":"2024-04-23T19:13:16.158375+0000","last_peered":"2024-04-23T19:13:16.158375+0000","last_clean":"2024-04-23T19:13:16.158375+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:13:16.158375+0000","last_undegraded":"2024-04-23T19:13:16.158375+0000","last_fullsized":"2024-04-23T19:13:16.158375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'3","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710919+0000","last_change":"2024-04-23T19:10:07.490623+0000","last_active":"2024-04-23T19:12:15.710919+0000","last_peered":"2024-04-23T19:12:15.710919+0000","last_clean":"2024-04-23T19:12:15.710919+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:12:15.710919+0000","last_undegraded":"2024-04-23T19:12:15.710919+0000","last_fullsized":"2024-04-23T19:12:15.710919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698550+0000","last_change":"2024-04-23T19:10:09.491248+0000","last_active":"2024-04-23T19:10:49.698550+0000","last_peered":"2024-04-23T19:10:49.698550+0000","last_clean":"2024-04-23T19:10:49.698550+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:10:49.698550+0000","last_undegraded":"2024-04-23T19:10:49.698550+0000","last_fullsized":"2024-04-23T19:10:49.698550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729029+0000","last_change":"2024-04-23T19:10:07.490715+0000","last_active":"2024-04-23T19:10:47.729029+0000","last_peered":"2024-04-23T19:10:47.729029+0000","last_clean":"2024-04-23T19:10:47.729029+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:10:47.729029+0000","last_undegraded":"2024-04-23T19:10:47.729029+0000","last_fullsized":"2024-04-23T19:10:47.729029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:03.712798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447089+0000","last_change":"2024-04-23T19:10:09.494954+0000","last_active":"2024-04-23T19:13:16.447089+0000","last_peered":"2024-04-23T19:13:16.447089+0000","last_clean":"2024-04-23T19:13:16.447089+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:16.447089+0000","last_undegraded":"2024-04-23T19:13:16.447089+0000","last_fullsized":"2024-04-23T19:13:16.447089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.351485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.548131+0000","last_change":"2024-04-23T19:10:07.490753+0000","last_active":"2024-04-23T19:10:48.548131+0000","last_peered":"2024-04-23T19:10:48.548131+0000","last_clean":"2024-04-23T19:10:48.548131+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:10:48.548131+0000","last_undegraded":"2024-04-23T19:10:48.548131+0000","last_fullsized":"2024-04-23T19:10:48.548131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.157844+0000","last_change":"2024-04-23T19:10:09.482692+0000","last_active":"2024-04-23T19:13:16.157844+0000","last_peered":"2024-04-23T19:13:16.157844+0000","last_clean":"2024-04-23T19:13:16.157844+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:13:16.157844+0000","last_undegraded":"2024-04-23T19:13:16.157844+0000","last_fullsized":"2024-04-23T19:13:16.157844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.380363+0000","last_change":"2024-04-23T19:10:07.500499+0000","last_active":"2024-04-23T19:13:16.380363+0000","last_peered":"2024-04-23T19:13:16.380363+0000","last_clean":"2024-04-23T19:13:16.380363+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:16.380363+0000","last_undegraded":"2024-04-23T19:13:16.380363+0000","last_fullsized":"2024-04-23T19:13:16.380363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258229+0000","last_change":"2024-04-23T19:10:09.484981+0000","last_active":"2024-04-23T19:13:16.258229+0000","last_peered":"2024-04-23T19:13:16.258229+0000","last_clean":"2024-04-23T19:13:16.258229+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:13:16.258229+0000","last_undegraded":"2024-04-23T19:13:16.258229+0000","last_fullsized":"2024-04-23T19:13:16.258229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.951884+0000","last_change":"2024-04-23T19:10:07.490970+0000","last_active":"2024-04-23T19:11:31.951884+0000","last_peered":"2024-04-23T19:11:31.951884+0000","last_clean":"2024-04-23T19:11:31.951884+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:11:31.951884+0000","last_undegraded":"2024-04-23T19:11:31.951884+0000","last_fullsized":"2024-04-23T19:11:31.951884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:19.094583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698565+0000","last_change":"2024-04-23T19:10:09.484268+0000","last_active":"2024-04-23T19:10:49.698565+0000","last_peered":"2024-04-23T19:10:49.698565+0000","last_clean":"2024-04-23T19:10:49.698565+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:10:49.698565+0000","last_undegraded":"2024-04-23T19:10:49.698565+0000","last_fullsized":"2024-04-23T19:10:49.698565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:46.058996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726273+0000","last_change":"2024-04-23T19:10:07.507068+0000","last_active":"2024-04-23T19:10:47.726273+0000","last_peered":"2024-04-23T19:10:47.726273+0000","last_clean":"2024-04-23T19:10:47.726273+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:10:47.726273+0000","last_undegraded":"2024-04-23T19:10:47.726273+0000","last_fullsized":"2024-04-23T19:10:47.726273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.361176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489138+0000","last_change":"2024-04-23T19:10:09.498436+0000","last_active":"2024-04-23T19:13:16.489138+0000","last_peered":"2024-04-23T19:13:16.489138+0000","last_clean":"2024-04-23T19:13:16.489138+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:13:16.489138+0000","last_undegraded":"2024-04-23T19:13:16.489138+0000","last_fullsized":"2024-04-23T19:13:16.489138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:55.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.685736+0000","last_change":"2024-04-23T19:10:07.489441+0000","last_active":"2024-04-23T19:10:49.685736+0000","last_peered":"2024-04-23T19:10:49.685736+0000","last_clean":"2024-04-23T19:10:49.685736+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:10:49.685736+0000","last_undegraded":"2024-04-23T19:10:49.685736+0000","last_fullsized":"2024-04-23T19:10:49.685736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'5","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712408+0000","last_change":"2024-04-23T19:10:09.492013+0000","last_active":"2024-04-23T19:12:25.712408+0000","last_peered":"2024-04-23T19:12:25.712408+0000","last_clean":"2024-04-23T19:12:25.712408+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:12:25.712408+0000","last_undegraded":"2024-04-23T19:12:25.712408+0000","last_fullsized":"2024-04-23T19:12:25.712408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717325+0000","last_change":"2024-04-23T19:10:07.490880+0000","last_active":"2024-04-23T19:10:47.717325+0000","last_peered":"2024-04-23T19:10:47.717325+0000","last_clean":"2024-04-23T19:10:47.717325+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:10:47.717325+0000","last_undegraded":"2024-04-23T19:10:47.717325+0000","last_fullsized":"2024-04-23T19:10:47.717325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.697786+0000","last_change":"2024-04-23T19:10:09.482277+0000","last_active":"2024-04-23T19:10:49.697786+0000","last_peered":"2024-04-23T19:10:49.697786+0000","last_clean":"2024-04-23T19:10:49.697786+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:10:49.697786+0000","last_undegraded":"2024-04-23T19:10:49.697786+0000","last_fullsized":"2024-04-23T19:10:49.697786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719034+0000","last_change":"2024-04-23T19:10:07.502896+0000","last_active":"2024-04-23T19:10:47.719034+0000","last_peered":"2024-04-23T19:10:47.719034+0000","last_clean":"2024-04-23T19:10:47.719034+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:10:47.719034+0000","last_undegraded":"2024-04-23T19:10:47.719034+0000","last_fullsized":"2024-04-23T19:10:47.719034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.964394+0000","last_change":"2024-04-23T19:10:09.492153+0000","last_active":"2024-04-23T19:11:31.964394+0000","last_peered":"2024-04-23T19:11:31.964394+0000","last_clean":"2024-04-23T19:11:31.964394+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:11:31.964394+0000","last_undegraded":"2024-04-23T19:11:31.964394+0000","last_fullsized":"2024-04-23T19:11:31.964394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.490658+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.568491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720741+0000","last_change":"2024-04-23T19:10:09.483025+0000","last_active":"2024-04-23T19:10:47.720741+0000","last_peered":"2024-04-23T19:10:47.720741+0000","last_clean":"2024-04-23T19:10:47.720741+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:10:47.720741+0000","last_undegraded":"2024-04-23T19:10:47.720741+0000","last_fullsized":"2024-04-23T19:10:47.720741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:47.851863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729169+0000","last_change":"2024-04-23T19:10:07.502808+0000","last_active":"2024-04-23T19:10:47.729169+0000","last_peered":"2024-04-23T19:10:47.729169+0000","last_clean":"2024-04-23T19:10:47.729169+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:10:47.729169+0000","last_undegraded":"2024-04-23T19:10:47.729169+0000","last_fullsized":"2024-04-23T19:10:47.729169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:11.223317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'5","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719983+0000","last_change":"2024-04-23T19:10:09.495999+0000","last_active":"2024-04-23T19:12:26.719983+0000","last_peered":"2024-04-23T19:12:26.719983+0000","last_clean":"2024-04-23T19:12:26.719983+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:12:26.719983+0000","last_undegraded":"2024-04-23T19:12:26.719983+0000","last_fullsized":"2024-04-23T19:12:26.719983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:49.818415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725442+0000","last_change":"2024-04-23T19:10:07.500476+0000","last_active":"2024-04-23T19:10:47.725442+0000","last_peered":"2024-04-23T19:10:47.725442+0000","last_clean":"2024-04-23T19:10:47.725442+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:10:47.725442+0000","last_undegraded":"2024-04-23T19:10:47.725442+0000","last_fullsized":"2024-04-23T19:10:47.725442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258427+0000","last_change":"2024-04-23T19:10:09.489995+0000","last_active":"2024-04-23T19:13:16.258427+0000","last_peered":"2024-04-23T19:13:16.258427+0000","last_clean":"2024-04-23T19:13:16.258427+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:13:16.258427+0000","last_undegraded":"2024-04-23T19:13:16.258427+0000","last_fullsized":"2024-04-23T19:13:16.258427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728417+0000","last_change":"2024-04-23T19:10:07.507910+0000","last_active":"2024-04-23T19:10:47.728417+0000","last_peered":"2024-04-23T19:10:47.728417+0000","last_clean":"2024-04-23T19:10:47.728417+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:10:47.728417+0000","last_undegraded":"2024-04-23T19:10:47.728417+0000","last_fullsized":"2024-04-23T19:10:47.728417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:27.025595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.503476+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:39.006602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399931+0000","last_change":"2024-04-23T19:10:09.487033+0000","last_active":"2024-04-23T19:13:16.399931+0000","last_peered":"2024-04-23T19:13:16.399931+0000","last_clean":"2024-04-23T19:13:16.399931+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:13:16.399931+0000","last_undegraded":"2024-04-23T19:13:16.399931+0000","last_fullsized":"2024-04-23T19:13:16.399931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:36.162060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.392374+0000","last_change":"2024-04-23T19:10:07.486697+0000","last_active":"2024-04-23T19:13:16.392374+0000","last_peered":"2024-04-23T19:13:16.392374+0000","last_clean":"2024-04-23T19:13:16.392374+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:13:16.392374+0000","last_undegraded":"2024-04-23T19:13:16.392374+0000","last_fullsized":"2024-04-23T19:13:16.392374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.159090+0000","last_change":"2024-04-23T19:10:09.491506+0000","last_active":"2024-04-23T19:13:16.159090+0000","last_peered":"2024-04-23T19:13:16.159090+0000","last_clean":"2024-04-23T19:13:16.159090+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:16.159090+0000","last_undegraded":"2024-04-23T19:13:16.159090+0000","last_fullsized":"2024-04-23T19:13:16.159090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13384,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":38719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673007,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46348,"kb_used_data":11012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325492,"statfs":{"total":96636764160,"available":96589303808,"internally_reserved":0,"allocated":11276288,"data_stored":10955671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673006,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44844,"kb_used_data":9508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326996,"statfs":{"total":96636764160,"available":96590843904,"internally_reserved":0,"allocated":9736192,"data_stored":9446289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46552,"kb_used_data":11216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325288,"statfs":{"total":96636764160,"available":96589094912,"internally_reserved":0,"allocated":11485184,"data_stored":11220394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40100,"kb_used_data":8924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331740,"statfs":{"total":96636764160,"available":96595701760,"internally_reserved":0,"allocated":9138176,"data_stored":8856588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10809344,"data_stored":10809344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:13:23.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:13:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.667+0000 7f289019d640 1 Processor -- start 2024-04-23T19:13:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.667+0000 7f289019d640 1 -- start start 2024-04-23T19:13:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f289019d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f288812b7a0 0x7f288812bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f289019d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288809aa00 con 0x7f288812c0c0 2024-04-23T19:13:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f289019d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f28880a20d0 con 0x7f288812b7a0 2024-04-23T19:13:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f288df39640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f288812b7a0 0x7f288812bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f288d738640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f288812c0c0 0x7f288812eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38530/0 (socket says 172.21.15.16:38530) 2024-04-23T19:13:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f288d738640 1 -- 172.21.15.16:0/4081435101 learned_addr learned my addr 172.21.15.16:0/4081435101 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f288df39640 1 -- 172.21.15.16:0/4081435101 >> v1:172.21.15.16:6789/0 conn(0x7f288812c0c0 legacy=0x7f288812eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.668+0000 7f288df39640 1 -- 172.21.15.16:0/4081435101 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288809af50 con 0x7f288812b7a0 2024-04-23T19:13:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.669+0000 7f288df39640 1 --2- 172.21.15.16:0/4081435101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f288812bb80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2878009720 tx=0x7f287802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f86da247e9e1ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.669+0000 7f288cf37640 1 -- 172.21.15.16:0/4081435101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28780316d0 con 0x7f288812b7a0 2024-04-23T19:13:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.669+0000 7f288cf37640 1 -- 172.21.15.16:0/4081435101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2878037070 con 0x7f288812b7a0 2024-04-23T19:13:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.669+0000 7f288cf37640 1 -- 172.21.15.16:0/4081435101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28780319c0 con 0x7f288812b7a0 2024-04-23T19:13:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.670+0000 7f289019d640 1 -- 172.21.15.16:0/4081435101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 msgr2=0x7f288812bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.670+0000 7f289019d640 1 --2- 172.21.15.16:0/4081435101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f288812bb80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2878009720 tx=0x7f287802b840 comp rx=0 tx=0).stop 2024-04-23T19:13:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.670+0000 7f289019d640 1 -- 172.21.15.16:0/4081435101 shutdown_connections 2024-04-23T19:13:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.670+0000 7f289019d640 1 --2- 172.21.15.16:0/4081435101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f288812bb80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.670+0000 7f289019d640 1 -- 172.21.15.16:0/4081435101 >> 172.21.15.16:0/4081435101 conn(0x7f28880a15b0 msgr2=0x7f28880a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.670+0000 7f289019d640 1 -- 172.21.15.16:0/4081435101 shutdown_connections 2024-04-23T19:13:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.670+0000 7f289019d640 1 -- 172.21.15.16:0/4081435101 wait complete. 2024-04-23T19:13:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.671+0000 7f289019d640 1 Processor -- start 2024-04-23T19:13:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.671+0000 7f289019d640 1 -- start start 2024-04-23T19:13:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.671+0000 7f289019d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f2888070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.671+0000 7f289019d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f288812c500 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.671+0000 7f288df39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f2888070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.671+0000 7f288df39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f2888070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38974/0 (socket says 172.21.15.16:38974) 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.671+0000 7f288df39640 1 -- 172.21.15.16:0/2619899407 learned_addr learned my addr 172.21.15.16:0/2619899407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.672+0000 7f288df39640 1 -- 172.21.15.16:0/2619899407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2888071e40 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.672+0000 7f288df39640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f2888070410 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2878009720 tx=0x7f28780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.672+0000 7f287e7fc640 1 -- 172.21.15.16:0/2619899407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2878032770 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.672+0000 7f287e7fc640 1 -- 172.21.15.16:0/2619899407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2878032910 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.672+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288808fad0 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.672+0000 7f287e7fc640 1 -- 172.21.15.16:0/2619899407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2878032bf0 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.672+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2888070950 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.673+0000 7f287e7fc640 1 -- 172.21.15.16:0/2619899407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f287804c060 con 0x7f288812b7a0 2024-04-23T19:13:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.673+0000 7f287e7fc640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f286c0385c0 0x7f286c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.673+0000 7f287e7fc640 1 -- 172.21.15.16:0/2619899407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f287802f050 con 0x7f288812b7a0 2024-04-23T19:13:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.673+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2888063660 con 0x7f288812b7a0 2024-04-23T19:13:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.673+0000 7f288d738640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f286c0385c0 0x7f286c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.674+0000 7f288d738640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f286c0385c0 0x7f286c03aa80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2884007fd0 tx=0x7f2884007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.678+0000 7f287e7fc640 1 -- 172.21.15.16:0/2619899407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2878040070 con 0x7f288812b7a0 2024-04-23T19:13:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.862+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2888067f20 con 0x7f286c0385c0 2024-04-23T19:13:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.878+0000 7f287e7fc640 1 -- 172.21.15.16:0/2619899407 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665664 (secure 0 0 0) 0x7f2888067f20 con 0x7f286c0385c0 2024-04-23T19:13:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:13:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f286c0385c0 msgr2=0x7f286c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f286c0385c0 0x7f286c03aa80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2884007fd0 tx=0x7f2884007ac0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 msgr2=0x7f2888070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f2888070410 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2878009720 tx=0x7f28780046c0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 shutdown_connections 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f286c0385c0 0x7f286c03aa80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 --2- 172.21.15.16:0/2619899407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288812b7a0 0x7f2888070410 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 >> 172.21.15.16:0/2619899407 conn(0x7f28880a15b0 msgr2=0x7f288809bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 shutdown_connections 2024-04-23T19:13:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:23.883+0000 7f289019d640 1 -- 172.21.15.16:0/2619899407 wait complete. 2024-04-23T19:13:23.899 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-04-23T19:13:22.700633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412789,"num_objects":227,"num_object_clones":0,"num_object_copies":461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":4876,"num_read_kb":4453,"num_write":2104,"num_write_kb":61372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4476,"ondisk_log_size":4476,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":177844,"kb_used_data":40660,"kb_used_omap":6,"kb_used_meta":137145,"kb_avail":377309516,"statfs":{"total":386547056640,"available":386364944384,"internally_reserved":0,"allocated":41635840,"data_stored":40478942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":140437271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811687,"num_objects":9,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003607"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722031+0000","last_change":"2024-04-23T19:10:07.492726+0000","last_active":"2024-04-23T19:10:47.722031+0000","last_peered":"2024-04-23T19:10:47.722031+0000","last_clean":"2024-04-23T19:10:47.722031+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:10:47.722031+0000","last_undegraded":"2024-04-23T19:10:47.722031+0000","last_fullsized":"2024-04-23T19:10:47.722031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771923+0000","last_change":"2024-04-23T19:10:09.485943+0000","last_active":"2024-04-23T19:12:15.771923+0000","last_peered":"2024-04-23T19:12:15.771923+0000","last_clean":"2024-04-23T19:12:15.771923+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:12:15.771923+0000","last_undegraded":"2024-04-23T19:12:15.771923+0000","last_fullsized":"2024-04-23T19:12:15.771923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729774+0000","last_change":"2024-04-23T19:10:07.490929+0000","last_active":"2024-04-23T19:10:47.729774+0000","last_peered":"2024-04-23T19:10:47.729774+0000","last_clean":"2024-04-23T19:10:47.729774+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:10:47.729774+0000","last_undegraded":"2024-04-23T19:10:47.729774+0000","last_fullsized":"2024-04-23T19:10:47.729774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:43.016228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.498665+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:54.763728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729513+0000","last_change":"2024-04-23T19:10:07.508341+0000","last_active":"2024-04-23T19:10:47.729513+0000","last_peered":"2024-04-23T19:10:47.729513+0000","last_clean":"2024-04-23T19:10:47.729513+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:10:47.729513+0000","last_undegraded":"2024-04-23T19:10:47.729513+0000","last_fullsized":"2024-04-23T19:10:47.729513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:50.865140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771899+0000","last_change":"2024-04-23T19:10:09.498971+0000","last_active":"2024-04-23T19:12:15.771899+0000","last_peered":"2024-04-23T19:12:15.771899+0000","last_clean":"2024-04-23T19:12:15.771899+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:12:15.771899+0000","last_undegraded":"2024-04-23T19:12:15.771899+0000","last_fullsized":"2024-04-23T19:12:15.771899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:58:56.677184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719199+0000","last_change":"2024-04-23T19:10:07.490859+0000","last_active":"2024-04-23T19:10:47.719199+0000","last_peered":"2024-04-23T19:10:47.719199+0000","last_clean":"2024-04-23T19:10:47.719199+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:10:47.719199+0000","last_undegraded":"2024-04-23T19:10:47.719199+0000","last_fullsized":"2024-04-23T19:10:47.719199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447391+0000","last_change":"2024-04-23T19:10:09.489556+0000","last_active":"2024-04-23T19:13:16.447391+0000","last_peered":"2024-04-23T19:13:16.447391+0000","last_clean":"2024-04-23T19:13:16.447391+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:13:16.447391+0000","last_undegraded":"2024-04-23T19:13:16.447391+0000","last_fullsized":"2024-04-23T19:13:16.447391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:55.937097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.675532+0000","last_change":"2024-04-23T19:10:07.504507+0000","last_active":"2024-04-23T19:12:15.675532+0000","last_peered":"2024-04-23T19:12:15.675532+0000","last_clean":"2024-04-23T19:12:15.675532+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:12:15.675532+0000","last_undegraded":"2024-04-23T19:12:15.675532+0000","last_fullsized":"2024-04-23T19:12:15.675532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:05.672303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.764033+0000","last_change":"2024-04-23T19:10:09.482365+0000","last_active":"2024-04-23T19:12:15.764033+0000","last_peered":"2024-04-23T19:12:15.764033+0000","last_clean":"2024-04-23T19:12:15.764033+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:12:15.764033+0000","last_undegraded":"2024-04-23T19:12:15.764033+0000","last_fullsized":"2024-04-23T19:12:15.764033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719720+0000","last_change":"2024-04-23T19:10:07.490892+0000","last_active":"2024-04-23T19:12:40.719720+0000","last_peered":"2024-04-23T19:12:40.719720+0000","last_clean":"2024-04-23T19:12:40.719720+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:12:40.719720+0000","last_undegraded":"2024-04-23T19:12:40.719720+0000","last_fullsized":"2024-04-23T19:12:40.719720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258616+0000","last_change":"2024-04-23T19:10:09.499345+0000","last_active":"2024-04-23T19:13:16.258616+0000","last_peered":"2024-04-23T19:13:16.258616+0000","last_clean":"2024-04-23T19:13:16.258616+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:16.258616+0000","last_undegraded":"2024-04-23T19:13:16.258616+0000","last_fullsized":"2024-04-23T19:13:16.258616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:28.869634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724671+0000","last_change":"2024-04-23T19:10:07.489309+0000","last_active":"2024-04-23T19:10:47.724671+0000","last_peered":"2024-04-23T19:10:47.724671+0000","last_clean":"2024-04-23T19:10:47.724671+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:10:47.724671+0000","last_undegraded":"2024-04-23T19:10:47.724671+0000","last_fullsized":"2024-04-23T19:10:47.724671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:18.611845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'23","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458768+0000","last_change":"2024-04-23T19:10:09.492130+0000","last_active":"2024-04-23T19:13:16.458768+0000","last_peered":"2024-04-23T19:13:16.458768+0000","last_clean":"2024-04-23T19:13:16.458768+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:16.458768+0000","last_undegraded":"2024-04-23T19:13:16.458768+0000","last_fullsized":"2024-04-23T19:13:16.458768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.714552+0000","last_change":"2024-04-23T19:10:07.487170+0000","last_active":"2024-04-23T19:12:40.714552+0000","last_peered":"2024-04-23T19:12:40.714552+0000","last_clean":"2024-04-23T19:12:40.714552+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:12:40.714552+0000","last_undegraded":"2024-04-23T19:12:40.714552+0000","last_fullsized":"2024-04-23T19:12:40.714552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'17","reported_seq":72,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458912+0000","last_change":"2024-04-23T19:10:09.493592+0000","last_active":"2024-04-23T19:13:16.458912+0000","last_peered":"2024-04-23T19:13:16.458912+0000","last_clean":"2024-04-23T19:13:16.458912+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:13:16.458912+0000","last_undegraded":"2024-04-23T19:13:16.458912+0000","last_fullsized":"2024-04-23T19:13:16.458912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727109+0000","last_change":"2024-04-23T19:10:09.499521+0000","last_active":"2024-04-23T19:10:47.727109+0000","last_peered":"2024-04-23T19:10:47.727109+0000","last_clean":"2024-04-23T19:10:47.727109+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:10:47.727109+0000","last_undegraded":"2024-04-23T19:10:47.727109+0000","last_fullsized":"2024-04-23T19:10:47.727109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:05.408478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722404+0000","last_change":"2024-04-23T19:10:07.491914+0000","last_active":"2024-04-23T19:10:47.722404+0000","last_peered":"2024-04-23T19:10:47.722404+0000","last_clean":"2024-04-23T19:10:47.722404+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:10:47.722404+0000","last_undegraded":"2024-04-23T19:10:47.722404+0000","last_fullsized":"2024-04-23T19:10:47.722404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449380+0000","last_change":"2024-04-23T19:10:09.482719+0000","last_active":"2024-04-23T19:13:16.449380+0000","last_peered":"2024-04-23T19:13:16.449380+0000","last_clean":"2024-04-23T19:13:16.449380+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:13:16.449380+0000","last_undegraded":"2024-04-23T19:13:16.449380+0000","last_fullsized":"2024-04-23T19:13:16.449380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725054+0000","last_change":"2024-04-23T19:10:07.489933+0000","last_active":"2024-04-23T19:10:47.725054+0000","last_peered":"2024-04-23T19:10:47.725054+0000","last_clean":"2024-04-23T19:10:47.725054+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:10:47.725054+0000","last_undegraded":"2024-04-23T19:10:47.725054+0000","last_fullsized":"2024-04-23T19:10:47.725054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720219+0000","last_change":"2024-04-23T19:10:09.484959+0000","last_active":"2024-04-23T19:10:47.720219+0000","last_peered":"2024-04-23T19:10:47.720219+0000","last_clean":"2024-04-23T19:10:47.720219+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:10:47.720219+0000","last_undegraded":"2024-04-23T19:10:47.720219+0000","last_fullsized":"2024-04-23T19:10:47.720219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:54.767051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720449+0000","last_change":"2024-04-23T19:10:07.493585+0000","last_active":"2024-04-23T19:12:31.720449+0000","last_peered":"2024-04-23T19:12:31.720449+0000","last_clean":"2024-04-23T19:12:31.720449+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:12:31.720449+0000","last_undegraded":"2024-04-23T19:12:31.720449+0000","last_fullsized":"2024-04-23T19:12:31.720449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408189+0000","last_change":"2024-04-23T19:10:09.491221+0000","last_active":"2024-04-23T19:13:16.408189+0000","last_peered":"2024-04-23T19:13:16.408189+0000","last_clean":"2024-04-23T19:13:16.408189+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:16.408189+0000","last_undegraded":"2024-04-23T19:13:16.408189+0000","last_fullsized":"2024-04-23T19:13:16.408189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718838+0000","last_change":"2024-04-23T19:10:07.501878+0000","last_active":"2024-04-23T19:10:47.718838+0000","last_peered":"2024-04-23T19:10:47.718838+0000","last_clean":"2024-04-23T19:10:47.718838+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:10:47.718838+0000","last_undegraded":"2024-04-23T19:10:47.718838+0000","last_fullsized":"2024-04-23T19:10:47.718838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771581+0000","last_change":"2024-04-23T19:10:09.492916+0000","last_active":"2024-04-23T19:12:15.771581+0000","last_peered":"2024-04-23T19:12:15.771581+0000","last_clean":"2024-04-23T19:12:15.771581+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:12:15.771581+0000","last_undegraded":"2024-04-23T19:12:15.771581+0000","last_fullsized":"2024-04-23T19:12:15.771581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728570+0000","last_change":"2024-04-23T19:10:07.490910+0000","last_active":"2024-04-23T19:10:47.728570+0000","last_peered":"2024-04-23T19:10:47.728570+0000","last_clean":"2024-04-23T19:10:47.728570+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:10:47.728570+0000","last_undegraded":"2024-04-23T19:10:47.728570+0000","last_fullsized":"2024-04-23T19:10:47.728570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482419+0000","last_change":"2024-04-23T19:10:09.482527+0000","last_active":"2024-04-23T19:13:16.482419+0000","last_peered":"2024-04-23T19:13:16.482419+0000","last_clean":"2024-04-23T19:13:16.482419+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:16.482419+0000","last_undegraded":"2024-04-23T19:13:16.482419+0000","last_fullsized":"2024-04-23T19:13:16.482419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:46.899076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727215+0000","last_change":"2024-04-23T19:10:07.493956+0000","last_active":"2024-04-23T19:10:47.727215+0000","last_peered":"2024-04-23T19:10:47.727215+0000","last_clean":"2024-04-23T19:10:47.727215+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:10:47.727215+0000","last_undegraded":"2024-04-23T19:10:47.727215+0000","last_fullsized":"2024-04-23T19:10:47.727215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:20.896254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'35","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.422839+0000","last_change":"2024-04-23T19:10:17.514587+0000","last_active":"2024-04-23T19:13:16.422839+0000","last_peered":"2024-04-23T19:13:16.422839+0000","last_clean":"2024-04-23T19:13:16.422839+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:16.422839+0000","last_undegraded":"2024-04-23T19:13:16.422839+0000","last_fullsized":"2024-04-23T19:13:16.422839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:18.256132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.432503+0000","last_change":"2024-04-23T19:10:09.487433+0000","last_active":"2024-04-23T19:10:49.432503+0000","last_peered":"2024-04-23T19:10:49.432503+0000","last_clean":"2024-04-23T19:10:49.432503+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:10:49.432503+0000","last_undegraded":"2024-04-23T19:10:49.432503+0000","last_fullsized":"2024-04-23T19:10:49.432503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:36.758021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728416+0000","last_change":"2024-04-23T19:10:07.507346+0000","last_active":"2024-04-23T19:10:47.728416+0000","last_peered":"2024-04-23T19:10:47.728416+0000","last_clean":"2024-04-23T19:10:47.728416+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:10:47.728416+0000","last_undegraded":"2024-04-23T19:10:47.728416+0000","last_fullsized":"2024-04-23T19:10:47.728416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:42.849025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728325+0000","last_change":"2024-04-23T19:10:11.488429+0000","last_active":"2024-04-23T19:10:47.728325+0000","last_peered":"2024-04-23T19:10:47.728325+0000","last_clean":"2024-04-23T19:10:47.728325+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:10:47.728325+0000","last_undegraded":"2024-04-23T19:10:47.728325+0000","last_fullsized":"2024-04-23T19:10:47.728325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:50.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722656+0000","last_change":"2024-04-23T19:10:13.501588+0000","last_active":"2024-04-23T19:12:41.722656+0000","last_peered":"2024-04-23T19:12:41.722656+0000","last_clean":"2024-04-23T19:12:41.722656+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:12:41.722656+0000","last_undegraded":"2024-04-23T19:12:41.722656+0000","last_fullsized":"2024-04-23T19:12:41.722656+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:34.948969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722273+0000","last_change":"2024-04-23T19:10:15.509911+0000","last_active":"2024-04-23T19:10:47.722273+0000","last_peered":"2024-04-23T19:10:47.722273+0000","last_clean":"2024-04-23T19:10:47.722273+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:10:47.722273+0000","last_undegraded":"2024-04-23T19:10:47.722273+0000","last_fullsized":"2024-04-23T19:10:47.722273+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'9","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.539127+0000","last_change":"2024-04-23T19:10:17.514347+0000","last_active":"2024-04-23T19:13:16.539127+0000","last_peered":"2024-04-23T19:13:16.539127+0000","last_clean":"2024-04-23T19:13:16.539127+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:16.539127+0000","last_undegraded":"2024-04-23T19:13:16.539127+0000","last_fullsized":"2024-04-23T19:13:16.539127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:53.299426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399880+0000","last_change":"2024-04-23T19:10:09.496014+0000","last_active":"2024-04-23T19:13:16.399880+0000","last_peered":"2024-04-23T19:13:16.399880+0000","last_clean":"2024-04-23T19:13:16.399880+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:13:16.399880+0000","last_undegraded":"2024-04-23T19:13:16.399880+0000","last_fullsized":"2024-04-23T19:13:16.399880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:32.575973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720263+0000","last_change":"2024-04-23T19:10:07.492730+0000","last_active":"2024-04-23T19:10:47.720263+0000","last_peered":"2024-04-23T19:10:47.720263+0000","last_clean":"2024-04-23T19:10:47.720263+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:10:47.720263+0000","last_undegraded":"2024-04-23T19:10:47.720263+0000","last_fullsized":"2024-04-23T19:10:47.720263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"23'1","reported_seq":218,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.386033+0000","last_change":"2024-04-23T19:10:11.489225+0000","last_active":"2024-04-23T19:12:15.386033+0000","last_peered":"2024-04-23T19:12:15.386033+0000","last_clean":"2024-04-23T19:12:15.386033+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:12:15.386033+0000","last_undegraded":"2024-04-23T19:12:15.386033+0000","last_fullsized":"2024-04-23T19:12:15.386033+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:01.595709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"30'6","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721580+0000","last_change":"2024-04-23T19:10:13.491509+0000","last_active":"2024-04-23T19:10:47.721580+0000","last_peered":"2024-04-23T19:10:47.721580+0000","last_clean":"2024-04-23T19:10:47.721580+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:10:47.721580+0000","last_undegraded":"2024-04-23T19:10:47.721580+0000","last_fullsized":"2024-04-23T19:10:47.721580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:32.880512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720747+0000","last_change":"2024-04-23T19:10:15.506746+0000","last_active":"2024-04-23T19:10:47.720747+0000","last_peered":"2024-04-23T19:10:47.720747+0000","last_clean":"2024-04-23T19:10:47.720747+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:10:47.720747+0000","last_undegraded":"2024-04-23T19:10:47.720747+0000","last_fullsized":"2024-04-23T19:10:47.720747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726990+0000","last_change":"2024-04-23T19:10:17.518249+0000","last_active":"2024-04-23T19:10:47.726990+0000","last_peered":"2024-04-23T19:10:47.726990+0000","last_clean":"2024-04-23T19:10:47.726990+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:10:47.726990+0000","last_undegraded":"2024-04-23T19:10:47.726990+0000","last_fullsized":"2024-04-23T19:10:47.726990+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:58.051610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447281+0000","last_change":"2024-04-23T19:10:09.490234+0000","last_active":"2024-04-23T19:13:16.447281+0000","last_peered":"2024-04-23T19:13:16.447281+0000","last_clean":"2024-04-23T19:13:16.447281+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:13:16.447281+0000","last_undegraded":"2024-04-23T19:13:16.447281+0000","last_fullsized":"2024-04-23T19:13:16.447281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:54.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.389417+0000","last_change":"2024-04-23T19:10:07.504260+0000","last_active":"2024-04-23T19:13:16.389417+0000","last_peered":"2024-04-23T19:13:16.389417+0000","last_clean":"2024-04-23T19:13:16.389417+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:13:16.389417+0000","last_undegraded":"2024-04-23T19:13:16.389417+0000","last_fullsized":"2024-04-23T19:13:16.389417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:46.596307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730636+0000","last_change":"2024-04-23T19:10:11.493566+0000","last_active":"2024-04-23T19:10:47.730636+0000","last_peered":"2024-04-23T19:10:47.730636+0000","last_clean":"2024-04-23T19:10:47.730636+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:10:47.730636+0000","last_undegraded":"2024-04-23T19:10:47.730636+0000","last_fullsized":"2024-04-23T19:10:47.730636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718899+0000","last_change":"2024-04-23T19:10:13.502669+0000","last_active":"2024-04-23T19:12:35.718899+0000","last_peered":"2024-04-23T19:12:35.718899+0000","last_clean":"2024-04-23T19:12:35.718899+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:12:35.718899+0000","last_undegraded":"2024-04-23T19:12:35.718899+0000","last_fullsized":"2024-04-23T19:12:35.718899+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:02.466667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'282","reported_seq":385,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.598276+0000","last_change":"2024-04-23T19:10:15.515398+0000","last_active":"2024-04-23T19:13:16.598276+0000","last_peered":"2024-04-23T19:13:16.598276+0000","last_clean":"2024-04-23T19:13:16.598276+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:13:16.598276+0000","last_undegraded":"2024-04-23T19:13:16.598276+0000","last_fullsized":"2024-04-23T19:13:16.598276+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.433967+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:10:49.433967+0000","last_peered":"2024-04-23T19:10:49.433967+0000","last_clean":"2024-04-23T19:10:49.433967+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:10:49.433967+0000","last_undegraded":"2024-04-23T19:10:49.433967+0000","last_fullsized":"2024-04-23T19:10:49.433967+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:55.634563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482380+0000","last_change":"2024-04-23T19:10:09.487095+0000","last_active":"2024-04-23T19:13:16.482380+0000","last_peered":"2024-04-23T19:13:16.482380+0000","last_clean":"2024-04-23T19:13:16.482380+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:13:16.482380+0000","last_undegraded":"2024-04-23T19:13:16.482380+0000","last_fullsized":"2024-04-23T19:13:16.482380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716719+0000","last_change":"2024-04-23T19:10:07.504154+0000","last_active":"2024-04-23T19:10:47.716719+0000","last_peered":"2024-04-23T19:10:47.716719+0000","last_clean":"2024-04-23T19:10:47.716719+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:10:47.716719+0000","last_undegraded":"2024-04-23T19:10:47.716719+0000","last_fullsized":"2024-04-23T19:10:47.716719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:24.070928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728932+0000","last_change":"2024-04-23T19:10:11.488228+0000","last_active":"2024-04-23T19:10:47.728932+0000","last_peered":"2024-04-23T19:10:47.728932+0000","last_clean":"2024-04-23T19:10:47.728932+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:10:47.728932+0000","last_undegraded":"2024-04-23T19:10:47.728932+0000","last_fullsized":"2024-04-23T19:10:47.728932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:01.154683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719829+0000","last_change":"2024-04-23T19:10:13.505813+0000","last_active":"2024-04-23T19:12:40.719829+0000","last_peered":"2024-04-23T19:12:40.719829+0000","last_clean":"2024-04-23T19:12:40.719829+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:12:40.719829+0000","last_undegraded":"2024-04-23T19:12:40.719829+0000","last_fullsized":"2024-04-23T19:12:40.719829+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:39.996179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'282","reported_seq":401,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575901+0000","last_change":"2024-04-23T19:10:15.500511+0000","last_active":"2024-04-23T19:13:16.575901+0000","last_peered":"2024-04-23T19:13:16.575901+0000","last_clean":"2024-04-23T19:13:16.575901+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:16.575901+0000","last_undegraded":"2024-04-23T19:13:16.575901+0000","last_fullsized":"2024-04-23T19:13:16.575901+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:23.813091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449319+0000","last_change":"2024-04-23T19:10:47.730482+0000","last_active":"2024-04-23T19:13:16.449319+0000","last_peered":"2024-04-23T19:13:16.449319+0000","last_clean":"2024-04-23T19:13:16.449319+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:13:16.449319+0000","last_undegraded":"2024-04-23T19:13:16.449319+0000","last_fullsized":"2024-04-23T19:13:16.449319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:33.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545310+0000","last_change":"2024-04-23T19:10:07.492480+0000","last_active":"2024-04-23T19:10:48.545310+0000","last_peered":"2024-04-23T19:10:48.545310+0000","last_clean":"2024-04-23T19:10:48.545310+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:10:48.545310+0000","last_undegraded":"2024-04-23T19:10:48.545310+0000","last_fullsized":"2024-04-23T19:10:48.545310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716980+0000","last_change":"2024-04-23T19:10:11.490040+0000","last_active":"2024-04-23T19:10:47.716980+0000","last_peered":"2024-04-23T19:10:47.716980+0000","last_clean":"2024-04-23T19:10:47.716980+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:10:47.716980+0000","last_undegraded":"2024-04-23T19:10:47.716980+0000","last_fullsized":"2024-04-23T19:10:47.716980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:07.412668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'40","reported_seq":87,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.316535+0000","last_change":"2024-04-23T19:10:13.494728+0000","last_active":"2024-04-23T19:12:14.316535+0000","last_peered":"2024-04-23T19:12:14.316535+0000","last_clean":"2024-04-23T19:12:14.316535+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:12:14.316535+0000","last_undegraded":"2024-04-23T19:12:14.316535+0000","last_fullsized":"2024-04-23T19:12:14.316535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:56.304108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728369+0000","last_change":"2024-04-23T19:10:15.506637+0000","last_active":"2024-04-23T19:10:47.728369+0000","last_peered":"2024-04-23T19:10:47.728369+0000","last_clean":"2024-04-23T19:10:47.728369+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:10:47.728369+0000","last_undegraded":"2024-04-23T19:10:47.728369+0000","last_fullsized":"2024-04-23T19:10:47.728369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:13.464606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.971818+0000","last_change":"2024-04-23T19:10:17.519710+0000","last_active":"2024-04-23T19:11:31.971818+0000","last_peered":"2024-04-23T19:11:31.971818+0000","last_clean":"2024-04-23T19:11:31.971818+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:11:31.971818+0000","last_undegraded":"2024-04-23T19:11:31.971818+0000","last_fullsized":"2024-04-23T19:11:31.971818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:10.239585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'7","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.488606+0000","last_change":"2024-04-23T19:10:09.482665+0000","last_active":"2024-04-23T19:13:16.488606+0000","last_peered":"2024-04-23T19:13:16.488606+0000","last_clean":"2024-04-23T19:13:16.488606+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:16.488606+0000","last_undegraded":"2024-04-23T19:13:16.488606+0000","last_fullsized":"2024-04-23T19:13:16.488606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724246+0000","last_change":"2024-04-23T19:10:07.500938+0000","last_active":"2024-04-23T19:10:47.724246+0000","last_peered":"2024-04-23T19:10:47.724246+0000","last_clean":"2024-04-23T19:10:47.724246+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:10:47.724246+0000","last_undegraded":"2024-04-23T19:10:47.724246+0000","last_fullsized":"2024-04-23T19:10:47.724246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727117+0000","last_change":"2024-04-23T19:10:11.493617+0000","last_active":"2024-04-23T19:10:47.727117+0000","last_peered":"2024-04-23T19:10:47.727117+0000","last_clean":"2024-04-23T19:10:47.727117+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:10:47.727117+0000","last_undegraded":"2024-04-23T19:10:47.727117+0000","last_fullsized":"2024-04-23T19:10:47.727117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:32.061098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.296627+0000","last_change":"2024-04-23T19:10:13.501851+0000","last_active":"2024-04-23T19:12:14.296627+0000","last_peered":"2024-04-23T19:12:14.296627+0000","last_clean":"2024-04-23T19:12:14.296627+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:12:14.296627+0000","last_undegraded":"2024-04-23T19:12:14.296627+0000","last_fullsized":"2024-04-23T19:12:14.296627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:30.608779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727229+0000","last_change":"2024-04-23T19:10:15.502245+0000","last_active":"2024-04-23T19:10:47.727229+0000","last_peered":"2024-04-23T19:10:47.727229+0000","last_clean":"2024-04-23T19:10:47.727229+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:10:47.727229+0000","last_undegraded":"2024-04-23T19:10:47.727229+0000","last_fullsized":"2024-04-23T19:10:47.727229+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'5","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.261614+0000","last_change":"2024-04-23T19:10:17.518314+0000","last_active":"2024-04-23T19:13:16.261614+0000","last_peered":"2024-04-23T19:13:16.261614+0000","last_clean":"2024-04-23T19:13:16.261614+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:13:16.261614+0000","last_undegraded":"2024-04-23T19:13:16.261614+0000","last_fullsized":"2024-04-23T19:13:16.261614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:11.577457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.166247+0000","last_change":"2024-04-23T19:10:09.492229+0000","last_active":"2024-04-23T19:13:16.166247+0000","last_peered":"2024-04-23T19:13:16.166247+0000","last_clean":"2024-04-23T19:13:16.166247+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:13:16.166247+0000","last_undegraded":"2024-04-23T19:13:16.166247+0000","last_fullsized":"2024-04-23T19:13:16.166247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:36.861914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729281+0000","last_change":"2024-04-23T19:10:07.492637+0000","last_active":"2024-04-23T19:10:47.729281+0000","last_peered":"2024-04-23T19:10:47.729281+0000","last_clean":"2024-04-23T19:10:47.729281+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:10:47.729281+0000","last_undegraded":"2024-04-23T19:10:47.729281+0000","last_fullsized":"2024-04-23T19:10:47.729281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.154249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729542+0000","last_change":"2024-04-23T19:10:11.491503+0000","last_active":"2024-04-23T19:10:47.729542+0000","last_peered":"2024-04-23T19:10:47.729542+0000","last_clean":"2024-04-23T19:10:47.729542+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:10:47.729542+0000","last_undegraded":"2024-04-23T19:10:47.729542+0000","last_fullsized":"2024-04-23T19:10:47.729542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.318445+0000","last_change":"2024-04-23T19:10:13.508355+0000","last_active":"2024-04-23T19:12:14.318445+0000","last_peered":"2024-04-23T19:12:14.318445+0000","last_clean":"2024-04-23T19:12:14.318445+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:12:14.318445+0000","last_undegraded":"2024-04-23T19:12:14.318445+0000","last_fullsized":"2024-04-23T19:12:14.318445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:19.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'282","reported_seq":348,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.608082+0000","last_change":"2024-04-23T19:10:15.509684+0000","last_active":"2024-04-23T19:13:16.608082+0000","last_peered":"2024-04-23T19:13:16.608082+0000","last_clean":"2024-04-23T19:13:16.608082+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:16.608082+0000","last_undegraded":"2024-04-23T19:13:16.608082+0000","last_fullsized":"2024-04-23T19:13:16.608082+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728722+0000","last_change":"2024-04-23T19:10:17.520791+0000","last_active":"2024-04-23T19:10:47.728722+0000","last_peered":"2024-04-23T19:10:47.728722+0000","last_clean":"2024-04-23T19:10:47.728722+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:10:47.728722+0000","last_undegraded":"2024-04-23T19:10:47.728722+0000","last_fullsized":"2024-04-23T19:10:47.728722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:03.997750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399785+0000","last_change":"2024-04-23T19:10:09.476300+0000","last_active":"2024-04-23T19:13:16.399785+0000","last_peered":"2024-04-23T19:13:16.399785+0000","last_clean":"2024-04-23T19:13:16.399785+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:16.399785+0000","last_undegraded":"2024-04-23T19:13:16.399785+0000","last_fullsized":"2024-04-23T19:13:16.399785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:59.576302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726343+0000","last_change":"2024-04-23T19:10:07.501083+0000","last_active":"2024-04-23T19:10:47.726343+0000","last_peered":"2024-04-23T19:10:47.726343+0000","last_clean":"2024-04-23T19:10:47.726343+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:10:47.726343+0000","last_undegraded":"2024-04-23T19:10:47.726343+0000","last_fullsized":"2024-04-23T19:10:47.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:08.925701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727624+0000","last_change":"2024-04-23T19:10:11.493665+0000","last_active":"2024-04-23T19:10:47.727624+0000","last_peered":"2024-04-23T19:10:47.727624+0000","last_clean":"2024-04-23T19:10:47.727624+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:10:47.727624+0000","last_undegraded":"2024-04-23T19:10:47.727624+0000","last_fullsized":"2024-04-23T19:10:47.727624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:29.762026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'54","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.320262+0000","last_change":"2024-04-23T19:10:13.499017+0000","last_active":"2024-04-23T19:12:14.320262+0000","last_peered":"2024-04-23T19:12:14.320262+0000","last_clean":"2024-04-23T19:12:14.320262+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:12:14.320262+0000","last_undegraded":"2024-04-23T19:12:14.320262+0000","last_fullsized":"2024-04-23T19:12:14.320262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:42.419535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726896+0000","last_change":"2024-04-23T19:10:15.503053+0000","last_active":"2024-04-23T19:10:47.726896+0000","last_peered":"2024-04-23T19:10:47.726896+0000","last_clean":"2024-04-23T19:10:47.726896+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:10:47.726896+0000","last_undegraded":"2024-04-23T19:10:47.726896+0000","last_fullsized":"2024-04-23T19:10:47.726896+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'18","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.583903+0000","last_change":"2024-04-23T19:10:17.513990+0000","last_active":"2024-04-23T19:13:16.583903+0000","last_peered":"2024-04-23T19:13:16.583903+0000","last_clean":"2024-04-23T19:13:16.583903+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:13:16.583903+0000","last_undegraded":"2024-04-23T19:13:16.583903+0000","last_fullsized":"2024-04-23T19:13:16.583903+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:57.264911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258650+0000","last_change":"2024-04-23T19:10:09.490325+0000","last_active":"2024-04-23T19:13:16.258650+0000","last_peered":"2024-04-23T19:13:16.258650+0000","last_clean":"2024-04-23T19:13:16.258650+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:16.258650+0000","last_undegraded":"2024-04-23T19:13:16.258650+0000","last_fullsized":"2024-04-23T19:13:16.258650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728226+0000","last_change":"2024-04-23T19:10:07.490182+0000","last_active":"2024-04-23T19:10:47.728226+0000","last_peered":"2024-04-23T19:10:47.728226+0000","last_clean":"2024-04-23T19:10:47.728226+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:10:47.728226+0000","last_undegraded":"2024-04-23T19:10:47.728226+0000","last_fullsized":"2024-04-23T19:10:47.728226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719940+0000","last_change":"2024-04-23T19:10:11.490064+0000","last_active":"2024-04-23T19:10:47.719940+0000","last_peered":"2024-04-23T19:10:47.719940+0000","last_clean":"2024-04-23T19:10:47.719940+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:10:47.719940+0000","last_undegraded":"2024-04-23T19:10:47.719940+0000","last_fullsized":"2024-04-23T19:10:47.719940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:48.063186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.300676+0000","last_change":"2024-04-23T19:10:13.491370+0000","last_active":"2024-04-23T19:12:14.300676+0000","last_peered":"2024-04-23T19:12:14.300676+0000","last_clean":"2024-04-23T19:12:14.300676+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:12:14.300676+0000","last_undegraded":"2024-04-23T19:12:14.300676+0000","last_fullsized":"2024-04-23T19:12:14.300676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:59.208837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728086+0000","last_change":"2024-04-23T19:10:15.512494+0000","last_active":"2024-04-23T19:10:47.728086+0000","last_peered":"2024-04-23T19:10:47.728086+0000","last_clean":"2024-04-23T19:10:47.728086+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:10:47.728086+0000","last_undegraded":"2024-04-23T19:10:47.728086+0000","last_fullsized":"2024-04-23T19:10:47.728086+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:54.341417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718208+0000","last_change":"2024-04-23T19:10:17.511337+0000","last_active":"2024-04-23T19:12:16.718208+0000","last_peered":"2024-04-23T19:12:16.718208+0000","last_clean":"2024-04-23T19:12:16.718208+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:12:16.718208+0000","last_undegraded":"2024-04-23T19:12:16.718208+0000","last_fullsized":"2024-04-23T19:12:16.718208+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:31.868099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536682+0000","last_change":"2024-04-23T19:10:09.499760+0000","last_active":"2024-04-23T19:13:16.536682+0000","last_peered":"2024-04-23T19:13:16.536682+0000","last_clean":"2024-04-23T19:13:16.536682+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:16.536682+0000","last_undegraded":"2024-04-23T19:13:16.536682+0000","last_fullsized":"2024-04-23T19:13:16.536682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:43.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727856+0000","last_change":"2024-04-23T19:10:07.494969+0000","last_active":"2024-04-23T19:10:47.727856+0000","last_peered":"2024-04-23T19:10:47.727856+0000","last_clean":"2024-04-23T19:10:47.727856+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:10:47.727856+0000","last_undegraded":"2024-04-23T19:10:47.727856+0000","last_fullsized":"2024-04-23T19:10:47.727856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:47.276166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728157+0000","last_change":"2024-04-23T19:10:11.491444+0000","last_active":"2024-04-23T19:10:47.728157+0000","last_peered":"2024-04-23T19:10:47.728157+0000","last_clean":"2024-04-23T19:10:47.728157+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:10:47.728157+0000","last_undegraded":"2024-04-23T19:10:47.728157+0000","last_fullsized":"2024-04-23T19:10:47.728157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:00.459979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.327052+0000","last_change":"2024-04-23T19:10:13.493342+0000","last_active":"2024-04-23T19:12:14.327052+0000","last_peered":"2024-04-23T19:12:14.327052+0000","last_clean":"2024-04-23T19:12:14.327052+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:12:14.327052+0000","last_undegraded":"2024-04-23T19:12:14.327052+0000","last_fullsized":"2024-04-23T19:12:14.327052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:44.904145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'282","reported_seq":403,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.587980+0000","last_change":"2024-04-23T19:10:15.507087+0000","last_active":"2024-04-23T19:13:16.587980+0000","last_peered":"2024-04-23T19:13:16.587980+0000","last_clean":"2024-04-23T19:13:16.587980+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:16.587980+0000","last_undegraded":"2024-04-23T19:13:16.587980+0000","last_fullsized":"2024-04-23T19:13:16.587980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:48.999181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'12","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712278+0000","last_change":"2024-04-23T19:10:17.518152+0000","last_active":"2024-04-23T19:12:25.712278+0000","last_peered":"2024-04-23T19:12:25.712278+0000","last_clean":"2024-04-23T19:12:25.712278+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:12:25.712278+0000","last_undegraded":"2024-04-23T19:12:25.712278+0000","last_fullsized":"2024-04-23T19:12:25.712278+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399883+0000","last_change":"2024-04-23T19:10:09.493242+0000","last_active":"2024-04-23T19:13:16.399883+0000","last_peered":"2024-04-23T19:13:16.399883+0000","last_clean":"2024-04-23T19:13:16.399883+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:13:16.399883+0000","last_undegraded":"2024-04-23T19:13:16.399883+0000","last_fullsized":"2024-04-23T19:13:16.399883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.679903+0000","last_change":"2024-04-23T19:10:07.508434+0000","last_active":"2024-04-23T19:10:49.679903+0000","last_peered":"2024-04-23T19:10:49.679903+0000","last_clean":"2024-04-23T19:10:49.679903+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:10:49.679903+0000","last_undegraded":"2024-04-23T19:10:49.679903+0000","last_fullsized":"2024-04-23T19:10:49.679903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:44.765151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'3","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.385678+0000","last_change":"2024-04-23T19:10:11.491234+0000","last_active":"2024-04-23T19:12:15.385678+0000","last_peered":"2024-04-23T19:12:15.385678+0000","last_clean":"2024-04-23T19:12:15.385678+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:12:15.385678+0000","last_undegraded":"2024-04-23T19:12:15.385678+0000","last_fullsized":"2024-04-23T19:12:15.385678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:03.467829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727806+0000","last_change":"2024-04-23T19:10:13.501643+0000","last_active":"2024-04-23T19:10:47.727806+0000","last_peered":"2024-04-23T19:10:47.727806+0000","last_clean":"2024-04-23T19:10:47.727806+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:10:47.727806+0000","last_undegraded":"2024-04-23T19:10:47.727806+0000","last_fullsized":"2024-04-23T19:10:47.727806+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:28.282546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'282","reported_seq":465,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575725+0000","last_change":"2024-04-23T19:10:15.501932+0000","last_active":"2024-04-23T19:13:16.575725+0000","last_peered":"2024-04-23T19:13:16.575725+0000","last_clean":"2024-04-23T19:13:16.575725+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:16.575725+0000","last_undegraded":"2024-04-23T19:13:16.575725+0000","last_fullsized":"2024-04-23T19:13:16.575725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.546370+0000","last_change":"2024-04-23T19:10:17.515498+0000","last_active":"2024-04-23T19:12:15.546370+0000","last_peered":"2024-04-23T19:12:15.546370+0000","last_clean":"2024-04-23T19:12:15.546370+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:12:15.546370+0000","last_undegraded":"2024-04-23T19:12:15.546370+0000","last_fullsized":"2024-04-23T19:12:15.546370+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:42.717317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.699282+0000","last_change":"2024-04-23T19:10:09.494353+0000","last_active":"2024-04-23T19:10:49.699282+0000","last_peered":"2024-04-23T19:10:49.699282+0000","last_clean":"2024-04-23T19:10:49.699282+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:10:49.699282+0000","last_undegraded":"2024-04-23T19:10:49.699282+0000","last_fullsized":"2024-04-23T19:10:49.699282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.419635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.386367+0000","last_change":"2024-04-23T19:10:07.507496+0000","last_active":"2024-04-23T19:13:16.386367+0000","last_peered":"2024-04-23T19:13:16.386367+0000","last_clean":"2024-04-23T19:13:16.386367+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:13:16.386367+0000","last_undegraded":"2024-04-23T19:13:16.386367+0000","last_fullsized":"2024-04-23T19:13:16.386367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:08.216950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728027+0000","last_change":"2024-04-23T19:10:11.490678+0000","last_active":"2024-04-23T19:10:47.728027+0000","last_peered":"2024-04-23T19:10:47.728027+0000","last_clean":"2024-04-23T19:10:47.728027+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:10:47.728027+0000","last_undegraded":"2024-04-23T19:10:47.728027+0000","last_fullsized":"2024-04-23T19:10:47.728027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:00.102054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'38","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.313282+0000","last_change":"2024-04-23T19:10:13.491356+0000","last_active":"2024-04-23T19:12:14.313282+0000","last_peered":"2024-04-23T19:12:14.313282+0000","last_clean":"2024-04-23T19:12:14.313282+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:12:14.313282+0000","last_undegraded":"2024-04-23T19:12:14.313282+0000","last_fullsized":"2024-04-23T19:12:14.313282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:58.150827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720298+0000","last_change":"2024-04-23T19:10:15.506621+0000","last_active":"2024-04-23T19:10:47.720298+0000","last_peered":"2024-04-23T19:10:47.720298+0000","last_clean":"2024-04-23T19:10:47.720298+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:10:47.720298+0000","last_undegraded":"2024-04-23T19:10:47.720298+0000","last_fullsized":"2024-04-23T19:10:47.720298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:01.215939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.516499+0000","last_change":"2024-04-23T19:10:17.506577+0000","last_active":"2024-04-23T19:13:16.516499+0000","last_peered":"2024-04-23T19:13:16.516499+0000","last_clean":"2024-04-23T19:13:16.516499+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:16.516499+0000","last_undegraded":"2024-04-23T19:13:16.516499+0000","last_fullsized":"2024-04-23T19:13:16.516499+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:44.079746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718448+0000","last_change":"2024-04-23T19:10:09.499812+0000","last_active":"2024-04-23T19:12:16.718448+0000","last_peered":"2024-04-23T19:12:16.718448+0000","last_clean":"2024-04-23T19:12:16.718448+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:12:16.718448+0000","last_undegraded":"2024-04-23T19:12:16.718448+0000","last_fullsized":"2024-04-23T19:12:16.718448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:34.586789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716709+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:10:47.716709+0000","last_peered":"2024-04-23T19:10:47.716709+0000","last_clean":"2024-04-23T19:10:47.716709+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:10:47.716709+0000","last_undegraded":"2024-04-23T19:10:47.716709+0000","last_fullsized":"2024-04-23T19:10:47.716709+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:35.747545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728488+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:10:47.728488+0000","last_peered":"2024-04-23T19:10:47.728488+0000","last_clean":"2024-04-23T19:10:47.728488+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:10:47.728488+0000","last_undegraded":"2024-04-23T19:10:47.728488+0000","last_fullsized":"2024-04-23T19:10:47.728488+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:09.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.547770+0000","last_change":"2024-04-23T19:10:07.508995+0000","last_active":"2024-04-23T19:10:48.547770+0000","last_peered":"2024-04-23T19:10:48.547770+0000","last_clean":"2024-04-23T19:10:48.547770+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:10:48.547770+0000","last_undegraded":"2024-04-23T19:10:48.547770+0000","last_fullsized":"2024-04-23T19:10:48.547770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:08.545897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718470+0000","last_change":"2024-04-23T19:10:11.486708+0000","last_active":"2024-04-23T19:10:47.718470+0000","last_peered":"2024-04-23T19:10:47.718470+0000","last_clean":"2024-04-23T19:10:47.718470+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:10:47.718470+0000","last_undegraded":"2024-04-23T19:10:47.718470+0000","last_fullsized":"2024-04-23T19:10:47.718470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'77","reported_seq":146,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.325024+0000","last_change":"2024-04-23T19:10:13.500080+0000","last_active":"2024-04-23T19:12:14.325024+0000","last_peered":"2024-04-23T19:12:14.325024+0000","last_clean":"2024-04-23T19:12:14.325024+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:12:14.325024+0000","last_undegraded":"2024-04-23T19:12:14.325024+0000","last_fullsized":"2024-04-23T19:12:14.325024+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:43.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728972+0000","last_change":"2024-04-23T19:10:15.502149+0000","last_active":"2024-04-23T19:10:47.728972+0000","last_peered":"2024-04-23T19:10:47.728972+0000","last_clean":"2024-04-23T19:10:47.728972+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:10:47.728972+0000","last_undegraded":"2024-04-23T19:10:47.728972+0000","last_fullsized":"2024-04-23T19:10:47.728972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:59.765301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":162,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.491783+0000","last_change":"2024-04-23T19:10:17.516475+0000","last_active":"2024-04-23T19:13:16.491783+0000","last_peered":"2024-04-23T19:13:16.491783+0000","last_clean":"2024-04-23T19:13:16.491783+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:16.491783+0000","last_undegraded":"2024-04-23T19:13:16.491783+0000","last_fullsized":"2024-04-23T19:13:16.491783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:20.418910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.149738+0000","last_change":"2024-04-23T19:10:09.488098+0000","last_active":"2024-04-23T19:11:32.149738+0000","last_peered":"2024-04-23T19:11:32.149738+0000","last_clean":"2024-04-23T19:11:32.149738+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:11:32.149738+0000","last_undegraded":"2024-04-23T19:11:32.149738+0000","last_fullsized":"2024-04-23T19:11:32.149738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:52.645415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725571+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:10:47.725571+0000","last_peered":"2024-04-23T19:10:47.725571+0000","last_clean":"2024-04-23T19:10:47.725571+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:10:47.725571+0000","last_undegraded":"2024-04-23T19:10:47.725571+0000","last_fullsized":"2024-04-23T19:10:47.725571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:17.746806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726921+0000","last_change":"2024-04-23T19:10:07.486483+0000","last_active":"2024-04-23T19:10:47.726921+0000","last_peered":"2024-04-23T19:10:47.726921+0000","last_clean":"2024-04-23T19:10:47.726921+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:10:47.726921+0000","last_undegraded":"2024-04-23T19:10:47.726921+0000","last_fullsized":"2024-04-23T19:10:47.726921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729492+0000","last_change":"2024-04-23T19:10:11.487708+0000","last_active":"2024-04-23T19:10:47.729492+0000","last_peered":"2024-04-23T19:10:47.729492+0000","last_clean":"2024-04-23T19:10:47.729492+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:10:47.729492+0000","last_undegraded":"2024-04-23T19:10:47.729492+0000","last_fullsized":"2024-04-23T19:10:47.729492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:07.101469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'273","reported_seq":476,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575779+0000","last_change":"2024-04-23T19:10:13.500713+0000","last_active":"2024-04-23T19:13:16.575779+0000","last_peered":"2024-04-23T19:13:16.575779+0000","last_clean":"2024-04-23T19:13:16.575779+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:16.575779+0000","last_undegraded":"2024-04-23T19:13:16.575779+0000","last_fullsized":"2024-04-23T19:13:16.575779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:46.992080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727751+0000","last_change":"2024-04-23T19:10:15.513032+0000","last_active":"2024-04-23T19:10:47.727751+0000","last_peered":"2024-04-23T19:10:47.727751+0000","last_clean":"2024-04-23T19:10:47.727751+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:10:47.727751+0000","last_undegraded":"2024-04-23T19:10:47.727751+0000","last_fullsized":"2024-04-23T19:10:47.727751+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:13.084568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.968215+0000","last_change":"2024-04-23T19:10:17.518750+0000","last_active":"2024-04-23T19:11:31.968215+0000","last_peered":"2024-04-23T19:11:31.968215+0000","last_clean":"2024-04-23T19:11:31.968215+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:11:31.968215+0000","last_undegraded":"2024-04-23T19:11:31.968215+0000","last_fullsized":"2024-04-23T19:11:31.968215+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:02.438494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482305+0000","last_change":"2024-04-23T19:10:09.486782+0000","last_active":"2024-04-23T19:13:16.482305+0000","last_peered":"2024-04-23T19:13:16.482305+0000","last_clean":"2024-04-23T19:13:16.482305+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:13:16.482305+0000","last_undegraded":"2024-04-23T19:13:16.482305+0000","last_fullsized":"2024-04-23T19:13:16.482305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718645+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:10:47.718645+0000","last_peered":"2024-04-23T19:10:47.718645+0000","last_clean":"2024-04-23T19:10:47.718645+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:10:47.718645+0000","last_undegraded":"2024-04-23T19:10:47.718645+0000","last_fullsized":"2024-04-23T19:10:47.718645+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:45.326443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726584+0000","last_change":"2024-04-23T19:10:07.491212+0000","last_active":"2024-04-23T19:10:47.726584+0000","last_peered":"2024-04-23T19:10:47.726584+0000","last_clean":"2024-04-23T19:10:47.726584+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:10:47.726584+0000","last_undegraded":"2024-04-23T19:10:47.726584+0000","last_fullsized":"2024-04-23T19:10:47.726584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"23'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.736899+0000","last_change":"2024-04-23T19:10:47.736369+0000","last_active":"2024-04-23T19:10:47.736899+0000","last_peered":"2024-04-23T19:10:47.736899+0000","last_clean":"2024-04-23T19:10:47.736899+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:10:47.736899+0000","last_undegraded":"2024-04-23T19:10:47.736899+0000","last_fullsized":"2024-04-23T19:10:47.736899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:05.556192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'16","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.576017+0000","last_change":"2024-04-23T19:10:13.495216+0000","last_active":"2024-04-23T19:13:16.576017+0000","last_peered":"2024-04-23T19:13:16.576017+0000","last_clean":"2024-04-23T19:13:16.576017+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:16.576017+0000","last_undegraded":"2024-04-23T19:13:16.576017+0000","last_fullsized":"2024-04-23T19:13:16.576017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:32.747522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728617+0000","last_change":"2024-04-23T19:10:15.502291+0000","last_active":"2024-04-23T19:10:47.728617+0000","last_peered":"2024-04-23T19:10:47.728617+0000","last_clean":"2024-04-23T19:10:47.728617+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:10:47.728617+0000","last_undegraded":"2024-04-23T19:10:47.728617+0000","last_fullsized":"2024-04-23T19:10:47.728617+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:04.115844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725858+0000","last_change":"2024-04-23T19:10:17.517093+0000","last_active":"2024-04-23T19:10:47.725858+0000","last_peered":"2024-04-23T19:10:47.725858+0000","last_clean":"2024-04-23T19:10:47.725858+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:10:47.725858+0000","last_undegraded":"2024-04-23T19:10:47.725858+0000","last_fullsized":"2024-04-23T19:10:47.725858+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:42.364548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399934+0000","last_change":"2024-04-23T19:10:09.477714+0000","last_active":"2024-04-23T19:13:16.399934+0000","last_peered":"2024-04-23T19:13:16.399934+0000","last_clean":"2024-04-23T19:13:16.399934+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:16.399934+0000","last_undegraded":"2024-04-23T19:13:16.399934+0000","last_fullsized":"2024-04-23T19:13:16.399934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726493+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:10:47.726493+0000","last_peered":"2024-04-23T19:10:47.726493+0000","last_clean":"2024-04-23T19:10:47.726493+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:10:47.726493+0000","last_undegraded":"2024-04-23T19:10:47.726493+0000","last_fullsized":"2024-04-23T19:10:47.726493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:51.612339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720012+0000","last_change":"2024-04-23T19:10:07.482292+0000","last_active":"2024-04-23T19:10:47.720012+0000","last_peered":"2024-04-23T19:10:47.720012+0000","last_clean":"2024-04-23T19:10:47.720012+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:10:47.720012+0000","last_undegraded":"2024-04-23T19:10:47.720012+0000","last_fullsized":"2024-04-23T19:10:47.720012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729072+0000","last_change":"2024-04-23T19:10:11.490475+0000","last_active":"2024-04-23T19:10:47.729072+0000","last_peered":"2024-04-23T19:10:47.729072+0000","last_clean":"2024-04-23T19:10:47.729072+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:10:47.729072+0000","last_undegraded":"2024-04-23T19:10:47.729072+0000","last_fullsized":"2024-04-23T19:10:47.729072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:37.989491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'14","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.784607+0000","last_change":"2024-04-23T19:10:13.501263+0000","last_active":"2024-04-23T19:13:15.784607+0000","last_peered":"2024-04-23T19:13:15.784607+0000","last_clean":"2024-04-23T19:13:15.784607+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:15.784607+0000","last_undegraded":"2024-04-23T19:13:15.784607+0000","last_fullsized":"2024-04-23T19:13:15.784607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:45:20.243610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728295+0000","last_change":"2024-04-23T19:10:15.512566+0000","last_active":"2024-04-23T19:10:47.728295+0000","last_peered":"2024-04-23T19:10:47.728295+0000","last_clean":"2024-04-23T19:10:47.728295+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:10:47.728295+0000","last_undegraded":"2024-04-23T19:10:47.728295+0000","last_fullsized":"2024-04-23T19:10:47.728295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:37:48.972551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.182118+0000","last_change":"2024-04-23T19:10:17.515693+0000","last_active":"2024-04-23T19:13:16.182118+0000","last_peered":"2024-04-23T19:13:16.182118+0000","last_clean":"2024-04-23T19:13:16.182118+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:13:16.182118+0000","last_undegraded":"2024-04-23T19:13:16.182118+0000","last_fullsized":"2024-04-23T19:13:16.182118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:12.374714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447462+0000","last_change":"2024-04-23T19:10:09.494884+0000","last_active":"2024-04-23T19:13:16.447462+0000","last_peered":"2024-04-23T19:13:16.447462+0000","last_clean":"2024-04-23T19:13:16.447462+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:16.447462+0000","last_undegraded":"2024-04-23T19:13:16.447462+0000","last_fullsized":"2024-04-23T19:13:16.447462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.383442+0000","last_change":"2024-04-23T19:10:07.483919+0000","last_active":"2024-04-23T19:13:16.383442+0000","last_peered":"2024-04-23T19:13:16.383442+0000","last_clean":"2024-04-23T19:13:16.383442+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:13:16.383442+0000","last_undegraded":"2024-04-23T19:13:16.383442+0000","last_fullsized":"2024-04-23T19:13:16.383442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728467+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:10:47.728467+0000","last_peered":"2024-04-23T19:10:47.728467+0000","last_clean":"2024-04-23T19:10:47.728467+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:10:47.728467+0000","last_undegraded":"2024-04-23T19:10:47.728467+0000","last_fullsized":"2024-04-23T19:10:47.728467+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:34:51.167423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:55.721760+0000","last_change":"2024-04-23T19:10:47.730021+0000","last_active":"2024-04-23T19:12:55.721760+0000","last_peered":"2024-04-23T19:12:55.721760+0000","last_clean":"2024-04-23T19:12:55.721760+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:12:55.721760+0000","last_undegraded":"2024-04-23T19:12:55.721760+0000","last_fullsized":"2024-04-23T19:12:55.721760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:56.417228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'63","reported_seq":122,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710844+0000","last_change":"2024-04-23T19:10:13.505680+0000","last_active":"2024-04-23T19:12:15.710844+0000","last_peered":"2024-04-23T19:12:15.710844+0000","last_clean":"2024-04-23T19:12:15.710844+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:12:15.710844+0000","last_undegraded":"2024-04-23T19:12:15.710844+0000","last_fullsized":"2024-04-23T19:12:15.710844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:59.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719607+0000","last_change":"2024-04-23T19:10:15.506374+0000","last_active":"2024-04-23T19:10:47.719607+0000","last_peered":"2024-04-23T19:10:47.719607+0000","last_clean":"2024-04-23T19:10:47.719607+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:10:47.719607+0000","last_undegraded":"2024-04-23T19:10:47.719607+0000","last_fullsized":"2024-04-23T19:10:47.719607+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:42.776000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.868259+0000","last_change":"2024-04-23T19:10:17.506903+0000","last_active":"2024-04-23T19:13:15.868259+0000","last_peered":"2024-04-23T19:13:15.868259+0000","last_clean":"2024-04-23T19:13:15.868259+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:15.868259+0000","last_undegraded":"2024-04-23T19:13:15.868259+0000","last_fullsized":"2024-04-23T19:13:15.868259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:24.436159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447101+0000","last_change":"2024-04-23T19:10:09.492787+0000","last_active":"2024-04-23T19:13:16.447101+0000","last_peered":"2024-04-23T19:13:16.447101+0000","last_clean":"2024-04-23T19:13:16.447101+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:16.447101+0000","last_undegraded":"2024-04-23T19:13:16.447101+0000","last_fullsized":"2024-04-23T19:13:16.447101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545103+0000","last_change":"2024-04-23T19:10:07.486351+0000","last_active":"2024-04-23T19:10:48.545103+0000","last_peered":"2024-04-23T19:10:48.545103+0000","last_clean":"2024-04-23T19:10:48.545103+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:10:48.545103+0000","last_undegraded":"2024-04-23T19:10:48.545103+0000","last_fullsized":"2024-04-23T19:10:48.545103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:48:50.668566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728626+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:10:47.728626+0000","last_peered":"2024-04-23T19:10:47.728626+0000","last_clean":"2024-04-23T19:10:47.728626+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:10:47.728626+0000","last_undegraded":"2024-04-23T19:10:47.728626+0000","last_fullsized":"2024-04-23T19:10:47.728626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:36.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726094+0000","last_change":"2024-04-23T19:10:11.488952+0000","last_active":"2024-04-23T19:10:47.726094+0000","last_peered":"2024-04-23T19:10:47.726094+0000","last_clean":"2024-04-23T19:10:47.726094+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:10:47.726094+0000","last_undegraded":"2024-04-23T19:10:47.726094+0000","last_fullsized":"2024-04-23T19:10:47.726094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:09.450816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'15","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.702102+0000","last_change":"2024-04-23T19:10:13.497890+0000","last_active":"2024-04-23T19:10:49.702102+0000","last_peered":"2024-04-23T19:10:49.702102+0000","last_clean":"2024-04-23T19:10:49.702102+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:10:49.702102+0000","last_undegraded":"2024-04-23T19:10:49.702102+0000","last_fullsized":"2024-04-23T19:10:49.702102+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:21.612895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'282","reported_seq":355,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575687+0000","last_change":"2024-04-23T19:10:15.502267+0000","last_active":"2024-04-23T19:13:16.575687+0000","last_peered":"2024-04-23T19:13:16.575687+0000","last_clean":"2024-04-23T19:13:16.575687+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:16.575687+0000","last_undegraded":"2024-04-23T19:13:16.575687+0000","last_fullsized":"2024-04-23T19:13:16.575687+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.147752+0000","last_change":"2024-04-23T19:10:17.513890+0000","last_active":"2024-04-23T19:11:32.147752+0000","last_peered":"2024-04-23T19:11:32.147752+0000","last_clean":"2024-04-23T19:11:32.147752+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:11:32.147752+0000","last_undegraded":"2024-04-23T19:11:32.147752+0000","last_fullsized":"2024-04-23T19:11:32.147752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:45.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482692+0000","last_change":"2024-04-23T19:10:09.496860+0000","last_active":"2024-04-23T19:13:16.482692+0000","last_peered":"2024-04-23T19:13:16.482692+0000","last_clean":"2024-04-23T19:13:16.482692+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:13:16.482692+0000","last_undegraded":"2024-04-23T19:13:16.482692+0000","last_fullsized":"2024-04-23T19:13:16.482692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:58.945396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729332+0000","last_change":"2024-04-23T19:10:07.493532+0000","last_active":"2024-04-23T19:10:47.729332+0000","last_peered":"2024-04-23T19:10:47.729332+0000","last_clean":"2024-04-23T19:10:47.729332+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:10:47.729332+0000","last_undegraded":"2024-04-23T19:10:47.729332+0000","last_fullsized":"2024-04-23T19:10:47.729332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:03.936999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729461+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:10:47.729461+0000","last_peered":"2024-04-23T19:10:47.729461+0000","last_clean":"2024-04-23T19:10:47.729461+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:10:47.729461+0000","last_undegraded":"2024-04-23T19:10:47.729461+0000","last_fullsized":"2024-04-23T19:10:47.729461+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:13.817485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728459+0000","last_change":"2024-04-23T19:10:11.487496+0000","last_active":"2024-04-23T19:10:47.728459+0000","last_peered":"2024-04-23T19:10:47.728459+0000","last_clean":"2024-04-23T19:10:47.728459+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:10:47.728459+0000","last_undegraded":"2024-04-23T19:10:47.728459+0000","last_fullsized":"2024-04-23T19:10:47.728459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:12.234867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"30'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728000+0000","last_change":"2024-04-23T19:10:13.503998+0000","last_active":"2024-04-23T19:10:47.728000+0000","last_peered":"2024-04-23T19:10:47.728000+0000","last_clean":"2024-04-23T19:10:47.728000+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:10:47.728000+0000","last_undegraded":"2024-04-23T19:10:47.728000+0000","last_fullsized":"2024-04-23T19:10:47.728000+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:33.974553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'282","reported_seq":429,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.584566+0000","last_change":"2024-04-23T19:10:15.509417+0000","last_active":"2024-04-23T19:13:16.584566+0000","last_peered":"2024-04-23T19:13:16.584566+0000","last_clean":"2024-04-23T19:13:16.584566+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:16.584566+0000","last_undegraded":"2024-04-23T19:13:16.584566+0000","last_fullsized":"2024-04-23T19:13:16.584566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718167+0000","last_change":"2024-04-23T19:10:17.516808+0000","last_active":"2024-04-23T19:10:47.718167+0000","last_peered":"2024-04-23T19:10:47.718167+0000","last_clean":"2024-04-23T19:10:47.718167+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:10:47.718167+0000","last_undegraded":"2024-04-23T19:10:47.718167+0000","last_fullsized":"2024-04-23T19:10:47.718167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:41.513557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449088+0000","last_change":"2024-04-23T19:10:09.493717+0000","last_active":"2024-04-23T19:13:16.449088+0000","last_peered":"2024-04-23T19:13:16.449088+0000","last_clean":"2024-04-23T19:13:16.449088+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:16.449088+0000","last_undegraded":"2024-04-23T19:13:16.449088+0000","last_fullsized":"2024-04-23T19:13:16.449088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:44.856835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.749316+0000","last_change":"2024-04-23T19:10:07.500858+0000","last_active":"2024-04-23T19:12:15.749316+0000","last_peered":"2024-04-23T19:12:15.749316+0000","last_clean":"2024-04-23T19:12:15.749316+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:12:15.749316+0000","last_undegraded":"2024-04-23T19:12:15.749316+0000","last_fullsized":"2024-04-23T19:12:15.749316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728818+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:10:47.728818+0000","last_peered":"2024-04-23T19:10:47.728818+0000","last_clean":"2024-04-23T19:10:47.728818+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:10:47.728818+0000","last_undegraded":"2024-04-23T19:10:47.728818+0000","last_fullsized":"2024-04-23T19:10:47.728818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.025384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727223+0000","last_change":"2024-04-23T19:10:11.489138+0000","last_active":"2024-04-23T19:10:47.727223+0000","last_peered":"2024-04-23T19:10:47.727223+0000","last_clean":"2024-04-23T19:10:47.727223+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:10:47.727223+0000","last_undegraded":"2024-04-23T19:10:47.727223+0000","last_fullsized":"2024-04-23T19:10:47.727223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:19.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'83","reported_seq":155,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.716534+0000","last_change":"2024-04-23T19:10:13.503368+0000","last_active":"2024-04-23T19:12:20.716534+0000","last_peered":"2024-04-23T19:12:20.716534+0000","last_clean":"2024-04-23T19:12:20.716534+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:12:20.716534+0000","last_undegraded":"2024-04-23T19:12:20.716534+0000","last_fullsized":"2024-04-23T19:12:20.716534+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:31.109448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6026,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":77,"num_read_kb":63,"num_write":30,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727890+0000","last_change":"2024-04-23T19:10:15.500592+0000","last_active":"2024-04-23T19:10:47.727890+0000","last_peered":"2024-04-23T19:10:47.727890+0000","last_clean":"2024-04-23T19:10:47.727890+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:10:47.727890+0000","last_undegraded":"2024-04-23T19:10:47.727890+0000","last_fullsized":"2024-04-23T19:10:47.727890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:11.115258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722791+0000","last_change":"2024-04-23T19:10:17.515628+0000","last_active":"2024-04-23T19:10:47.722791+0000","last_peered":"2024-04-23T19:10:47.722791+0000","last_clean":"2024-04-23T19:10:47.722791+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:10:47.722791+0000","last_undegraded":"2024-04-23T19:10:47.722791+0000","last_fullsized":"2024-04-23T19:10:47.722791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.641905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722100+0000","last_change":"2024-04-23T19:10:17.515232+0000","last_active":"2024-04-23T19:10:47.722100+0000","last_peered":"2024-04-23T19:10:47.722100+0000","last_clean":"2024-04-23T19:10:47.722100+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:10:47.722100+0000","last_undegraded":"2024-04-23T19:10:47.722100+0000","last_fullsized":"2024-04-23T19:10:47.722100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:40.692521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720221+0000","last_change":"2024-04-23T19:10:09.492213+0000","last_active":"2024-04-23T19:10:47.720221+0000","last_peered":"2024-04-23T19:10:47.720221+0000","last_clean":"2024-04-23T19:10:47.720221+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:10:47.720221+0000","last_undegraded":"2024-04-23T19:10:47.720221+0000","last_fullsized":"2024-04-23T19:10:47.720221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:54.477619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726467+0000","last_change":"2024-04-23T19:10:07.500107+0000","last_active":"2024-04-23T19:10:47.726467+0000","last_peered":"2024-04-23T19:10:47.726467+0000","last_clean":"2024-04-23T19:10:47.726467+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:10:47.726467+0000","last_undegraded":"2024-04-23T19:10:47.726467+0000","last_fullsized":"2024-04-23T19:10:47.726467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:14.278078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727809+0000","last_change":"2024-04-23T19:10:47.725041+0000","last_active":"2024-04-23T19:10:47.727809+0000","last_peered":"2024-04-23T19:10:47.727809+0000","last_clean":"2024-04-23T19:10:47.727809+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:10:47.727809+0000","last_undegraded":"2024-04-23T19:10:47.727809+0000","last_fullsized":"2024-04-23T19:10:47.727809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:17.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.317498+0000","last_change":"2024-04-23T19:10:13.497156+0000","last_active":"2024-04-23T19:12:14.317498+0000","last_peered":"2024-04-23T19:12:14.317498+0000","last_clean":"2024-04-23T19:12:14.317498+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:12:14.317498+0000","last_undegraded":"2024-04-23T19:12:14.317498+0000","last_fullsized":"2024-04-23T19:12:14.317498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:49.136472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726980+0000","last_change":"2024-04-23T19:10:15.514289+0000","last_active":"2024-04-23T19:10:47.726980+0000","last_peered":"2024-04-23T19:10:47.726980+0000","last_clean":"2024-04-23T19:10:47.726980+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:10:47.726980+0000","last_undegraded":"2024-04-23T19:10:47.726980+0000","last_fullsized":"2024-04-23T19:10:47.726980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:12.517542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.023661+0000","last_change":"2024-04-23T19:10:17.514011+0000","last_active":"2024-04-23T19:12:14.023661+0000","last_peered":"2024-04-23T19:12:14.023661+0000","last_clean":"2024-04-23T19:12:14.023661+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:12:14.023661+0000","last_undegraded":"2024-04-23T19:12:14.023661+0000","last_fullsized":"2024-04-23T19:12:14.023661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:55:22.164549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.733183+0000","last_change":"2024-04-23T19:10:47.732302+0000","last_active":"2024-04-23T19:10:47.733183+0000","last_peered":"2024-04-23T19:10:47.733183+0000","last_clean":"2024-04-23T19:10:47.733183+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:10:47.733183+0000","last_undegraded":"2024-04-23T19:10:47.733183+0000","last_fullsized":"2024-04-23T19:10:47.733183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:05.759643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.139371+0000","last_change":"2024-04-23T19:10:07.493914+0000","last_active":"2024-04-23T19:13:16.139371+0000","last_peered":"2024-04-23T19:13:16.139371+0000","last_clean":"2024-04-23T19:13:16.139371+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:13:16.139371+0000","last_undegraded":"2024-04-23T19:13:16.139371+0000","last_fullsized":"2024-04-23T19:13:16.139371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729231+0000","last_change":"2024-04-23T19:10:11.487861+0000","last_active":"2024-04-23T19:10:47.729231+0000","last_peered":"2024-04-23T19:10:47.729231+0000","last_clean":"2024-04-23T19:10:47.729231+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:10:47.729231+0000","last_undegraded":"2024-04-23T19:10:47.729231+0000","last_fullsized":"2024-04-23T19:10:47.729231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:47.278160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'5","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789104+0000","last_change":"2024-04-23T19:10:13.494591+0000","last_active":"2024-04-23T19:13:15.789104+0000","last_peered":"2024-04-23T19:13:15.789104+0000","last_clean":"2024-04-23T19:13:15.789104+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:15.789104+0000","last_undegraded":"2024-04-23T19:13:15.789104+0000","last_fullsized":"2024-04-23T19:13:15.789104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:58.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718895+0000","last_change":"2024-04-23T19:10:15.513825+0000","last_active":"2024-04-23T19:10:47.718895+0000","last_peered":"2024-04-23T19:10:47.718895+0000","last_clean":"2024-04-23T19:10:47.718895+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:10:47.718895+0000","last_undegraded":"2024-04-23T19:10:47.718895+0000","last_fullsized":"2024-04-23T19:10:47.718895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728340+0000","last_change":"2024-04-23T19:10:17.520788+0000","last_active":"2024-04-23T19:10:47.728340+0000","last_peered":"2024-04-23T19:10:47.728340+0000","last_clean":"2024-04-23T19:10:47.728340+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:10:47.728340+0000","last_undegraded":"2024-04-23T19:10:47.728340+0000","last_fullsized":"2024-04-23T19:10:47.728340+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:36.438306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399875+0000","last_change":"2024-04-23T19:10:09.491571+0000","last_active":"2024-04-23T19:13:16.399875+0000","last_peered":"2024-04-23T19:13:16.399875+0000","last_clean":"2024-04-23T19:13:16.399875+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:16.399875+0000","last_undegraded":"2024-04-23T19:13:16.399875+0000","last_fullsized":"2024-04-23T19:13:16.399875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724424+0000","last_change":"2024-04-23T19:10:07.500769+0000","last_active":"2024-04-23T19:10:47.724424+0000","last_peered":"2024-04-23T19:10:47.724424+0000","last_clean":"2024-04-23T19:10:47.724424+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:10:47.724424+0000","last_undegraded":"2024-04-23T19:10:47.724424+0000","last_fullsized":"2024-04-23T19:10:47.724424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718451+0000","last_change":"2024-04-23T19:10:11.493115+0000","last_active":"2024-04-23T19:10:47.718451+0000","last_peered":"2024-04-23T19:10:47.718451+0000","last_clean":"2024-04-23T19:10:47.718451+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:10:47.718451+0000","last_undegraded":"2024-04-23T19:10:47.718451+0000","last_fullsized":"2024-04-23T19:10:47.718451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:51.288167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'40","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.848055+0000","last_change":"2024-04-23T19:10:13.491151+0000","last_active":"2024-04-23T19:13:15.848055+0000","last_peered":"2024-04-23T19:13:15.848055+0000","last_clean":"2024-04-23T19:13:15.848055+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:15.848055+0000","last_undegraded":"2024-04-23T19:13:15.848055+0000","last_fullsized":"2024-04-23T19:13:15.848055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:56.656293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717052+0000","last_change":"2024-04-23T19:10:15.497695+0000","last_active":"2024-04-23T19:10:47.717052+0000","last_peered":"2024-04-23T19:10:47.717052+0000","last_clean":"2024-04-23T19:10:47.717052+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:10:47.717052+0000","last_undegraded":"2024-04-23T19:10:47.717052+0000","last_fullsized":"2024-04-23T19:10:47.717052+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:40.709841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.164233+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:10:50.164233+0000","last_peered":"2024-04-23T19:10:50.164233+0000","last_clean":"2024-04-23T19:10:50.164233+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:10:50.164233+0000","last_undegraded":"2024-04-23T19:10:50.164233+0000","last_fullsized":"2024-04-23T19:10:50.164233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:38:04.095526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'9","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165987+0000","last_change":"2024-04-23T19:10:09.498393+0000","last_active":"2024-04-23T19:13:16.165987+0000","last_peered":"2024-04-23T19:13:16.165987+0000","last_clean":"2024-04-23T19:13:16.165987+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:13:16.165987+0000","last_undegraded":"2024-04-23T19:13:16.165987+0000","last_fullsized":"2024-04-23T19:13:16.165987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:24.946389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722073+0000","last_change":"2024-04-23T19:10:07.496451+0000","last_active":"2024-04-23T19:10:47.722073+0000","last_peered":"2024-04-23T19:10:47.722073+0000","last_clean":"2024-04-23T19:10:47.722073+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:10:47.722073+0000","last_undegraded":"2024-04-23T19:10:47.722073+0000","last_fullsized":"2024-04-23T19:10:47.722073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724117+0000","last_change":"2024-04-23T19:10:11.490728+0000","last_active":"2024-04-23T19:10:47.724117+0000","last_peered":"2024-04-23T19:10:47.724117+0000","last_clean":"2024-04-23T19:10:47.724117+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:10:47.724117+0000","last_undegraded":"2024-04-23T19:10:47.724117+0000","last_fullsized":"2024-04-23T19:10:47.724117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:55.751511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'32","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.302845+0000","last_change":"2024-04-23T19:10:13.493330+0000","last_active":"2024-04-23T19:12:14.302845+0000","last_peered":"2024-04-23T19:12:14.302845+0000","last_clean":"2024-04-23T19:12:14.302845+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:12:14.302845+0000","last_undegraded":"2024-04-23T19:12:14.302845+0000","last_fullsized":"2024-04-23T19:12:14.302845+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:48.481253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":44,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719061+0000","last_change":"2024-04-23T19:10:15.508565+0000","last_active":"2024-04-23T19:10:47.719061+0000","last_peered":"2024-04-23T19:10:47.719061+0000","last_clean":"2024-04-23T19:10:47.719061+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:10:47.719061+0000","last_undegraded":"2024-04-23T19:10:47.719061+0000","last_fullsized":"2024-04-23T19:10:47.719061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:02.695164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703848+0000","last_change":"2024-04-23T19:10:47.728408+0000","last_active":"2024-04-23T19:10:50.703848+0000","last_peered":"2024-04-23T19:10:50.703848+0000","last_clean":"2024-04-23T19:10:50.703848+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:10:50.703848+0000","last_undegraded":"2024-04-23T19:10:50.703848+0000","last_fullsized":"2024-04-23T19:10:50.703848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:36.054291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.720730+0000","last_change":"2024-04-23T19:10:07.503907+0000","last_active":"2024-04-23T19:12:15.720730+0000","last_peered":"2024-04-23T19:12:15.720730+0000","last_clean":"2024-04-23T19:12:15.720730+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:12:15.720730+0000","last_undegraded":"2024-04-23T19:12:15.720730+0000","last_fullsized":"2024-04-23T19:12:15.720730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:52.469809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730560+0000","last_change":"2024-04-23T19:10:11.490872+0000","last_active":"2024-04-23T19:10:47.730560+0000","last_peered":"2024-04-23T19:10:47.730560+0000","last_clean":"2024-04-23T19:10:47.730560+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:10:47.730560+0000","last_undegraded":"2024-04-23T19:10:47.730560+0000","last_fullsized":"2024-04-23T19:10:47.730560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'54","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.323713+0000","last_change":"2024-04-23T19:10:13.494268+0000","last_active":"2024-04-23T19:12:14.323713+0000","last_peered":"2024-04-23T19:12:14.323713+0000","last_clean":"2024-04-23T19:12:14.323713+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:12:14.323713+0000","last_undegraded":"2024-04-23T19:12:14.323713+0000","last_fullsized":"2024-04-23T19:12:14.323713+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:00.688062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719088+0000","last_change":"2024-04-23T19:10:15.505820+0000","last_active":"2024-04-23T19:10:47.719088+0000","last_peered":"2024-04-23T19:10:47.719088+0000","last_clean":"2024-04-23T19:10:47.719088+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:10:47.719088+0000","last_undegraded":"2024-04-23T19:10:47.719088+0000","last_fullsized":"2024-04-23T19:10:47.719088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:23.619996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.704248+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:15.704248+0000","last_peered":"2024-04-23T19:12:15.704248+0000","last_clean":"2024-04-23T19:12:15.704248+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:12:15.704248+0000","last_undegraded":"2024-04-23T19:12:15.704248+0000","last_fullsized":"2024-04-23T19:12:15.704248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:58.312384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482427+0000","last_change":"2024-04-23T19:10:09.482621+0000","last_active":"2024-04-23T19:13:16.482427+0000","last_peered":"2024-04-23T19:13:16.482427+0000","last_clean":"2024-04-23T19:13:16.482427+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:13:16.482427+0000","last_undegraded":"2024-04-23T19:13:16.482427+0000","last_fullsized":"2024-04-23T19:13:16.482427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720227+0000","last_change":"2024-04-23T19:10:07.493942+0000","last_active":"2024-04-23T19:12:31.720227+0000","last_peered":"2024-04-23T19:12:31.720227+0000","last_clean":"2024-04-23T19:12:31.720227+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:12:31.720227+0000","last_undegraded":"2024-04-23T19:12:31.720227+0000","last_fullsized":"2024-04-23T19:12:31.720227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721926+0000","last_change":"2024-04-23T19:10:11.486920+0000","last_active":"2024-04-23T19:10:47.721926+0000","last_peered":"2024-04-23T19:10:47.721926+0000","last_clean":"2024-04-23T19:10:47.721926+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:10:47.721926+0000","last_undegraded":"2024-04-23T19:10:47.721926+0000","last_fullsized":"2024-04-23T19:10:47.721926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:56.355675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.303836+0000","last_change":"2024-04-23T19:10:13.502855+0000","last_active":"2024-04-23T19:12:14.303836+0000","last_peered":"2024-04-23T19:12:14.303836+0000","last_clean":"2024-04-23T19:12:14.303836+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:12:14.303836+0000","last_undegraded":"2024-04-23T19:12:14.303836+0000","last_fullsized":"2024-04-23T19:12:14.303836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:09.770521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717958+0000","last_change":"2024-04-23T19:10:15.510094+0000","last_active":"2024-04-23T19:10:47.717958+0000","last_peered":"2024-04-23T19:10:47.717958+0000","last_clean":"2024-04-23T19:10:47.717958+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:10:47.717958+0000","last_undegraded":"2024-04-23T19:10:47.717958+0000","last_fullsized":"2024-04-23T19:10:47.717958+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.609469+0000","last_change":"2024-04-23T19:10:17.517391+0000","last_active":"2024-04-23T19:12:12.609469+0000","last_peered":"2024-04-23T19:12:12.609469+0000","last_clean":"2024-04-23T19:12:12.609469+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:12:12.609469+0000","last_undegraded":"2024-04-23T19:12:12.609469+0000","last_fullsized":"2024-04-23T19:12:12.609469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:35:39.248022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703799+0000","last_change":"2024-04-23T19:10:47.733611+0000","last_active":"2024-04-23T19:10:50.703799+0000","last_peered":"2024-04-23T19:10:50.703799+0000","last_clean":"2024-04-23T19:10:50.703799+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:10:50.703799+0000","last_undegraded":"2024-04-23T19:10:50.703799+0000","last_fullsized":"2024-04-23T19:10:50.703799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:04.349258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717131+0000","last_change":"2024-04-23T19:10:07.502477+0000","last_active":"2024-04-23T19:10:47.717131+0000","last_peered":"2024-04-23T19:10:47.717131+0000","last_clean":"2024-04-23T19:10:47.717131+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:10:47.717131+0000","last_undegraded":"2024-04-23T19:10:47.717131+0000","last_fullsized":"2024-04-23T19:10:47.717131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719460+0000","last_change":"2024-04-23T19:10:11.487445+0000","last_active":"2024-04-23T19:10:47.719460+0000","last_peered":"2024-04-23T19:10:47.719460+0000","last_clean":"2024-04-23T19:10:47.719460+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:10:47.719460+0000","last_undegraded":"2024-04-23T19:10:47.719460+0000","last_fullsized":"2024-04-23T19:10:47.719460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:56.771240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.321490+0000","last_change":"2024-04-23T19:10:13.498101+0000","last_active":"2024-04-23T19:12:14.321490+0000","last_peered":"2024-04-23T19:12:14.321490+0000","last_clean":"2024-04-23T19:12:14.321490+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:12:14.321490+0000","last_undegraded":"2024-04-23T19:12:14.321490+0000","last_fullsized":"2024-04-23T19:12:14.321490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729093+0000","last_change":"2024-04-23T19:10:15.504279+0000","last_active":"2024-04-23T19:10:47.729093+0000","last_peered":"2024-04-23T19:10:47.729093+0000","last_clean":"2024-04-23T19:10:47.729093+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:10:47.729093+0000","last_undegraded":"2024-04-23T19:10:47.729093+0000","last_fullsized":"2024-04-23T19:10:47.729093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:05.168473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.451981+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.451981+0000","last_peered":"2024-04-23T19:13:16.451981+0000","last_clean":"2024-04-23T19:13:16.451981+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.451981+0000","last_undegraded":"2024-04-23T19:13:16.451981+0000","last_fullsized":"2024-04-23T19:13:16.451981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:52.410520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399955+0000","last_change":"2024-04-23T19:10:09.477657+0000","last_active":"2024-04-23T19:13:16.399955+0000","last_peered":"2024-04-23T19:13:16.399955+0000","last_clean":"2024-04-23T19:13:16.399955+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:16.399955+0000","last_undegraded":"2024-04-23T19:13:16.399955+0000","last_fullsized":"2024-04-23T19:13:16.399955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.117950+0000","last_change":"2024-04-23T19:10:07.504027+0000","last_active":"2024-04-23T19:13:16.117950+0000","last_peered":"2024-04-23T19:13:16.117950+0000","last_clean":"2024-04-23T19:13:16.117950+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:13:16.117950+0000","last_undegraded":"2024-04-23T19:13:16.117950+0000","last_fullsized":"2024-04-23T19:13:16.117950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:34.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'4","reported_seq":226,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.374959+0000","last_change":"2024-04-23T19:10:11.485789+0000","last_active":"2024-04-23T19:12:15.374959+0000","last_peered":"2024-04-23T19:12:15.374959+0000","last_clean":"2024-04-23T19:12:15.374959+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:12:15.374959+0000","last_undegraded":"2024-04-23T19:12:15.374959+0000","last_fullsized":"2024-04-23T19:12:15.374959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:54.297420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728532+0000","last_change":"2024-04-23T19:10:13.502626+0000","last_active":"2024-04-23T19:10:47.728532+0000","last_peered":"2024-04-23T19:10:47.728532+0000","last_clean":"2024-04-23T19:10:47.728532+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:10:47.728532+0000","last_undegraded":"2024-04-23T19:10:47.728532+0000","last_fullsized":"2024-04-23T19:10:47.728532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:27.703823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'282","reported_seq":417,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575740+0000","last_change":"2024-04-23T19:10:15.510611+0000","last_active":"2024-04-23T19:13:16.575740+0000","last_peered":"2024-04-23T19:13:16.575740+0000","last_clean":"2024-04-23T19:13:16.575740+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:16.575740+0000","last_undegraded":"2024-04-23T19:13:16.575740+0000","last_fullsized":"2024-04-23T19:13:16.575740+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:00.157929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725003+0000","last_change":"2024-04-23T19:10:17.513830+0000","last_active":"2024-04-23T19:10:47.725003+0000","last_peered":"2024-04-23T19:10:47.725003+0000","last_clean":"2024-04-23T19:10:47.725003+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:10:47.725003+0000","last_undegraded":"2024-04-23T19:10:47.725003+0000","last_fullsized":"2024-04-23T19:10:47.725003+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.594849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.150535+0000","last_change":"2024-04-23T19:10:09.476601+0000","last_active":"2024-04-23T19:11:32.150535+0000","last_peered":"2024-04-23T19:11:32.150535+0000","last_clean":"2024-04-23T19:11:32.150535+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:11:32.150535+0000","last_undegraded":"2024-04-23T19:11:32.150535+0000","last_fullsized":"2024-04-23T19:11:32.150535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:10.224761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718929+0000","last_change":"2024-04-23T19:10:07.497992+0000","last_active":"2024-04-23T19:10:47.718929+0000","last_peered":"2024-04-23T19:10:47.718929+0000","last_clean":"2024-04-23T19:10:47.718929+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:10:47.718929+0000","last_undegraded":"2024-04-23T19:10:47.718929+0000","last_fullsized":"2024-04-23T19:10:47.718929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720669+0000","last_change":"2024-04-23T19:10:11.489334+0000","last_active":"2024-04-23T19:10:47.720669+0000","last_peered":"2024-04-23T19:10:47.720669+0000","last_clean":"2024-04-23T19:10:47.720669+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:10:47.720669+0000","last_undegraded":"2024-04-23T19:10:47.720669+0000","last_fullsized":"2024-04-23T19:10:47.720669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'37","reported_seq":82,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.294646+0000","last_change":"2024-04-23T19:10:13.492238+0000","last_active":"2024-04-23T19:12:14.294646+0000","last_peered":"2024-04-23T19:12:14.294646+0000","last_clean":"2024-04-23T19:12:14.294646+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:12:14.294646+0000","last_undegraded":"2024-04-23T19:12:14.294646+0000","last_fullsized":"2024-04-23T19:12:14.294646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726624+0000","last_change":"2024-04-23T19:10:15.502657+0000","last_active":"2024-04-23T19:10:47.726624+0000","last_peered":"2024-04-23T19:10:47.726624+0000","last_clean":"2024-04-23T19:10:47.726624+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:10:47.726624+0000","last_undegraded":"2024-04-23T19:10:47.726624+0000","last_fullsized":"2024-04-23T19:10:47.726624+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722842+0000","last_change":"2024-04-23T19:10:17.518482+0000","last_active":"2024-04-23T19:10:47.722842+0000","last_peered":"2024-04-23T19:10:47.722842+0000","last_clean":"2024-04-23T19:10:47.722842+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:10:47.722842+0000","last_undegraded":"2024-04-23T19:10:47.722842+0000","last_fullsized":"2024-04-23T19:10:47.722842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'3","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.405706+0000","last_change":"2024-04-23T19:10:09.477644+0000","last_active":"2024-04-23T19:13:16.405706+0000","last_peered":"2024-04-23T19:13:16.405706+0000","last_clean":"2024-04-23T19:13:16.405706+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:16.405706+0000","last_undegraded":"2024-04-23T19:13:16.405706+0000","last_fullsized":"2024-04-23T19:13:16.405706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728885+0000","last_change":"2024-04-23T19:10:07.503802+0000","last_active":"2024-04-23T19:10:47.728885+0000","last_peered":"2024-04-23T19:10:47.728885+0000","last_clean":"2024-04-23T19:10:47.728885+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:10:47.728885+0000","last_undegraded":"2024-04-23T19:10:47.728885+0000","last_fullsized":"2024-04-23T19:10:47.728885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:03.446904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720608+0000","last_change":"2024-04-23T19:10:11.489940+0000","last_active":"2024-04-23T19:10:47.720608+0000","last_peered":"2024-04-23T19:10:47.720608+0000","last_clean":"2024-04-23T19:10:47.720608+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:10:47.720608+0000","last_undegraded":"2024-04-23T19:10:47.720608+0000","last_fullsized":"2024-04-23T19:10:47.720608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:35.423825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.295694+0000","last_change":"2024-04-23T19:10:13.505761+0000","last_active":"2024-04-23T19:12:14.295694+0000","last_peered":"2024-04-23T19:12:14.295694+0000","last_clean":"2024-04-23T19:12:14.295694+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:12:14.295694+0000","last_undegraded":"2024-04-23T19:12:14.295694+0000","last_fullsized":"2024-04-23T19:12:14.295694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:28.957267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.723783+0000","last_change":"2024-04-23T19:10:15.514687+0000","last_active":"2024-04-23T19:10:47.723783+0000","last_peered":"2024-04-23T19:10:47.723783+0000","last_clean":"2024-04-23T19:10:47.723783+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:10:47.723783+0000","last_undegraded":"2024-04-23T19:10:47.723783+0000","last_fullsized":"2024-04-23T19:10:47.723783+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:29.022733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.768829+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:15.768829+0000","last_peered":"2024-04-23T19:12:15.768829+0000","last_clean":"2024-04-23T19:12:15.768829+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:15.768829+0000","last_undegraded":"2024-04-23T19:12:15.768829+0000","last_fullsized":"2024-04-23T19:12:15.768829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:42.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'11","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165914+0000","last_change":"2024-04-23T19:10:09.490516+0000","last_active":"2024-04-23T19:13:16.165914+0000","last_peered":"2024-04-23T19:13:16.165914+0000","last_clean":"2024-04-23T19:13:16.165914+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:13:16.165914+0000","last_undegraded":"2024-04-23T19:13:16.165914+0000","last_fullsized":"2024-04-23T19:13:16.165914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:54.586490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729712+0000","last_change":"2024-04-23T19:10:07.486770+0000","last_active":"2024-04-23T19:10:47.729712+0000","last_peered":"2024-04-23T19:10:47.729712+0000","last_clean":"2024-04-23T19:10:47.729712+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:10:47.729712+0000","last_undegraded":"2024-04-23T19:10:47.729712+0000","last_fullsized":"2024-04-23T19:10:47.729712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:56:06.039657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717835+0000","last_change":"2024-04-23T19:10:11.491433+0000","last_active":"2024-04-23T19:10:47.717835+0000","last_peered":"2024-04-23T19:10:47.717835+0000","last_clean":"2024-04-23T19:10:47.717835+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:10:47.717835+0000","last_undegraded":"2024-04-23T19:10:47.717835+0000","last_fullsized":"2024-04-23T19:10:47.717835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'26","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.319314+0000","last_change":"2024-04-23T19:10:13.503113+0000","last_active":"2024-04-23T19:12:14.319314+0000","last_peered":"2024-04-23T19:12:14.319314+0000","last_clean":"2024-04-23T19:12:14.319314+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:12:14.319314+0000","last_undegraded":"2024-04-23T19:12:14.319314+0000","last_fullsized":"2024-04-23T19:12:14.319314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:25.616871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726203+0000","last_change":"2024-04-23T19:10:15.512937+0000","last_active":"2024-04-23T19:10:47.726203+0000","last_peered":"2024-04-23T19:10:47.726203+0000","last_clean":"2024-04-23T19:10:47.726203+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:10:47.726203+0000","last_undegraded":"2024-04-23T19:10:47.726203+0000","last_fullsized":"2024-04-23T19:10:47.726203+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:07.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.329498+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:14.329498+0000","last_peered":"2024-04-23T19:12:14.329498+0000","last_clean":"2024-04-23T19:12:14.329498+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:14.329498+0000","last_undegraded":"2024-04-23T19:12:14.329498+0000","last_fullsized":"2024-04-23T19:12:14.329498+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:30.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482420+0000","last_change":"2024-04-23T19:10:09.476153+0000","last_active":"2024-04-23T19:13:16.482420+0000","last_peered":"2024-04-23T19:13:16.482420+0000","last_clean":"2024-04-23T19:13:16.482420+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:16.482420+0000","last_undegraded":"2024-04-23T19:13:16.482420+0000","last_fullsized":"2024-04-23T19:13:16.482420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:07.680114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.716957+0000","last_change":"2024-04-23T19:10:07.501211+0000","last_active":"2024-04-23T19:12:11.716957+0000","last_peered":"2024-04-23T19:12:11.716957+0000","last_clean":"2024-04-23T19:12:11.716957+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:12:11.716957+0000","last_undegraded":"2024-04-23T19:12:11.716957+0000","last_fullsized":"2024-04-23T19:12:11.716957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'2","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.979180+0000","last_change":"2024-04-23T19:10:11.491220+0000","last_active":"2024-04-23T19:12:12.979180+0000","last_peered":"2024-04-23T19:12:12.979180+0000","last_clean":"2024-04-23T19:12:12.979180+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:12:12.979180+0000","last_undegraded":"2024-04-23T19:12:12.979180+0000","last_fullsized":"2024-04-23T19:12:12.979180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:56.767918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'32","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789943+0000","last_change":"2024-04-23T19:10:13.502743+0000","last_active":"2024-04-23T19:13:15.789943+0000","last_peered":"2024-04-23T19:13:15.789943+0000","last_clean":"2024-04-23T19:13:15.789943+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:15.789943+0000","last_undegraded":"2024-04-23T19:13:15.789943+0000","last_fullsized":"2024-04-23T19:13:15.789943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:13.230238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727977+0000","last_change":"2024-04-23T19:10:15.502314+0000","last_active":"2024-04-23T19:10:47.727977+0000","last_peered":"2024-04-23T19:10:47.727977+0000","last_clean":"2024-04-23T19:10:47.727977+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:10:47.727977+0000","last_undegraded":"2024-04-23T19:10:47.727977+0000","last_fullsized":"2024-04-23T19:10:47.727977+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:26.867666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.434484+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.434484+0000","last_peered":"2024-04-23T19:13:16.434484+0000","last_clean":"2024-04-23T19:13:16.434484+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.434484+0000","last_undegraded":"2024-04-23T19:13:16.434484+0000","last_fullsized":"2024-04-23T19:13:16.434484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:47.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489162+0000","last_change":"2024-04-23T19:10:09.494269+0000","last_active":"2024-04-23T19:13:16.489162+0000","last_peered":"2024-04-23T19:13:16.489162+0000","last_clean":"2024-04-23T19:13:16.489162+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:16.489162+0000","last_undegraded":"2024-04-23T19:13:16.489162+0000","last_fullsized":"2024-04-23T19:13:16.489162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:25.844962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.477833+0000","last_change":"2024-04-23T19:10:07.492825+0000","last_active":"2024-04-23T19:13:16.477833+0000","last_peered":"2024-04-23T19:13:16.477833+0000","last_clean":"2024-04-23T19:13:16.477833+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:13:16.477833+0000","last_undegraded":"2024-04-23T19:13:16.477833+0000","last_fullsized":"2024-04-23T19:13:16.477833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771507+0000","last_change":"2024-04-23T19:10:09.478781+0000","last_active":"2024-04-23T19:12:15.771507+0000","last_peered":"2024-04-23T19:12:15.771507+0000","last_clean":"2024-04-23T19:12:15.771507+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:12:15.771507+0000","last_undegraded":"2024-04-23T19:12:15.771507+0000","last_fullsized":"2024-04-23T19:12:15.771507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710865+0000","last_change":"2024-04-23T19:10:07.489213+0000","last_active":"2024-04-23T19:12:15.710865+0000","last_peered":"2024-04-23T19:12:15.710865+0000","last_clean":"2024-04-23T19:12:15.710865+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:12:15.710865+0000","last_undegraded":"2024-04-23T19:12:15.710865+0000","last_fullsized":"2024-04-23T19:12:15.710865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.495393+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:17.602017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726729+0000","last_change":"2024-04-23T19:10:07.506697+0000","last_active":"2024-04-23T19:10:47.726729+0000","last_peered":"2024-04-23T19:10:47.726729+0000","last_clean":"2024-04-23T19:10:47.726729+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:10:47.726729+0000","last_undegraded":"2024-04-23T19:10:47.726729+0000","last_fullsized":"2024-04-23T19:10:47.726729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:36.637883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.158375+0000","last_change":"2024-04-23T19:10:09.478697+0000","last_active":"2024-04-23T19:13:16.158375+0000","last_peered":"2024-04-23T19:13:16.158375+0000","last_clean":"2024-04-23T19:13:16.158375+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:13:16.158375+0000","last_undegraded":"2024-04-23T19:13:16.158375+0000","last_fullsized":"2024-04-23T19:13:16.158375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'3","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710919+0000","last_change":"2024-04-23T19:10:07.490623+0000","last_active":"2024-04-23T19:12:15.710919+0000","last_peered":"2024-04-23T19:12:15.710919+0000","last_clean":"2024-04-23T19:12:15.710919+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:12:15.710919+0000","last_undegraded":"2024-04-23T19:12:15.710919+0000","last_fullsized":"2024-04-23T19:12:15.710919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698550+0000","last_change":"2024-04-23T19:10:09.491248+0000","last_active":"2024-04-23T19:10:49.698550+0000","last_peered":"2024-04-23T19:10:49.698550+0000","last_clean":"2024-04-23T19:10:49.698550+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:10:49.698550+0000","last_undegraded":"2024-04-23T19:10:49.698550+0000","last_fullsized":"2024-04-23T19:10:49.698550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729029+0000","last_change":"2024-04-23T19:10:07.490715+0000","last_active":"2024-04-23T19:10:47.729029+0000","last_peered":"2024-04-23T19:10:47.729029+0000","last_clean":"2024-04-23T19:10:47.729029+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:10:47.729029+0000","last_undegraded":"2024-04-23T19:10:47.729029+0000","last_fullsized":"2024-04-23T19:10:47.729029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:03.712798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447089+0000","last_change":"2024-04-23T19:10:09.494954+0000","last_active":"2024-04-23T19:13:16.447089+0000","last_peered":"2024-04-23T19:13:16.447089+0000","last_clean":"2024-04-23T19:13:16.447089+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:16.447089+0000","last_undegraded":"2024-04-23T19:13:16.447089+0000","last_fullsized":"2024-04-23T19:13:16.447089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.351485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.548131+0000","last_change":"2024-04-23T19:10:07.490753+0000","last_active":"2024-04-23T19:10:48.548131+0000","last_peered":"2024-04-23T19:10:48.548131+0000","last_clean":"2024-04-23T19:10:48.548131+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:10:48.548131+0000","last_undegraded":"2024-04-23T19:10:48.548131+0000","last_fullsized":"2024-04-23T19:10:48.548131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.157844+0000","last_change":"2024-04-23T19:10:09.482692+0000","last_active":"2024-04-23T19:13:16.157844+0000","last_peered":"2024-04-23T19:13:16.157844+0000","last_clean":"2024-04-23T19:13:16.157844+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:13:16.157844+0000","last_undegraded":"2024-04-23T19:13:16.157844+0000","last_fullsized":"2024-04-23T19:13:16.157844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.380363+0000","last_change":"2024-04-23T19:10:07.500499+0000","last_active":"2024-04-23T19:13:16.380363+0000","last_peered":"2024-04-23T19:13:16.380363+0000","last_clean":"2024-04-23T19:13:16.380363+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:16.380363+0000","last_undegraded":"2024-04-23T19:13:16.380363+0000","last_fullsized":"2024-04-23T19:13:16.380363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258229+0000","last_change":"2024-04-23T19:10:09.484981+0000","last_active":"2024-04-23T19:13:16.258229+0000","last_peered":"2024-04-23T19:13:16.258229+0000","last_clean":"2024-04-23T19:13:16.258229+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:13:16.258229+0000","last_undegraded":"2024-04-23T19:13:16.258229+0000","last_fullsized":"2024-04-23T19:13:16.258229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.951884+0000","last_change":"2024-04-23T19:10:07.490970+0000","last_active":"2024-04-23T19:11:31.951884+0000","last_peered":"2024-04-23T19:11:31.951884+0000","last_clean":"2024-04-23T19:11:31.951884+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:11:31.951884+0000","last_undegraded":"2024-04-23T19:11:31.951884+0000","last_fullsized":"2024-04-23T19:11:31.951884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:19.094583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698565+0000","last_change":"2024-04-23T19:10:09.484268+0000","last_active":"2024-04-23T19:10:49.698565+0000","last_peered":"2024-04-23T19:10:49.698565+0000","last_clean":"2024-04-23T19:10:49.698565+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:10:49.698565+0000","last_undegraded":"2024-04-23T19:10:49.698565+0000","last_fullsized":"2024-04-23T19:10:49.698565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:46.058996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726273+0000","last_change":"2024-04-23T19:10:07.507068+0000","last_active":"2024-04-23T19:10:47.726273+0000","last_peered":"2024-04-23T19:10:47.726273+0000","last_clean":"2024-04-23T19:10:47.726273+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:10:47.726273+0000","last_undegraded":"2024-04-23T19:10:47.726273+0000","last_fullsized":"2024-04-23T19:10:47.726273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.361176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489138+0000","last_change":"2024-04-23T19:10:09.498436+0000","last_active":"2024-04-23T19:13:16.489138+0000","last_peered":"2024-04-23T19:13:16.489138+0000","last_clean":"2024-04-23T19:13:16.489138+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:13:16.489138+0000","last_undegraded":"2024-04-23T19:13:16.489138+0000","last_fullsized":"2024-04-23T19:13:16.489138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:55.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.685736+0000","last_change":"2024-04-23T19:10:07.489441+0000","last_active":"2024-04-23T19:10:49.685736+0000","last_peered":"2024-04-23T19:10:49.685736+0000","last_clean":"2024-04-23T19:10:49.685736+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:10:49.685736+0000","last_undegraded":"2024-04-23T19:10:49.685736+0000","last_fullsized":"2024-04-23T19:10:49.685736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'5","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712408+0000","last_change":"2024-04-23T19:10:09.492013+0000","last_active":"2024-04-23T19:12:25.712408+0000","last_peered":"2024-04-23T19:12:25.712408+0000","last_clean":"2024-04-23T19:12:25.712408+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:12:25.712408+0000","last_undegraded":"2024-04-23T19:12:25.712408+0000","last_fullsized":"2024-04-23T19:12:25.712408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717325+0000","last_change":"2024-04-23T19:10:07.490880+0000","last_active":"2024-04-23T19:10:47.717325+0000","last_peered":"2024-04-23T19:10:47.717325+0000","last_clean":"2024-04-23T19:10:47.717325+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:10:47.717325+0000","last_undegraded":"2024-04-23T19:10:47.717325+0000","last_fullsized":"2024-04-23T19:10:47.717325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.697786+0000","last_change":"2024-04-23T19:10:09.482277+0000","last_active":"2024-04-23T19:10:49.697786+0000","last_peered":"2024-04-23T19:10:49.697786+0000","last_clean":"2024-04-23T19:10:49.697786+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:10:49.697786+0000","last_undegraded":"2024-04-23T19:10:49.697786+0000","last_fullsized":"2024-04-23T19:10:49.697786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719034+0000","last_change":"2024-04-23T19:10:07.502896+0000","last_active":"2024-04-23T19:10:47.719034+0000","last_peered":"2024-04-23T19:10:47.719034+0000","last_clean":"2024-04-23T19:10:47.719034+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:10:47.719034+0000","last_undegraded":"2024-04-23T19:10:47.719034+0000","last_fullsized":"2024-04-23T19:10:47.719034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.964394+0000","last_change":"2024-04-23T19:10:09.492153+0000","last_active":"2024-04-23T19:11:31.964394+0000","last_peered":"2024-04-23T19:11:31.964394+0000","last_clean":"2024-04-23T19:11:31.964394+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:11:31.964394+0000","last_undegraded":"2024-04-23T19:11:31.964394+0000","last_fullsized":"2024-04-23T19:11:31.964394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.490658+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.568491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720741+0000","last_change":"2024-04-23T19:10:09.483025+0000","last_active":"2024-04-23T19:10:47.720741+0000","last_peered":"2024-04-23T19:10:47.720741+0000","last_clean":"2024-04-23T19:10:47.720741+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:10:47.720741+0000","last_undegraded":"2024-04-23T19:10:47.720741+0000","last_fullsized":"2024-04-23T19:10:47.720741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:47.851863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729169+0000","last_change":"2024-04-23T19:10:07.502808+0000","last_active":"2024-04-23T19:10:47.729169+0000","last_peered":"2024-04-23T19:10:47.729169+0000","last_clean":"2024-04-23T19:10:47.729169+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:10:47.729169+0000","last_undegraded":"2024-04-23T19:10:47.729169+0000","last_fullsized":"2024-04-23T19:10:47.729169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:11.223317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'5","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719983+0000","last_change":"2024-04-23T19:10:09.495999+0000","last_active":"2024-04-23T19:12:26.719983+0000","last_peered":"2024-04-23T19:12:26.719983+0000","last_clean":"2024-04-23T19:12:26.719983+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:12:26.719983+0000","last_undegraded":"2024-04-23T19:12:26.719983+0000","last_fullsized":"2024-04-23T19:12:26.719983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:49.818415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725442+0000","last_change":"2024-04-23T19:10:07.500476+0000","last_active":"2024-04-23T19:10:47.725442+0000","last_peered":"2024-04-23T19:10:47.725442+0000","last_clean":"2024-04-23T19:10:47.725442+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:10:47.725442+0000","last_undegraded":"2024-04-23T19:10:47.725442+0000","last_fullsized":"2024-04-23T19:10:47.725442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258427+0000","last_change":"2024-04-23T19:10:09.489995+0000","last_active":"2024-04-23T19:13:16.258427+0000","last_peered":"2024-04-23T19:13:16.258427+0000","last_clean":"2024-04-23T19:13:16.258427+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:13:16.258427+0000","last_undegraded":"2024-04-23T19:13:16.258427+0000","last_fullsized":"2024-04-23T19:13:16.258427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728417+0000","last_change":"2024-04-23T19:10:07.507910+0000","last_active":"2024-04-23T19:10:47.728417+0000","last_peered":"2024-04-23T19:10:47.728417+0000","last_clean":"2024-04-23T19:10:47.728417+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:10:47.728417+0000","last_undegraded":"2024-04-23T19:10:47.728417+0000","last_fullsized":"2024-04-23T19:10:47.728417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:27.025595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.503476+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:39.006602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399931+0000","last_change":"2024-04-23T19:10:09.487033+0000","last_active":"2024-04-23T19:13:16.399931+0000","last_peered":"2024-04-23T19:13:16.399931+0000","last_clean":"2024-04-23T19:13:16.399931+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:13:16.399931+0000","last_undegraded":"2024-04-23T19:13:16.399931+0000","last_fullsized":"2024-04-23T19:13:16.399931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:36.162060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.392374+0000","last_change":"2024-04-23T19:10:07.486697+0000","last_active":"2024-04-23T19:13:16.392374+0000","last_peered":"2024-04-23T19:13:16.392374+0000","last_clean":"2024-04-23T19:13:16.392374+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:13:16.392374+0000","last_undegraded":"2024-04-23T19:13:16.392374+0000","last_fullsized":"2024-04-23T19:13:16.392374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.159090+0000","last_change":"2024-04-23T19:10:09.491506+0000","last_active":"2024-04-23T19:13:16.159090+0000","last_peered":"2024-04-23T19:13:16.159090+0000","last_clean":"2024-04-23T19:13:16.159090+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:16.159090+0000","last_undegraded":"2024-04-23T19:13:16.159090+0000","last_fullsized":"2024-04-23T19:13:16.159090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13384,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":38719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673007,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46348,"kb_used_data":11012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325492,"statfs":{"total":96636764160,"available":96589303808,"internally_reserved":0,"allocated":11276288,"data_stored":10955671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673006,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44844,"kb_used_data":9508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326996,"statfs":{"total":96636764160,"available":96590843904,"internally_reserved":0,"allocated":9736192,"data_stored":9446289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46552,"kb_used_data":11216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325288,"statfs":{"total":96636764160,"available":96589094912,"internally_reserved":0,"allocated":11485184,"data_stored":11220394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40100,"kb_used_data":8924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331740,"statfs":{"total":96636764160,"available":96595701760,"internally_reserved":0,"allocated":9138176,"data_stored":8856588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10809344,"data_stored":10809344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:13:23.910 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T19:13:23.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:13:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.037+0000 7fcd7efa7640 1 Processor -- start 2024-04-23T19:13:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.037+0000 7fcd7efa7640 1 -- start start 2024-04-23T19:13:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.038+0000 7fcd7efa7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd7812c290 0x7fcd78133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.038+0000 7fcd7efa7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7809aa00 con 0x7fcd7812b7a0 2024-04-23T19:13:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.038+0000 7fcd7efa7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcd780a20d0 con 0x7fcd7812c290 2024-04-23T19:13:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.038+0000 7fcd77fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd7812c290 0x7fcd78133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.038+0000 7fcd7cd43640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd7812b7a0 0x7fcd7812bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38534/0 (socket says 172.21.15.16:38534) 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.038+0000 7fcd7cd43640 1 -- 172.21.15.16:0/28910152 learned_addr learned my addr 172.21.15.16:0/28910152 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.039+0000 7fcd77fff640 1 -- 172.21.15.16:0/28910152 >> v1:172.21.15.16:6789/0 conn(0x7fcd7812b7a0 legacy=0x7fcd7812bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.039+0000 7fcd777fe640 1 -- 172.21.15.16:0/28910152 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605304080 0 0) 0x7fcd7809aa00 con 0x7fcd7812b7a0 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.039+0000 7fcd77fff640 1 -- 172.21.15.16:0/28910152 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7809af50 con 0x7fcd7812c290 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.039+0000 7fcd77fff640 1 --2- 172.21.15.16:0/28910152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812c290 0x7fcd78133fc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009720 tx=0x7fcd6402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172512d4a95ac9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd777fe640 1 -- 172.21.15.16:0/28910152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd640316d0 con 0x7fcd7812c290 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd777fe640 1 -- 172.21.15.16:0/28910152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd64037070 con 0x7fcd7812c290 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd777fe640 1 -- 172.21.15.16:0/28910152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd640319c0 con 0x7fcd7812c290 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd7efa7640 1 -- 172.21.15.16:0/28910152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812c290 msgr2=0x7fcd78133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd7efa7640 1 --2- 172.21.15.16:0/28910152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812c290 0x7fcd78133fc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009720 tx=0x7fcd6402b840 comp rx=0 tx=0).stop 2024-04-23T19:13:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd7efa7640 1 -- 172.21.15.16:0/28910152 shutdown_connections 2024-04-23T19:13:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd7efa7640 1 --2- 172.21.15.16:0/28910152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812c290 0x7fcd78133fc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd7efa7640 1 -- 172.21.15.16:0/28910152 >> 172.21.15.16:0/28910152 conn(0x7fcd780a15b0 msgr2=0x7fcd780a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd7efa7640 1 -- 172.21.15.16:0/28910152 shutdown_connections 2024-04-23T19:13:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.040+0000 7fcd7efa7640 1 -- 172.21.15.16:0/28910152 wait complete. 2024-04-23T19:13:24.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.041+0000 7fcd7efa7640 1 Processor -- start 2024-04-23T19:13:24.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.041+0000 7fcd7efa7640 1 -- start start 2024-04-23T19:13:24.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7efa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812b7a0 0x7fcd781c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:24.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7efa7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78134500 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7cd43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812b7a0 0x7fcd781c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7cd43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812b7a0 0x7fcd781c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38992/0 (socket says 172.21.15.16:38992) 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7cd43640 1 -- 172.21.15.16:0/4066900660 learned_addr learned my addr 172.21.15.16:0/4066900660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7cd43640 1 -- 172.21.15.16:0/4066900660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7806d610 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7cd43640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812b7a0 0x7fcd781c3940 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c420 tx=0x7fcd6c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd757fa640 1 -- 172.21.15.16:0/4066900660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd6c018070 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd757fa640 1 -- 172.21.15.16:0/4066900660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6c0074d0 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7812b510 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd757fa640 1 -- 172.21.15.16:0/4066900660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd6c004950 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.042+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7806dab0 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.043+0000 7fcd757fa640 1 -- 172.21.15.16:0/4066900660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fcd6c01b070 con 0x7fcd7812b7a0 2024-04-23T19:13:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.043+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd78063660 con 0x7fcd7812b7a0 2024-04-23T19:13:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.044+0000 7fcd757fa640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd5c0385c0 0x7fcd5c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.044+0000 7fcd757fa640 1 -- 172.21.15.16:0/4066900660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fcd6c04b430 con 0x7fcd7812b7a0 2024-04-23T19:13:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.044+0000 7fcd77fff640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd5c0385c0 0x7fcd5c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.044+0000 7fcd77fff640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd5c0385c0 0x7fcd5c03aa80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd64036000 tx=0x7fcd64008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:24.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.048+0000 7fcd757fa640 1 -- 172.21.15.16:0/4066900660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd6c011960 con 0x7fcd7812b7a0 2024-04-23T19:13:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.234+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd7809b4d0 con 0x7fcd5c0385c0 2024-04-23T19:13:24.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.250+0000 7fcd757fa640 1 -- 172.21.15.16:0/4066900660 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665664 (secure 0 0 0) 0x7fcd7809b4d0 con 0x7fcd5c0385c0 2024-04-23T19:13:24.261 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:13:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd5c0385c0 msgr2=0x7fcd5c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd5c0385c0 0x7fcd5c03aa80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd64036000 tx=0x7fcd64008030 comp rx=0 tx=0).stop 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812b7a0 msgr2=0x7fcd781c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812b7a0 0x7fcd781c3940 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c420 tx=0x7fcd6c00c8e0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 shutdown_connections 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fcd5c0385c0 0x7fcd5c03aa80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 --2- 172.21.15.16:0/4066900660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7812b7a0 0x7fcd781c3940 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 >> 172.21.15.16:0/4066900660 conn(0x7fcd780a15b0 msgr2=0x7fcd7812e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 shutdown_connections 2024-04-23T19:13:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.255+0000 7fcd7efa7640 1 -- 172.21.15.16:0/4066900660 wait complete. 2024-04-23T19:13:24.271 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-04-23T19:13:22.700633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412789,"num_objects":227,"num_object_clones":0,"num_object_copies":461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":4876,"num_read_kb":4453,"num_write":2104,"num_write_kb":61372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4476,"ondisk_log_size":4476,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":177844,"kb_used_data":40660,"kb_used_omap":6,"kb_used_meta":137145,"kb_avail":377309516,"statfs":{"total":386547056640,"available":386364944384,"internally_reserved":0,"allocated":41635840,"data_stored":40478942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":140437271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811687,"num_objects":9,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003607"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722031+0000","last_change":"2024-04-23T19:10:07.492726+0000","last_active":"2024-04-23T19:10:47.722031+0000","last_peered":"2024-04-23T19:10:47.722031+0000","last_clean":"2024-04-23T19:10:47.722031+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:10:47.722031+0000","last_undegraded":"2024-04-23T19:10:47.722031+0000","last_fullsized":"2024-04-23T19:10:47.722031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771923+0000","last_change":"2024-04-23T19:10:09.485943+0000","last_active":"2024-04-23T19:12:15.771923+0000","last_peered":"2024-04-23T19:12:15.771923+0000","last_clean":"2024-04-23T19:12:15.771923+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:12:15.771923+0000","last_undegraded":"2024-04-23T19:12:15.771923+0000","last_fullsized":"2024-04-23T19:12:15.771923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729774+0000","last_change":"2024-04-23T19:10:07.490929+0000","last_active":"2024-04-23T19:10:47.729774+0000","last_peered":"2024-04-23T19:10:47.729774+0000","last_clean":"2024-04-23T19:10:47.729774+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:10:47.729774+0000","last_undegraded":"2024-04-23T19:10:47.729774+0000","last_fullsized":"2024-04-23T19:10:47.729774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:43.016228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.498665+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:54.763728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729513+0000","last_change":"2024-04-23T19:10:07.508341+0000","last_active":"2024-04-23T19:10:47.729513+0000","last_peered":"2024-04-23T19:10:47.729513+0000","last_clean":"2024-04-23T19:10:47.729513+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:10:47.729513+0000","last_undegraded":"2024-04-23T19:10:47.729513+0000","last_fullsized":"2024-04-23T19:10:47.729513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:50.865140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771899+0000","last_change":"2024-04-23T19:10:09.498971+0000","last_active":"2024-04-23T19:12:15.771899+0000","last_peered":"2024-04-23T19:12:15.771899+0000","last_clean":"2024-04-23T19:12:15.771899+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:12:15.771899+0000","last_undegraded":"2024-04-23T19:12:15.771899+0000","last_fullsized":"2024-04-23T19:12:15.771899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:58:56.677184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719199+0000","last_change":"2024-04-23T19:10:07.490859+0000","last_active":"2024-04-23T19:10:47.719199+0000","last_peered":"2024-04-23T19:10:47.719199+0000","last_clean":"2024-04-23T19:10:47.719199+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:10:47.719199+0000","last_undegraded":"2024-04-23T19:10:47.719199+0000","last_fullsized":"2024-04-23T19:10:47.719199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447391+0000","last_change":"2024-04-23T19:10:09.489556+0000","last_active":"2024-04-23T19:13:16.447391+0000","last_peered":"2024-04-23T19:13:16.447391+0000","last_clean":"2024-04-23T19:13:16.447391+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:13:16.447391+0000","last_undegraded":"2024-04-23T19:13:16.447391+0000","last_fullsized":"2024-04-23T19:13:16.447391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:55.937097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.675532+0000","last_change":"2024-04-23T19:10:07.504507+0000","last_active":"2024-04-23T19:12:15.675532+0000","last_peered":"2024-04-23T19:12:15.675532+0000","last_clean":"2024-04-23T19:12:15.675532+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:12:15.675532+0000","last_undegraded":"2024-04-23T19:12:15.675532+0000","last_fullsized":"2024-04-23T19:12:15.675532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:05.672303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.764033+0000","last_change":"2024-04-23T19:10:09.482365+0000","last_active":"2024-04-23T19:12:15.764033+0000","last_peered":"2024-04-23T19:12:15.764033+0000","last_clean":"2024-04-23T19:12:15.764033+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:12:15.764033+0000","last_undegraded":"2024-04-23T19:12:15.764033+0000","last_fullsized":"2024-04-23T19:12:15.764033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719720+0000","last_change":"2024-04-23T19:10:07.490892+0000","last_active":"2024-04-23T19:12:40.719720+0000","last_peered":"2024-04-23T19:12:40.719720+0000","last_clean":"2024-04-23T19:12:40.719720+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:12:40.719720+0000","last_undegraded":"2024-04-23T19:12:40.719720+0000","last_fullsized":"2024-04-23T19:12:40.719720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258616+0000","last_change":"2024-04-23T19:10:09.499345+0000","last_active":"2024-04-23T19:13:16.258616+0000","last_peered":"2024-04-23T19:13:16.258616+0000","last_clean":"2024-04-23T19:13:16.258616+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:16.258616+0000","last_undegraded":"2024-04-23T19:13:16.258616+0000","last_fullsized":"2024-04-23T19:13:16.258616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:28.869634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724671+0000","last_change":"2024-04-23T19:10:07.489309+0000","last_active":"2024-04-23T19:10:47.724671+0000","last_peered":"2024-04-23T19:10:47.724671+0000","last_clean":"2024-04-23T19:10:47.724671+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:10:47.724671+0000","last_undegraded":"2024-04-23T19:10:47.724671+0000","last_fullsized":"2024-04-23T19:10:47.724671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:18.611845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'23","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458768+0000","last_change":"2024-04-23T19:10:09.492130+0000","last_active":"2024-04-23T19:13:16.458768+0000","last_peered":"2024-04-23T19:13:16.458768+0000","last_clean":"2024-04-23T19:13:16.458768+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:16.458768+0000","last_undegraded":"2024-04-23T19:13:16.458768+0000","last_fullsized":"2024-04-23T19:13:16.458768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.714552+0000","last_change":"2024-04-23T19:10:07.487170+0000","last_active":"2024-04-23T19:12:40.714552+0000","last_peered":"2024-04-23T19:12:40.714552+0000","last_clean":"2024-04-23T19:12:40.714552+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:12:40.714552+0000","last_undegraded":"2024-04-23T19:12:40.714552+0000","last_fullsized":"2024-04-23T19:12:40.714552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'17","reported_seq":72,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458912+0000","last_change":"2024-04-23T19:10:09.493592+0000","last_active":"2024-04-23T19:13:16.458912+0000","last_peered":"2024-04-23T19:13:16.458912+0000","last_clean":"2024-04-23T19:13:16.458912+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:13:16.458912+0000","last_undegraded":"2024-04-23T19:13:16.458912+0000","last_fullsized":"2024-04-23T19:13:16.458912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727109+0000","last_change":"2024-04-23T19:10:09.499521+0000","last_active":"2024-04-23T19:10:47.727109+0000","last_peered":"2024-04-23T19:10:47.727109+0000","last_clean":"2024-04-23T19:10:47.727109+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:10:47.727109+0000","last_undegraded":"2024-04-23T19:10:47.727109+0000","last_fullsized":"2024-04-23T19:10:47.727109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:05.408478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722404+0000","last_change":"2024-04-23T19:10:07.491914+0000","last_active":"2024-04-23T19:10:47.722404+0000","last_peered":"2024-04-23T19:10:47.722404+0000","last_clean":"2024-04-23T19:10:47.722404+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:10:47.722404+0000","last_undegraded":"2024-04-23T19:10:47.722404+0000","last_fullsized":"2024-04-23T19:10:47.722404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449380+0000","last_change":"2024-04-23T19:10:09.482719+0000","last_active":"2024-04-23T19:13:16.449380+0000","last_peered":"2024-04-23T19:13:16.449380+0000","last_clean":"2024-04-23T19:13:16.449380+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:13:16.449380+0000","last_undegraded":"2024-04-23T19:13:16.449380+0000","last_fullsized":"2024-04-23T19:13:16.449380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725054+0000","last_change":"2024-04-23T19:10:07.489933+0000","last_active":"2024-04-23T19:10:47.725054+0000","last_peered":"2024-04-23T19:10:47.725054+0000","last_clean":"2024-04-23T19:10:47.725054+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:10:47.725054+0000","last_undegraded":"2024-04-23T19:10:47.725054+0000","last_fullsized":"2024-04-23T19:10:47.725054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720219+0000","last_change":"2024-04-23T19:10:09.484959+0000","last_active":"2024-04-23T19:10:47.720219+0000","last_peered":"2024-04-23T19:10:47.720219+0000","last_clean":"2024-04-23T19:10:47.720219+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:10:47.720219+0000","last_undegraded":"2024-04-23T19:10:47.720219+0000","last_fullsized":"2024-04-23T19:10:47.720219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:54.767051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720449+0000","last_change":"2024-04-23T19:10:07.493585+0000","last_active":"2024-04-23T19:12:31.720449+0000","last_peered":"2024-04-23T19:12:31.720449+0000","last_clean":"2024-04-23T19:12:31.720449+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:12:31.720449+0000","last_undegraded":"2024-04-23T19:12:31.720449+0000","last_fullsized":"2024-04-23T19:12:31.720449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408189+0000","last_change":"2024-04-23T19:10:09.491221+0000","last_active":"2024-04-23T19:13:16.408189+0000","last_peered":"2024-04-23T19:13:16.408189+0000","last_clean":"2024-04-23T19:13:16.408189+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:16.408189+0000","last_undegraded":"2024-04-23T19:13:16.408189+0000","last_fullsized":"2024-04-23T19:13:16.408189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718838+0000","last_change":"2024-04-23T19:10:07.501878+0000","last_active":"2024-04-23T19:10:47.718838+0000","last_peered":"2024-04-23T19:10:47.718838+0000","last_clean":"2024-04-23T19:10:47.718838+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:10:47.718838+0000","last_undegraded":"2024-04-23T19:10:47.718838+0000","last_fullsized":"2024-04-23T19:10:47.718838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771581+0000","last_change":"2024-04-23T19:10:09.492916+0000","last_active":"2024-04-23T19:12:15.771581+0000","last_peered":"2024-04-23T19:12:15.771581+0000","last_clean":"2024-04-23T19:12:15.771581+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:12:15.771581+0000","last_undegraded":"2024-04-23T19:12:15.771581+0000","last_fullsized":"2024-04-23T19:12:15.771581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728570+0000","last_change":"2024-04-23T19:10:07.490910+0000","last_active":"2024-04-23T19:10:47.728570+0000","last_peered":"2024-04-23T19:10:47.728570+0000","last_clean":"2024-04-23T19:10:47.728570+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:10:47.728570+0000","last_undegraded":"2024-04-23T19:10:47.728570+0000","last_fullsized":"2024-04-23T19:10:47.728570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482419+0000","last_change":"2024-04-23T19:10:09.482527+0000","last_active":"2024-04-23T19:13:16.482419+0000","last_peered":"2024-04-23T19:13:16.482419+0000","last_clean":"2024-04-23T19:13:16.482419+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:16.482419+0000","last_undegraded":"2024-04-23T19:13:16.482419+0000","last_fullsized":"2024-04-23T19:13:16.482419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:46.899076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727215+0000","last_change":"2024-04-23T19:10:07.493956+0000","last_active":"2024-04-23T19:10:47.727215+0000","last_peered":"2024-04-23T19:10:47.727215+0000","last_clean":"2024-04-23T19:10:47.727215+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:10:47.727215+0000","last_undegraded":"2024-04-23T19:10:47.727215+0000","last_fullsized":"2024-04-23T19:10:47.727215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:20.896254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'35","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.422839+0000","last_change":"2024-04-23T19:10:17.514587+0000","last_active":"2024-04-23T19:13:16.422839+0000","last_peered":"2024-04-23T19:13:16.422839+0000","last_clean":"2024-04-23T19:13:16.422839+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:16.422839+0000","last_undegraded":"2024-04-23T19:13:16.422839+0000","last_fullsized":"2024-04-23T19:13:16.422839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:18.256132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.432503+0000","last_change":"2024-04-23T19:10:09.487433+0000","last_active":"2024-04-23T19:10:49.432503+0000","last_peered":"2024-04-23T19:10:49.432503+0000","last_clean":"2024-04-23T19:10:49.432503+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:10:49.432503+0000","last_undegraded":"2024-04-23T19:10:49.432503+0000","last_fullsized":"2024-04-23T19:10:49.432503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:36.758021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728416+0000","last_change":"2024-04-23T19:10:07.507346+0000","last_active":"2024-04-23T19:10:47.728416+0000","last_peered":"2024-04-23T19:10:47.728416+0000","last_clean":"2024-04-23T19:10:47.728416+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:10:47.728416+0000","last_undegraded":"2024-04-23T19:10:47.728416+0000","last_fullsized":"2024-04-23T19:10:47.728416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:42.849025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728325+0000","last_change":"2024-04-23T19:10:11.488429+0000","last_active":"2024-04-23T19:10:47.728325+0000","last_peered":"2024-04-23T19:10:47.728325+0000","last_clean":"2024-04-23T19:10:47.728325+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:10:47.728325+0000","last_undegraded":"2024-04-23T19:10:47.728325+0000","last_fullsized":"2024-04-23T19:10:47.728325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:50.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722656+0000","last_change":"2024-04-23T19:10:13.501588+0000","last_active":"2024-04-23T19:12:41.722656+0000","last_peered":"2024-04-23T19:12:41.722656+0000","last_clean":"2024-04-23T19:12:41.722656+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:12:41.722656+0000","last_undegraded":"2024-04-23T19:12:41.722656+0000","last_fullsized":"2024-04-23T19:12:41.722656+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:34.948969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722273+0000","last_change":"2024-04-23T19:10:15.509911+0000","last_active":"2024-04-23T19:10:47.722273+0000","last_peered":"2024-04-23T19:10:47.722273+0000","last_clean":"2024-04-23T19:10:47.722273+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:10:47.722273+0000","last_undegraded":"2024-04-23T19:10:47.722273+0000","last_fullsized":"2024-04-23T19:10:47.722273+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'9","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.539127+0000","last_change":"2024-04-23T19:10:17.514347+0000","last_active":"2024-04-23T19:13:16.539127+0000","last_peered":"2024-04-23T19:13:16.539127+0000","last_clean":"2024-04-23T19:13:16.539127+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:16.539127+0000","last_undegraded":"2024-04-23T19:13:16.539127+0000","last_fullsized":"2024-04-23T19:13:16.539127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:53.299426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399880+0000","last_change":"2024-04-23T19:10:09.496014+0000","last_active":"2024-04-23T19:13:16.399880+0000","last_peered":"2024-04-23T19:13:16.399880+0000","last_clean":"2024-04-23T19:13:16.399880+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:13:16.399880+0000","last_undegraded":"2024-04-23T19:13:16.399880+0000","last_fullsized":"2024-04-23T19:13:16.399880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:32.575973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720263+0000","last_change":"2024-04-23T19:10:07.492730+0000","last_active":"2024-04-23T19:10:47.720263+0000","last_peered":"2024-04-23T19:10:47.720263+0000","last_clean":"2024-04-23T19:10:47.720263+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:10:47.720263+0000","last_undegraded":"2024-04-23T19:10:47.720263+0000","last_fullsized":"2024-04-23T19:10:47.720263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"23'1","reported_seq":218,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.386033+0000","last_change":"2024-04-23T19:10:11.489225+0000","last_active":"2024-04-23T19:12:15.386033+0000","last_peered":"2024-04-23T19:12:15.386033+0000","last_clean":"2024-04-23T19:12:15.386033+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:12:15.386033+0000","last_undegraded":"2024-04-23T19:12:15.386033+0000","last_fullsized":"2024-04-23T19:12:15.386033+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:01.595709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"30'6","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721580+0000","last_change":"2024-04-23T19:10:13.491509+0000","last_active":"2024-04-23T19:10:47.721580+0000","last_peered":"2024-04-23T19:10:47.721580+0000","last_clean":"2024-04-23T19:10:47.721580+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:10:47.721580+0000","last_undegraded":"2024-04-23T19:10:47.721580+0000","last_fullsized":"2024-04-23T19:10:47.721580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:32.880512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720747+0000","last_change":"2024-04-23T19:10:15.506746+0000","last_active":"2024-04-23T19:10:47.720747+0000","last_peered":"2024-04-23T19:10:47.720747+0000","last_clean":"2024-04-23T19:10:47.720747+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:10:47.720747+0000","last_undegraded":"2024-04-23T19:10:47.720747+0000","last_fullsized":"2024-04-23T19:10:47.720747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726990+0000","last_change":"2024-04-23T19:10:17.518249+0000","last_active":"2024-04-23T19:10:47.726990+0000","last_peered":"2024-04-23T19:10:47.726990+0000","last_clean":"2024-04-23T19:10:47.726990+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:10:47.726990+0000","last_undegraded":"2024-04-23T19:10:47.726990+0000","last_fullsized":"2024-04-23T19:10:47.726990+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:58.051610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447281+0000","last_change":"2024-04-23T19:10:09.490234+0000","last_active":"2024-04-23T19:13:16.447281+0000","last_peered":"2024-04-23T19:13:16.447281+0000","last_clean":"2024-04-23T19:13:16.447281+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:13:16.447281+0000","last_undegraded":"2024-04-23T19:13:16.447281+0000","last_fullsized":"2024-04-23T19:13:16.447281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:54.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.389417+0000","last_change":"2024-04-23T19:10:07.504260+0000","last_active":"2024-04-23T19:13:16.389417+0000","last_peered":"2024-04-23T19:13:16.389417+0000","last_clean":"2024-04-23T19:13:16.389417+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:13:16.389417+0000","last_undegraded":"2024-04-23T19:13:16.389417+0000","last_fullsized":"2024-04-23T19:13:16.389417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:46.596307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730636+0000","last_change":"2024-04-23T19:10:11.493566+0000","last_active":"2024-04-23T19:10:47.730636+0000","last_peered":"2024-04-23T19:10:47.730636+0000","last_clean":"2024-04-23T19:10:47.730636+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:10:47.730636+0000","last_undegraded":"2024-04-23T19:10:47.730636+0000","last_fullsized":"2024-04-23T19:10:47.730636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718899+0000","last_change":"2024-04-23T19:10:13.502669+0000","last_active":"2024-04-23T19:12:35.718899+0000","last_peered":"2024-04-23T19:12:35.718899+0000","last_clean":"2024-04-23T19:12:35.718899+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:12:35.718899+0000","last_undegraded":"2024-04-23T19:12:35.718899+0000","last_fullsized":"2024-04-23T19:12:35.718899+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:02.466667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'282","reported_seq":385,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.598276+0000","last_change":"2024-04-23T19:10:15.515398+0000","last_active":"2024-04-23T19:13:16.598276+0000","last_peered":"2024-04-23T19:13:16.598276+0000","last_clean":"2024-04-23T19:13:16.598276+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:13:16.598276+0000","last_undegraded":"2024-04-23T19:13:16.598276+0000","last_fullsized":"2024-04-23T19:13:16.598276+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.433967+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:10:49.433967+0000","last_peered":"2024-04-23T19:10:49.433967+0000","last_clean":"2024-04-23T19:10:49.433967+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:10:49.433967+0000","last_undegraded":"2024-04-23T19:10:49.433967+0000","last_fullsized":"2024-04-23T19:10:49.433967+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:55.634563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482380+0000","last_change":"2024-04-23T19:10:09.487095+0000","last_active":"2024-04-23T19:13:16.482380+0000","last_peered":"2024-04-23T19:13:16.482380+0000","last_clean":"2024-04-23T19:13:16.482380+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:13:16.482380+0000","last_undegraded":"2024-04-23T19:13:16.482380+0000","last_fullsized":"2024-04-23T19:13:16.482380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716719+0000","last_change":"2024-04-23T19:10:07.504154+0000","last_active":"2024-04-23T19:10:47.716719+0000","last_peered":"2024-04-23T19:10:47.716719+0000","last_clean":"2024-04-23T19:10:47.716719+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:10:47.716719+0000","last_undegraded":"2024-04-23T19:10:47.716719+0000","last_fullsized":"2024-04-23T19:10:47.716719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:24.070928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728932+0000","last_change":"2024-04-23T19:10:11.488228+0000","last_active":"2024-04-23T19:10:47.728932+0000","last_peered":"2024-04-23T19:10:47.728932+0000","last_clean":"2024-04-23T19:10:47.728932+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:10:47.728932+0000","last_undegraded":"2024-04-23T19:10:47.728932+0000","last_fullsized":"2024-04-23T19:10:47.728932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:01.154683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719829+0000","last_change":"2024-04-23T19:10:13.505813+0000","last_active":"2024-04-23T19:12:40.719829+0000","last_peered":"2024-04-23T19:12:40.719829+0000","last_clean":"2024-04-23T19:12:40.719829+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:12:40.719829+0000","last_undegraded":"2024-04-23T19:12:40.719829+0000","last_fullsized":"2024-04-23T19:12:40.719829+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:39.996179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'282","reported_seq":401,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575901+0000","last_change":"2024-04-23T19:10:15.500511+0000","last_active":"2024-04-23T19:13:16.575901+0000","last_peered":"2024-04-23T19:13:16.575901+0000","last_clean":"2024-04-23T19:13:16.575901+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:16.575901+0000","last_undegraded":"2024-04-23T19:13:16.575901+0000","last_fullsized":"2024-04-23T19:13:16.575901+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:23.813091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449319+0000","last_change":"2024-04-23T19:10:47.730482+0000","last_active":"2024-04-23T19:13:16.449319+0000","last_peered":"2024-04-23T19:13:16.449319+0000","last_clean":"2024-04-23T19:13:16.449319+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:13:16.449319+0000","last_undegraded":"2024-04-23T19:13:16.449319+0000","last_fullsized":"2024-04-23T19:13:16.449319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:33.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545310+0000","last_change":"2024-04-23T19:10:07.492480+0000","last_active":"2024-04-23T19:10:48.545310+0000","last_peered":"2024-04-23T19:10:48.545310+0000","last_clean":"2024-04-23T19:10:48.545310+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:10:48.545310+0000","last_undegraded":"2024-04-23T19:10:48.545310+0000","last_fullsized":"2024-04-23T19:10:48.545310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716980+0000","last_change":"2024-04-23T19:10:11.490040+0000","last_active":"2024-04-23T19:10:47.716980+0000","last_peered":"2024-04-23T19:10:47.716980+0000","last_clean":"2024-04-23T19:10:47.716980+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:10:47.716980+0000","last_undegraded":"2024-04-23T19:10:47.716980+0000","last_fullsized":"2024-04-23T19:10:47.716980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:07.412668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'40","reported_seq":87,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.316535+0000","last_change":"2024-04-23T19:10:13.494728+0000","last_active":"2024-04-23T19:12:14.316535+0000","last_peered":"2024-04-23T19:12:14.316535+0000","last_clean":"2024-04-23T19:12:14.316535+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:12:14.316535+0000","last_undegraded":"2024-04-23T19:12:14.316535+0000","last_fullsized":"2024-04-23T19:12:14.316535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:56.304108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728369+0000","last_change":"2024-04-23T19:10:15.506637+0000","last_active":"2024-04-23T19:10:47.728369+0000","last_peered":"2024-04-23T19:10:47.728369+0000","last_clean":"2024-04-23T19:10:47.728369+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:10:47.728369+0000","last_undegraded":"2024-04-23T19:10:47.728369+0000","last_fullsized":"2024-04-23T19:10:47.728369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:13.464606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.971818+0000","last_change":"2024-04-23T19:10:17.519710+0000","last_active":"2024-04-23T19:11:31.971818+0000","last_peered":"2024-04-23T19:11:31.971818+0000","last_clean":"2024-04-23T19:11:31.971818+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:11:31.971818+0000","last_undegraded":"2024-04-23T19:11:31.971818+0000","last_fullsized":"2024-04-23T19:11:31.971818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:10.239585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'7","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.488606+0000","last_change":"2024-04-23T19:10:09.482665+0000","last_active":"2024-04-23T19:13:16.488606+0000","last_peered":"2024-04-23T19:13:16.488606+0000","last_clean":"2024-04-23T19:13:16.488606+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:16.488606+0000","last_undegraded":"2024-04-23T19:13:16.488606+0000","last_fullsized":"2024-04-23T19:13:16.488606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724246+0000","last_change":"2024-04-23T19:10:07.500938+0000","last_active":"2024-04-23T19:10:47.724246+0000","last_peered":"2024-04-23T19:10:47.724246+0000","last_clean":"2024-04-23T19:10:47.724246+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:10:47.724246+0000","last_undegraded":"2024-04-23T19:10:47.724246+0000","last_fullsized":"2024-04-23T19:10:47.724246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727117+0000","last_change":"2024-04-23T19:10:11.493617+0000","last_active":"2024-04-23T19:10:47.727117+0000","last_peered":"2024-04-23T19:10:47.727117+0000","last_clean":"2024-04-23T19:10:47.727117+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:10:47.727117+0000","last_undegraded":"2024-04-23T19:10:47.727117+0000","last_fullsized":"2024-04-23T19:10:47.727117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:32.061098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.296627+0000","last_change":"2024-04-23T19:10:13.501851+0000","last_active":"2024-04-23T19:12:14.296627+0000","last_peered":"2024-04-23T19:12:14.296627+0000","last_clean":"2024-04-23T19:12:14.296627+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:12:14.296627+0000","last_undegraded":"2024-04-23T19:12:14.296627+0000","last_fullsized":"2024-04-23T19:12:14.296627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:30.608779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727229+0000","last_change":"2024-04-23T19:10:15.502245+0000","last_active":"2024-04-23T19:10:47.727229+0000","last_peered":"2024-04-23T19:10:47.727229+0000","last_clean":"2024-04-23T19:10:47.727229+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:10:47.727229+0000","last_undegraded":"2024-04-23T19:10:47.727229+0000","last_fullsized":"2024-04-23T19:10:47.727229+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'5","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.261614+0000","last_change":"2024-04-23T19:10:17.518314+0000","last_active":"2024-04-23T19:13:16.261614+0000","last_peered":"2024-04-23T19:13:16.261614+0000","last_clean":"2024-04-23T19:13:16.261614+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:13:16.261614+0000","last_undegraded":"2024-04-23T19:13:16.261614+0000","last_fullsized":"2024-04-23T19:13:16.261614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:11.577457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.166247+0000","last_change":"2024-04-23T19:10:09.492229+0000","last_active":"2024-04-23T19:13:16.166247+0000","last_peered":"2024-04-23T19:13:16.166247+0000","last_clean":"2024-04-23T19:13:16.166247+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:13:16.166247+0000","last_undegraded":"2024-04-23T19:13:16.166247+0000","last_fullsized":"2024-04-23T19:13:16.166247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:36.861914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729281+0000","last_change":"2024-04-23T19:10:07.492637+0000","last_active":"2024-04-23T19:10:47.729281+0000","last_peered":"2024-04-23T19:10:47.729281+0000","last_clean":"2024-04-23T19:10:47.729281+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:10:47.729281+0000","last_undegraded":"2024-04-23T19:10:47.729281+0000","last_fullsized":"2024-04-23T19:10:47.729281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.154249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729542+0000","last_change":"2024-04-23T19:10:11.491503+0000","last_active":"2024-04-23T19:10:47.729542+0000","last_peered":"2024-04-23T19:10:47.729542+0000","last_clean":"2024-04-23T19:10:47.729542+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:10:47.729542+0000","last_undegraded":"2024-04-23T19:10:47.729542+0000","last_fullsized":"2024-04-23T19:10:47.729542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.318445+0000","last_change":"2024-04-23T19:10:13.508355+0000","last_active":"2024-04-23T19:12:14.318445+0000","last_peered":"2024-04-23T19:12:14.318445+0000","last_clean":"2024-04-23T19:12:14.318445+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:12:14.318445+0000","last_undegraded":"2024-04-23T19:12:14.318445+0000","last_fullsized":"2024-04-23T19:12:14.318445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:19.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'282","reported_seq":348,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.608082+0000","last_change":"2024-04-23T19:10:15.509684+0000","last_active":"2024-04-23T19:13:16.608082+0000","last_peered":"2024-04-23T19:13:16.608082+0000","last_clean":"2024-04-23T19:13:16.608082+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:16.608082+0000","last_undegraded":"2024-04-23T19:13:16.608082+0000","last_fullsized":"2024-04-23T19:13:16.608082+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728722+0000","last_change":"2024-04-23T19:10:17.520791+0000","last_active":"2024-04-23T19:10:47.728722+0000","last_peered":"2024-04-23T19:10:47.728722+0000","last_clean":"2024-04-23T19:10:47.728722+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:10:47.728722+0000","last_undegraded":"2024-04-23T19:10:47.728722+0000","last_fullsized":"2024-04-23T19:10:47.728722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:03.997750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399785+0000","last_change":"2024-04-23T19:10:09.476300+0000","last_active":"2024-04-23T19:13:16.399785+0000","last_peered":"2024-04-23T19:13:16.399785+0000","last_clean":"2024-04-23T19:13:16.399785+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:16.399785+0000","last_undegraded":"2024-04-23T19:13:16.399785+0000","last_fullsized":"2024-04-23T19:13:16.399785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:59.576302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726343+0000","last_change":"2024-04-23T19:10:07.501083+0000","last_active":"2024-04-23T19:10:47.726343+0000","last_peered":"2024-04-23T19:10:47.726343+0000","last_clean":"2024-04-23T19:10:47.726343+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:10:47.726343+0000","last_undegraded":"2024-04-23T19:10:47.726343+0000","last_fullsized":"2024-04-23T19:10:47.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:08.925701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727624+0000","last_change":"2024-04-23T19:10:11.493665+0000","last_active":"2024-04-23T19:10:47.727624+0000","last_peered":"2024-04-23T19:10:47.727624+0000","last_clean":"2024-04-23T19:10:47.727624+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:10:47.727624+0000","last_undegraded":"2024-04-23T19:10:47.727624+0000","last_fullsized":"2024-04-23T19:10:47.727624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:29.762026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'54","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.320262+0000","last_change":"2024-04-23T19:10:13.499017+0000","last_active":"2024-04-23T19:12:14.320262+0000","last_peered":"2024-04-23T19:12:14.320262+0000","last_clean":"2024-04-23T19:12:14.320262+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:12:14.320262+0000","last_undegraded":"2024-04-23T19:12:14.320262+0000","last_fullsized":"2024-04-23T19:12:14.320262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:42.419535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726896+0000","last_change":"2024-04-23T19:10:15.503053+0000","last_active":"2024-04-23T19:10:47.726896+0000","last_peered":"2024-04-23T19:10:47.726896+0000","last_clean":"2024-04-23T19:10:47.726896+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:10:47.726896+0000","last_undegraded":"2024-04-23T19:10:47.726896+0000","last_fullsized":"2024-04-23T19:10:47.726896+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'18","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.583903+0000","last_change":"2024-04-23T19:10:17.513990+0000","last_active":"2024-04-23T19:13:16.583903+0000","last_peered":"2024-04-23T19:13:16.583903+0000","last_clean":"2024-04-23T19:13:16.583903+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:13:16.583903+0000","last_undegraded":"2024-04-23T19:13:16.583903+0000","last_fullsized":"2024-04-23T19:13:16.583903+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:57.264911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258650+0000","last_change":"2024-04-23T19:10:09.490325+0000","last_active":"2024-04-23T19:13:16.258650+0000","last_peered":"2024-04-23T19:13:16.258650+0000","last_clean":"2024-04-23T19:13:16.258650+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:16.258650+0000","last_undegraded":"2024-04-23T19:13:16.258650+0000","last_fullsized":"2024-04-23T19:13:16.258650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728226+0000","last_change":"2024-04-23T19:10:07.490182+0000","last_active":"2024-04-23T19:10:47.728226+0000","last_peered":"2024-04-23T19:10:47.728226+0000","last_clean":"2024-04-23T19:10:47.728226+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:10:47.728226+0000","last_undegraded":"2024-04-23T19:10:47.728226+0000","last_fullsized":"2024-04-23T19:10:47.728226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719940+0000","last_change":"2024-04-23T19:10:11.490064+0000","last_active":"2024-04-23T19:10:47.719940+0000","last_peered":"2024-04-23T19:10:47.719940+0000","last_clean":"2024-04-23T19:10:47.719940+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:10:47.719940+0000","last_undegraded":"2024-04-23T19:10:47.719940+0000","last_fullsized":"2024-04-23T19:10:47.719940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:48.063186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.300676+0000","last_change":"2024-04-23T19:10:13.491370+0000","last_active":"2024-04-23T19:12:14.300676+0000","last_peered":"2024-04-23T19:12:14.300676+0000","last_clean":"2024-04-23T19:12:14.300676+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:12:14.300676+0000","last_undegraded":"2024-04-23T19:12:14.300676+0000","last_fullsized":"2024-04-23T19:12:14.300676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:59.208837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728086+0000","last_change":"2024-04-23T19:10:15.512494+0000","last_active":"2024-04-23T19:10:47.728086+0000","last_peered":"2024-04-23T19:10:47.728086+0000","last_clean":"2024-04-23T19:10:47.728086+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:10:47.728086+0000","last_undegraded":"2024-04-23T19:10:47.728086+0000","last_fullsized":"2024-04-23T19:10:47.728086+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:54.341417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718208+0000","last_change":"2024-04-23T19:10:17.511337+0000","last_active":"2024-04-23T19:12:16.718208+0000","last_peered":"2024-04-23T19:12:16.718208+0000","last_clean":"2024-04-23T19:12:16.718208+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:12:16.718208+0000","last_undegraded":"2024-04-23T19:12:16.718208+0000","last_fullsized":"2024-04-23T19:12:16.718208+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:31.868099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536682+0000","last_change":"2024-04-23T19:10:09.499760+0000","last_active":"2024-04-23T19:13:16.536682+0000","last_peered":"2024-04-23T19:13:16.536682+0000","last_clean":"2024-04-23T19:13:16.536682+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:16.536682+0000","last_undegraded":"2024-04-23T19:13:16.536682+0000","last_fullsized":"2024-04-23T19:13:16.536682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:43.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727856+0000","last_change":"2024-04-23T19:10:07.494969+0000","last_active":"2024-04-23T19:10:47.727856+0000","last_peered":"2024-04-23T19:10:47.727856+0000","last_clean":"2024-04-23T19:10:47.727856+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:10:47.727856+0000","last_undegraded":"2024-04-23T19:10:47.727856+0000","last_fullsized":"2024-04-23T19:10:47.727856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:47.276166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728157+0000","last_change":"2024-04-23T19:10:11.491444+0000","last_active":"2024-04-23T19:10:47.728157+0000","last_peered":"2024-04-23T19:10:47.728157+0000","last_clean":"2024-04-23T19:10:47.728157+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:10:47.728157+0000","last_undegraded":"2024-04-23T19:10:47.728157+0000","last_fullsized":"2024-04-23T19:10:47.728157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:00.459979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.327052+0000","last_change":"2024-04-23T19:10:13.493342+0000","last_active":"2024-04-23T19:12:14.327052+0000","last_peered":"2024-04-23T19:12:14.327052+0000","last_clean":"2024-04-23T19:12:14.327052+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:12:14.327052+0000","last_undegraded":"2024-04-23T19:12:14.327052+0000","last_fullsized":"2024-04-23T19:12:14.327052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:44.904145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'282","reported_seq":403,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.587980+0000","last_change":"2024-04-23T19:10:15.507087+0000","last_active":"2024-04-23T19:13:16.587980+0000","last_peered":"2024-04-23T19:13:16.587980+0000","last_clean":"2024-04-23T19:13:16.587980+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:16.587980+0000","last_undegraded":"2024-04-23T19:13:16.587980+0000","last_fullsized":"2024-04-23T19:13:16.587980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:48.999181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'12","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712278+0000","last_change":"2024-04-23T19:10:17.518152+0000","last_active":"2024-04-23T19:12:25.712278+0000","last_peered":"2024-04-23T19:12:25.712278+0000","last_clean":"2024-04-23T19:12:25.712278+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:12:25.712278+0000","last_undegraded":"2024-04-23T19:12:25.712278+0000","last_fullsized":"2024-04-23T19:12:25.712278+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399883+0000","last_change":"2024-04-23T19:10:09.493242+0000","last_active":"2024-04-23T19:13:16.399883+0000","last_peered":"2024-04-23T19:13:16.399883+0000","last_clean":"2024-04-23T19:13:16.399883+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:13:16.399883+0000","last_undegraded":"2024-04-23T19:13:16.399883+0000","last_fullsized":"2024-04-23T19:13:16.399883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.679903+0000","last_change":"2024-04-23T19:10:07.508434+0000","last_active":"2024-04-23T19:10:49.679903+0000","last_peered":"2024-04-23T19:10:49.679903+0000","last_clean":"2024-04-23T19:10:49.679903+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:10:49.679903+0000","last_undegraded":"2024-04-23T19:10:49.679903+0000","last_fullsized":"2024-04-23T19:10:49.679903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:44.765151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'3","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.385678+0000","last_change":"2024-04-23T19:10:11.491234+0000","last_active":"2024-04-23T19:12:15.385678+0000","last_peered":"2024-04-23T19:12:15.385678+0000","last_clean":"2024-04-23T19:12:15.385678+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:12:15.385678+0000","last_undegraded":"2024-04-23T19:12:15.385678+0000","last_fullsized":"2024-04-23T19:12:15.385678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:03.467829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727806+0000","last_change":"2024-04-23T19:10:13.501643+0000","last_active":"2024-04-23T19:10:47.727806+0000","last_peered":"2024-04-23T19:10:47.727806+0000","last_clean":"2024-04-23T19:10:47.727806+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:10:47.727806+0000","last_undegraded":"2024-04-23T19:10:47.727806+0000","last_fullsized":"2024-04-23T19:10:47.727806+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:28.282546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'282","reported_seq":465,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575725+0000","last_change":"2024-04-23T19:10:15.501932+0000","last_active":"2024-04-23T19:13:16.575725+0000","last_peered":"2024-04-23T19:13:16.575725+0000","last_clean":"2024-04-23T19:13:16.575725+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:16.575725+0000","last_undegraded":"2024-04-23T19:13:16.575725+0000","last_fullsized":"2024-04-23T19:13:16.575725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.546370+0000","last_change":"2024-04-23T19:10:17.515498+0000","last_active":"2024-04-23T19:12:15.546370+0000","last_peered":"2024-04-23T19:12:15.546370+0000","last_clean":"2024-04-23T19:12:15.546370+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:12:15.546370+0000","last_undegraded":"2024-04-23T19:12:15.546370+0000","last_fullsized":"2024-04-23T19:12:15.546370+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:42.717317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.699282+0000","last_change":"2024-04-23T19:10:09.494353+0000","last_active":"2024-04-23T19:10:49.699282+0000","last_peered":"2024-04-23T19:10:49.699282+0000","last_clean":"2024-04-23T19:10:49.699282+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:10:49.699282+0000","last_undegraded":"2024-04-23T19:10:49.699282+0000","last_fullsized":"2024-04-23T19:10:49.699282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.419635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.386367+0000","last_change":"2024-04-23T19:10:07.507496+0000","last_active":"2024-04-23T19:13:16.386367+0000","last_peered":"2024-04-23T19:13:16.386367+0000","last_clean":"2024-04-23T19:13:16.386367+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:13:16.386367+0000","last_undegraded":"2024-04-23T19:13:16.386367+0000","last_fullsized":"2024-04-23T19:13:16.386367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:08.216950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728027+0000","last_change":"2024-04-23T19:10:11.490678+0000","last_active":"2024-04-23T19:10:47.728027+0000","last_peered":"2024-04-23T19:10:47.728027+0000","last_clean":"2024-04-23T19:10:47.728027+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:10:47.728027+0000","last_undegraded":"2024-04-23T19:10:47.728027+0000","last_fullsized":"2024-04-23T19:10:47.728027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:00.102054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'38","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.313282+0000","last_change":"2024-04-23T19:10:13.491356+0000","last_active":"2024-04-23T19:12:14.313282+0000","last_peered":"2024-04-23T19:12:14.313282+0000","last_clean":"2024-04-23T19:12:14.313282+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:12:14.313282+0000","last_undegraded":"2024-04-23T19:12:14.313282+0000","last_fullsized":"2024-04-23T19:12:14.313282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:58.150827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720298+0000","last_change":"2024-04-23T19:10:15.506621+0000","last_active":"2024-04-23T19:10:47.720298+0000","last_peered":"2024-04-23T19:10:47.720298+0000","last_clean":"2024-04-23T19:10:47.720298+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:10:47.720298+0000","last_undegraded":"2024-04-23T19:10:47.720298+0000","last_fullsized":"2024-04-23T19:10:47.720298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:01.215939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.516499+0000","last_change":"2024-04-23T19:10:17.506577+0000","last_active":"2024-04-23T19:13:16.516499+0000","last_peered":"2024-04-23T19:13:16.516499+0000","last_clean":"2024-04-23T19:13:16.516499+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:16.516499+0000","last_undegraded":"2024-04-23T19:13:16.516499+0000","last_fullsized":"2024-04-23T19:13:16.516499+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:44.079746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718448+0000","last_change":"2024-04-23T19:10:09.499812+0000","last_active":"2024-04-23T19:12:16.718448+0000","last_peered":"2024-04-23T19:12:16.718448+0000","last_clean":"2024-04-23T19:12:16.718448+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:12:16.718448+0000","last_undegraded":"2024-04-23T19:12:16.718448+0000","last_fullsized":"2024-04-23T19:12:16.718448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:34.586789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716709+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:10:47.716709+0000","last_peered":"2024-04-23T19:10:47.716709+0000","last_clean":"2024-04-23T19:10:47.716709+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:10:47.716709+0000","last_undegraded":"2024-04-23T19:10:47.716709+0000","last_fullsized":"2024-04-23T19:10:47.716709+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:35.747545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728488+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:10:47.728488+0000","last_peered":"2024-04-23T19:10:47.728488+0000","last_clean":"2024-04-23T19:10:47.728488+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:10:47.728488+0000","last_undegraded":"2024-04-23T19:10:47.728488+0000","last_fullsized":"2024-04-23T19:10:47.728488+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:09.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.547770+0000","last_change":"2024-04-23T19:10:07.508995+0000","last_active":"2024-04-23T19:10:48.547770+0000","last_peered":"2024-04-23T19:10:48.547770+0000","last_clean":"2024-04-23T19:10:48.547770+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:10:48.547770+0000","last_undegraded":"2024-04-23T19:10:48.547770+0000","last_fullsized":"2024-04-23T19:10:48.547770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:08.545897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718470+0000","last_change":"2024-04-23T19:10:11.486708+0000","last_active":"2024-04-23T19:10:47.718470+0000","last_peered":"2024-04-23T19:10:47.718470+0000","last_clean":"2024-04-23T19:10:47.718470+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:10:47.718470+0000","last_undegraded":"2024-04-23T19:10:47.718470+0000","last_fullsized":"2024-04-23T19:10:47.718470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'77","reported_seq":146,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.325024+0000","last_change":"2024-04-23T19:10:13.500080+0000","last_active":"2024-04-23T19:12:14.325024+0000","last_peered":"2024-04-23T19:12:14.325024+0000","last_clean":"2024-04-23T19:12:14.325024+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:12:14.325024+0000","last_undegraded":"2024-04-23T19:12:14.325024+0000","last_fullsized":"2024-04-23T19:12:14.325024+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:43.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728972+0000","last_change":"2024-04-23T19:10:15.502149+0000","last_active":"2024-04-23T19:10:47.728972+0000","last_peered":"2024-04-23T19:10:47.728972+0000","last_clean":"2024-04-23T19:10:47.728972+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:10:47.728972+0000","last_undegraded":"2024-04-23T19:10:47.728972+0000","last_fullsized":"2024-04-23T19:10:47.728972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:59.765301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":162,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.491783+0000","last_change":"2024-04-23T19:10:17.516475+0000","last_active":"2024-04-23T19:13:16.491783+0000","last_peered":"2024-04-23T19:13:16.491783+0000","last_clean":"2024-04-23T19:13:16.491783+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:16.491783+0000","last_undegraded":"2024-04-23T19:13:16.491783+0000","last_fullsized":"2024-04-23T19:13:16.491783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:20.418910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.149738+0000","last_change":"2024-04-23T19:10:09.488098+0000","last_active":"2024-04-23T19:11:32.149738+0000","last_peered":"2024-04-23T19:11:32.149738+0000","last_clean":"2024-04-23T19:11:32.149738+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:11:32.149738+0000","last_undegraded":"2024-04-23T19:11:32.149738+0000","last_fullsized":"2024-04-23T19:11:32.149738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:52.645415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725571+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:10:47.725571+0000","last_peered":"2024-04-23T19:10:47.725571+0000","last_clean":"2024-04-23T19:10:47.725571+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:10:47.725571+0000","last_undegraded":"2024-04-23T19:10:47.725571+0000","last_fullsized":"2024-04-23T19:10:47.725571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:17.746806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726921+0000","last_change":"2024-04-23T19:10:07.486483+0000","last_active":"2024-04-23T19:10:47.726921+0000","last_peered":"2024-04-23T19:10:47.726921+0000","last_clean":"2024-04-23T19:10:47.726921+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:10:47.726921+0000","last_undegraded":"2024-04-23T19:10:47.726921+0000","last_fullsized":"2024-04-23T19:10:47.726921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729492+0000","last_change":"2024-04-23T19:10:11.487708+0000","last_active":"2024-04-23T19:10:47.729492+0000","last_peered":"2024-04-23T19:10:47.729492+0000","last_clean":"2024-04-23T19:10:47.729492+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:10:47.729492+0000","last_undegraded":"2024-04-23T19:10:47.729492+0000","last_fullsized":"2024-04-23T19:10:47.729492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:07.101469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'273","reported_seq":476,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575779+0000","last_change":"2024-04-23T19:10:13.500713+0000","last_active":"2024-04-23T19:13:16.575779+0000","last_peered":"2024-04-23T19:13:16.575779+0000","last_clean":"2024-04-23T19:13:16.575779+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:16.575779+0000","last_undegraded":"2024-04-23T19:13:16.575779+0000","last_fullsized":"2024-04-23T19:13:16.575779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:46.992080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727751+0000","last_change":"2024-04-23T19:10:15.513032+0000","last_active":"2024-04-23T19:10:47.727751+0000","last_peered":"2024-04-23T19:10:47.727751+0000","last_clean":"2024-04-23T19:10:47.727751+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:10:47.727751+0000","last_undegraded":"2024-04-23T19:10:47.727751+0000","last_fullsized":"2024-04-23T19:10:47.727751+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:13.084568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.968215+0000","last_change":"2024-04-23T19:10:17.518750+0000","last_active":"2024-04-23T19:11:31.968215+0000","last_peered":"2024-04-23T19:11:31.968215+0000","last_clean":"2024-04-23T19:11:31.968215+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:11:31.968215+0000","last_undegraded":"2024-04-23T19:11:31.968215+0000","last_fullsized":"2024-04-23T19:11:31.968215+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:02.438494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482305+0000","last_change":"2024-04-23T19:10:09.486782+0000","last_active":"2024-04-23T19:13:16.482305+0000","last_peered":"2024-04-23T19:13:16.482305+0000","last_clean":"2024-04-23T19:13:16.482305+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:13:16.482305+0000","last_undegraded":"2024-04-23T19:13:16.482305+0000","last_fullsized":"2024-04-23T19:13:16.482305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718645+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:10:47.718645+0000","last_peered":"2024-04-23T19:10:47.718645+0000","last_clean":"2024-04-23T19:10:47.718645+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:10:47.718645+0000","last_undegraded":"2024-04-23T19:10:47.718645+0000","last_fullsized":"2024-04-23T19:10:47.718645+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:45.326443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726584+0000","last_change":"2024-04-23T19:10:07.491212+0000","last_active":"2024-04-23T19:10:47.726584+0000","last_peered":"2024-04-23T19:10:47.726584+0000","last_clean":"2024-04-23T19:10:47.726584+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:10:47.726584+0000","last_undegraded":"2024-04-23T19:10:47.726584+0000","last_fullsized":"2024-04-23T19:10:47.726584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"23'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.736899+0000","last_change":"2024-04-23T19:10:47.736369+0000","last_active":"2024-04-23T19:10:47.736899+0000","last_peered":"2024-04-23T19:10:47.736899+0000","last_clean":"2024-04-23T19:10:47.736899+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:10:47.736899+0000","last_undegraded":"2024-04-23T19:10:47.736899+0000","last_fullsized":"2024-04-23T19:10:47.736899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:05.556192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'16","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.576017+0000","last_change":"2024-04-23T19:10:13.495216+0000","last_active":"2024-04-23T19:13:16.576017+0000","last_peered":"2024-04-23T19:13:16.576017+0000","last_clean":"2024-04-23T19:13:16.576017+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:16.576017+0000","last_undegraded":"2024-04-23T19:13:16.576017+0000","last_fullsized":"2024-04-23T19:13:16.576017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:32.747522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728617+0000","last_change":"2024-04-23T19:10:15.502291+0000","last_active":"2024-04-23T19:10:47.728617+0000","last_peered":"2024-04-23T19:10:47.728617+0000","last_clean":"2024-04-23T19:10:47.728617+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:10:47.728617+0000","last_undegraded":"2024-04-23T19:10:47.728617+0000","last_fullsized":"2024-04-23T19:10:47.728617+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:04.115844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725858+0000","last_change":"2024-04-23T19:10:17.517093+0000","last_active":"2024-04-23T19:10:47.725858+0000","last_peered":"2024-04-23T19:10:47.725858+0000","last_clean":"2024-04-23T19:10:47.725858+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:10:47.725858+0000","last_undegraded":"2024-04-23T19:10:47.725858+0000","last_fullsized":"2024-04-23T19:10:47.725858+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:42.364548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399934+0000","last_change":"2024-04-23T19:10:09.477714+0000","last_active":"2024-04-23T19:13:16.399934+0000","last_peered":"2024-04-23T19:13:16.399934+0000","last_clean":"2024-04-23T19:13:16.399934+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:16.399934+0000","last_undegraded":"2024-04-23T19:13:16.399934+0000","last_fullsized":"2024-04-23T19:13:16.399934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726493+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:10:47.726493+0000","last_peered":"2024-04-23T19:10:47.726493+0000","last_clean":"2024-04-23T19:10:47.726493+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:10:47.726493+0000","last_undegraded":"2024-04-23T19:10:47.726493+0000","last_fullsized":"2024-04-23T19:10:47.726493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:51.612339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720012+0000","last_change":"2024-04-23T19:10:07.482292+0000","last_active":"2024-04-23T19:10:47.720012+0000","last_peered":"2024-04-23T19:10:47.720012+0000","last_clean":"2024-04-23T19:10:47.720012+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:10:47.720012+0000","last_undegraded":"2024-04-23T19:10:47.720012+0000","last_fullsized":"2024-04-23T19:10:47.720012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729072+0000","last_change":"2024-04-23T19:10:11.490475+0000","last_active":"2024-04-23T19:10:47.729072+0000","last_peered":"2024-04-23T19:10:47.729072+0000","last_clean":"2024-04-23T19:10:47.729072+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:10:47.729072+0000","last_undegraded":"2024-04-23T19:10:47.729072+0000","last_fullsized":"2024-04-23T19:10:47.729072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:37.989491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'14","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.784607+0000","last_change":"2024-04-23T19:10:13.501263+0000","last_active":"2024-04-23T19:13:15.784607+0000","last_peered":"2024-04-23T19:13:15.784607+0000","last_clean":"2024-04-23T19:13:15.784607+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:15.784607+0000","last_undegraded":"2024-04-23T19:13:15.784607+0000","last_fullsized":"2024-04-23T19:13:15.784607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:45:20.243610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728295+0000","last_change":"2024-04-23T19:10:15.512566+0000","last_active":"2024-04-23T19:10:47.728295+0000","last_peered":"2024-04-23T19:10:47.728295+0000","last_clean":"2024-04-23T19:10:47.728295+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:10:47.728295+0000","last_undegraded":"2024-04-23T19:10:47.728295+0000","last_fullsized":"2024-04-23T19:10:47.728295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:37:48.972551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.182118+0000","last_change":"2024-04-23T19:10:17.515693+0000","last_active":"2024-04-23T19:13:16.182118+0000","last_peered":"2024-04-23T19:13:16.182118+0000","last_clean":"2024-04-23T19:13:16.182118+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:13:16.182118+0000","last_undegraded":"2024-04-23T19:13:16.182118+0000","last_fullsized":"2024-04-23T19:13:16.182118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:12.374714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447462+0000","last_change":"2024-04-23T19:10:09.494884+0000","last_active":"2024-04-23T19:13:16.447462+0000","last_peered":"2024-04-23T19:13:16.447462+0000","last_clean":"2024-04-23T19:13:16.447462+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:16.447462+0000","last_undegraded":"2024-04-23T19:13:16.447462+0000","last_fullsized":"2024-04-23T19:13:16.447462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.383442+0000","last_change":"2024-04-23T19:10:07.483919+0000","last_active":"2024-04-23T19:13:16.383442+0000","last_peered":"2024-04-23T19:13:16.383442+0000","last_clean":"2024-04-23T19:13:16.383442+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:13:16.383442+0000","last_undegraded":"2024-04-23T19:13:16.383442+0000","last_fullsized":"2024-04-23T19:13:16.383442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728467+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:10:47.728467+0000","last_peered":"2024-04-23T19:10:47.728467+0000","last_clean":"2024-04-23T19:10:47.728467+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:10:47.728467+0000","last_undegraded":"2024-04-23T19:10:47.728467+0000","last_fullsized":"2024-04-23T19:10:47.728467+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:34:51.167423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:55.721760+0000","last_change":"2024-04-23T19:10:47.730021+0000","last_active":"2024-04-23T19:12:55.721760+0000","last_peered":"2024-04-23T19:12:55.721760+0000","last_clean":"2024-04-23T19:12:55.721760+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:12:55.721760+0000","last_undegraded":"2024-04-23T19:12:55.721760+0000","last_fullsized":"2024-04-23T19:12:55.721760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:56.417228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'63","reported_seq":122,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710844+0000","last_change":"2024-04-23T19:10:13.505680+0000","last_active":"2024-04-23T19:12:15.710844+0000","last_peered":"2024-04-23T19:12:15.710844+0000","last_clean":"2024-04-23T19:12:15.710844+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:12:15.710844+0000","last_undegraded":"2024-04-23T19:12:15.710844+0000","last_fullsized":"2024-04-23T19:12:15.710844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:59.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719607+0000","last_change":"2024-04-23T19:10:15.506374+0000","last_active":"2024-04-23T19:10:47.719607+0000","last_peered":"2024-04-23T19:10:47.719607+0000","last_clean":"2024-04-23T19:10:47.719607+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:10:47.719607+0000","last_undegraded":"2024-04-23T19:10:47.719607+0000","last_fullsized":"2024-04-23T19:10:47.719607+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:42.776000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.868259+0000","last_change":"2024-04-23T19:10:17.506903+0000","last_active":"2024-04-23T19:13:15.868259+0000","last_peered":"2024-04-23T19:13:15.868259+0000","last_clean":"2024-04-23T19:13:15.868259+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:15.868259+0000","last_undegraded":"2024-04-23T19:13:15.868259+0000","last_fullsized":"2024-04-23T19:13:15.868259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:24.436159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447101+0000","last_change":"2024-04-23T19:10:09.492787+0000","last_active":"2024-04-23T19:13:16.447101+0000","last_peered":"2024-04-23T19:13:16.447101+0000","last_clean":"2024-04-23T19:13:16.447101+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:16.447101+0000","last_undegraded":"2024-04-23T19:13:16.447101+0000","last_fullsized":"2024-04-23T19:13:16.447101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545103+0000","last_change":"2024-04-23T19:10:07.486351+0000","last_active":"2024-04-23T19:10:48.545103+0000","last_peered":"2024-04-23T19:10:48.545103+0000","last_clean":"2024-04-23T19:10:48.545103+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:10:48.545103+0000","last_undegraded":"2024-04-23T19:10:48.545103+0000","last_fullsized":"2024-04-23T19:10:48.545103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:48:50.668566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728626+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:10:47.728626+0000","last_peered":"2024-04-23T19:10:47.728626+0000","last_clean":"2024-04-23T19:10:47.728626+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:10:47.728626+0000","last_undegraded":"2024-04-23T19:10:47.728626+0000","last_fullsized":"2024-04-23T19:10:47.728626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:36.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726094+0000","last_change":"2024-04-23T19:10:11.488952+0000","last_active":"2024-04-23T19:10:47.726094+0000","last_peered":"2024-04-23T19:10:47.726094+0000","last_clean":"2024-04-23T19:10:47.726094+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:10:47.726094+0000","last_undegraded":"2024-04-23T19:10:47.726094+0000","last_fullsized":"2024-04-23T19:10:47.726094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:09.450816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'15","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.702102+0000","last_change":"2024-04-23T19:10:13.497890+0000","last_active":"2024-04-23T19:10:49.702102+0000","last_peered":"2024-04-23T19:10:49.702102+0000","last_clean":"2024-04-23T19:10:49.702102+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:10:49.702102+0000","last_undegraded":"2024-04-23T19:10:49.702102+0000","last_fullsized":"2024-04-23T19:10:49.702102+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:21.612895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'282","reported_seq":355,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575687+0000","last_change":"2024-04-23T19:10:15.502267+0000","last_active":"2024-04-23T19:13:16.575687+0000","last_peered":"2024-04-23T19:13:16.575687+0000","last_clean":"2024-04-23T19:13:16.575687+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:16.575687+0000","last_undegraded":"2024-04-23T19:13:16.575687+0000","last_fullsized":"2024-04-23T19:13:16.575687+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.147752+0000","last_change":"2024-04-23T19:10:17.513890+0000","last_active":"2024-04-23T19:11:32.147752+0000","last_peered":"2024-04-23T19:11:32.147752+0000","last_clean":"2024-04-23T19:11:32.147752+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:11:32.147752+0000","last_undegraded":"2024-04-23T19:11:32.147752+0000","last_fullsized":"2024-04-23T19:11:32.147752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:45.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482692+0000","last_change":"2024-04-23T19:10:09.496860+0000","last_active":"2024-04-23T19:13:16.482692+0000","last_peered":"2024-04-23T19:13:16.482692+0000","last_clean":"2024-04-23T19:13:16.482692+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:13:16.482692+0000","last_undegraded":"2024-04-23T19:13:16.482692+0000","last_fullsized":"2024-04-23T19:13:16.482692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:58.945396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729332+0000","last_change":"2024-04-23T19:10:07.493532+0000","last_active":"2024-04-23T19:10:47.729332+0000","last_peered":"2024-04-23T19:10:47.729332+0000","last_clean":"2024-04-23T19:10:47.729332+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:10:47.729332+0000","last_undegraded":"2024-04-23T19:10:47.729332+0000","last_fullsized":"2024-04-23T19:10:47.729332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:03.936999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729461+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:10:47.729461+0000","last_peered":"2024-04-23T19:10:47.729461+0000","last_clean":"2024-04-23T19:10:47.729461+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:10:47.729461+0000","last_undegraded":"2024-04-23T19:10:47.729461+0000","last_fullsized":"2024-04-23T19:10:47.729461+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:13.817485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728459+0000","last_change":"2024-04-23T19:10:11.487496+0000","last_active":"2024-04-23T19:10:47.728459+0000","last_peered":"2024-04-23T19:10:47.728459+0000","last_clean":"2024-04-23T19:10:47.728459+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:10:47.728459+0000","last_undegraded":"2024-04-23T19:10:47.728459+0000","last_fullsized":"2024-04-23T19:10:47.728459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:12.234867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"30'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728000+0000","last_change":"2024-04-23T19:10:13.503998+0000","last_active":"2024-04-23T19:10:47.728000+0000","last_peered":"2024-04-23T19:10:47.728000+0000","last_clean":"2024-04-23T19:10:47.728000+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:10:47.728000+0000","last_undegraded":"2024-04-23T19:10:47.728000+0000","last_fullsized":"2024-04-23T19:10:47.728000+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:33.974553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'282","reported_seq":429,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.584566+0000","last_change":"2024-04-23T19:10:15.509417+0000","last_active":"2024-04-23T19:13:16.584566+0000","last_peered":"2024-04-23T19:13:16.584566+0000","last_clean":"2024-04-23T19:13:16.584566+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:16.584566+0000","last_undegraded":"2024-04-23T19:13:16.584566+0000","last_fullsized":"2024-04-23T19:13:16.584566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718167+0000","last_change":"2024-04-23T19:10:17.516808+0000","last_active":"2024-04-23T19:10:47.718167+0000","last_peered":"2024-04-23T19:10:47.718167+0000","last_clean":"2024-04-23T19:10:47.718167+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:10:47.718167+0000","last_undegraded":"2024-04-23T19:10:47.718167+0000","last_fullsized":"2024-04-23T19:10:47.718167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:41.513557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449088+0000","last_change":"2024-04-23T19:10:09.493717+0000","last_active":"2024-04-23T19:13:16.449088+0000","last_peered":"2024-04-23T19:13:16.449088+0000","last_clean":"2024-04-23T19:13:16.449088+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:16.449088+0000","last_undegraded":"2024-04-23T19:13:16.449088+0000","last_fullsized":"2024-04-23T19:13:16.449088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:44.856835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.749316+0000","last_change":"2024-04-23T19:10:07.500858+0000","last_active":"2024-04-23T19:12:15.749316+0000","last_peered":"2024-04-23T19:12:15.749316+0000","last_clean":"2024-04-23T19:12:15.749316+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:12:15.749316+0000","last_undegraded":"2024-04-23T19:12:15.749316+0000","last_fullsized":"2024-04-23T19:12:15.749316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728818+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:10:47.728818+0000","last_peered":"2024-04-23T19:10:47.728818+0000","last_clean":"2024-04-23T19:10:47.728818+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:10:47.728818+0000","last_undegraded":"2024-04-23T19:10:47.728818+0000","last_fullsized":"2024-04-23T19:10:47.728818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.025384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727223+0000","last_change":"2024-04-23T19:10:11.489138+0000","last_active":"2024-04-23T19:10:47.727223+0000","last_peered":"2024-04-23T19:10:47.727223+0000","last_clean":"2024-04-23T19:10:47.727223+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:10:47.727223+0000","last_undegraded":"2024-04-23T19:10:47.727223+0000","last_fullsized":"2024-04-23T19:10:47.727223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:19.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'83","reported_seq":155,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.716534+0000","last_change":"2024-04-23T19:10:13.503368+0000","last_active":"2024-04-23T19:12:20.716534+0000","last_peered":"2024-04-23T19:12:20.716534+0000","last_clean":"2024-04-23T19:12:20.716534+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:12:20.716534+0000","last_undegraded":"2024-04-23T19:12:20.716534+0000","last_fullsized":"2024-04-23T19:12:20.716534+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:31.109448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6026,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":77,"num_read_kb":63,"num_write":30,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727890+0000","last_change":"2024-04-23T19:10:15.500592+0000","last_active":"2024-04-23T19:10:47.727890+0000","last_peered":"2024-04-23T19:10:47.727890+0000","last_clean":"2024-04-23T19:10:47.727890+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:10:47.727890+0000","last_undegraded":"2024-04-23T19:10:47.727890+0000","last_fullsized":"2024-04-23T19:10:47.727890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:11.115258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722791+0000","last_change":"2024-04-23T19:10:17.515628+0000","last_active":"2024-04-23T19:10:47.722791+0000","last_peered":"2024-04-23T19:10:47.722791+0000","last_clean":"2024-04-23T19:10:47.722791+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:10:47.722791+0000","last_undegraded":"2024-04-23T19:10:47.722791+0000","last_fullsized":"2024-04-23T19:10:47.722791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.641905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722100+0000","last_change":"2024-04-23T19:10:17.515232+0000","last_active":"2024-04-23T19:10:47.722100+0000","last_peered":"2024-04-23T19:10:47.722100+0000","last_clean":"2024-04-23T19:10:47.722100+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:10:47.722100+0000","last_undegraded":"2024-04-23T19:10:47.722100+0000","last_fullsized":"2024-04-23T19:10:47.722100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:40.692521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720221+0000","last_change":"2024-04-23T19:10:09.492213+0000","last_active":"2024-04-23T19:10:47.720221+0000","last_peered":"2024-04-23T19:10:47.720221+0000","last_clean":"2024-04-23T19:10:47.720221+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:10:47.720221+0000","last_undegraded":"2024-04-23T19:10:47.720221+0000","last_fullsized":"2024-04-23T19:10:47.720221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:54.477619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726467+0000","last_change":"2024-04-23T19:10:07.500107+0000","last_active":"2024-04-23T19:10:47.726467+0000","last_peered":"2024-04-23T19:10:47.726467+0000","last_clean":"2024-04-23T19:10:47.726467+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:10:47.726467+0000","last_undegraded":"2024-04-23T19:10:47.726467+0000","last_fullsized":"2024-04-23T19:10:47.726467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:14.278078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727809+0000","last_change":"2024-04-23T19:10:47.725041+0000","last_active":"2024-04-23T19:10:47.727809+0000","last_peered":"2024-04-23T19:10:47.727809+0000","last_clean":"2024-04-23T19:10:47.727809+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:10:47.727809+0000","last_undegraded":"2024-04-23T19:10:47.727809+0000","last_fullsized":"2024-04-23T19:10:47.727809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:17.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.317498+0000","last_change":"2024-04-23T19:10:13.497156+0000","last_active":"2024-04-23T19:12:14.317498+0000","last_peered":"2024-04-23T19:12:14.317498+0000","last_clean":"2024-04-23T19:12:14.317498+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:12:14.317498+0000","last_undegraded":"2024-04-23T19:12:14.317498+0000","last_fullsized":"2024-04-23T19:12:14.317498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:49.136472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726980+0000","last_change":"2024-04-23T19:10:15.514289+0000","last_active":"2024-04-23T19:10:47.726980+0000","last_peered":"2024-04-23T19:10:47.726980+0000","last_clean":"2024-04-23T19:10:47.726980+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:10:47.726980+0000","last_undegraded":"2024-04-23T19:10:47.726980+0000","last_fullsized":"2024-04-23T19:10:47.726980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:12.517542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.023661+0000","last_change":"2024-04-23T19:10:17.514011+0000","last_active":"2024-04-23T19:12:14.023661+0000","last_peered":"2024-04-23T19:12:14.023661+0000","last_clean":"2024-04-23T19:12:14.023661+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:12:14.023661+0000","last_undegraded":"2024-04-23T19:12:14.023661+0000","last_fullsized":"2024-04-23T19:12:14.023661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:55:22.164549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.733183+0000","last_change":"2024-04-23T19:10:47.732302+0000","last_active":"2024-04-23T19:10:47.733183+0000","last_peered":"2024-04-23T19:10:47.733183+0000","last_clean":"2024-04-23T19:10:47.733183+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:10:47.733183+0000","last_undegraded":"2024-04-23T19:10:47.733183+0000","last_fullsized":"2024-04-23T19:10:47.733183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:05.759643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.139371+0000","last_change":"2024-04-23T19:10:07.493914+0000","last_active":"2024-04-23T19:13:16.139371+0000","last_peered":"2024-04-23T19:13:16.139371+0000","last_clean":"2024-04-23T19:13:16.139371+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:13:16.139371+0000","last_undegraded":"2024-04-23T19:13:16.139371+0000","last_fullsized":"2024-04-23T19:13:16.139371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729231+0000","last_change":"2024-04-23T19:10:11.487861+0000","last_active":"2024-04-23T19:10:47.729231+0000","last_peered":"2024-04-23T19:10:47.729231+0000","last_clean":"2024-04-23T19:10:47.729231+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:10:47.729231+0000","last_undegraded":"2024-04-23T19:10:47.729231+0000","last_fullsized":"2024-04-23T19:10:47.729231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:47.278160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'5","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789104+0000","last_change":"2024-04-23T19:10:13.494591+0000","last_active":"2024-04-23T19:13:15.789104+0000","last_peered":"2024-04-23T19:13:15.789104+0000","last_clean":"2024-04-23T19:13:15.789104+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:15.789104+0000","last_undegraded":"2024-04-23T19:13:15.789104+0000","last_fullsized":"2024-04-23T19:13:15.789104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:58.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718895+0000","last_change":"2024-04-23T19:10:15.513825+0000","last_active":"2024-04-23T19:10:47.718895+0000","last_peered":"2024-04-23T19:10:47.718895+0000","last_clean":"2024-04-23T19:10:47.718895+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:10:47.718895+0000","last_undegraded":"2024-04-23T19:10:47.718895+0000","last_fullsized":"2024-04-23T19:10:47.718895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728340+0000","last_change":"2024-04-23T19:10:17.520788+0000","last_active":"2024-04-23T19:10:47.728340+0000","last_peered":"2024-04-23T19:10:47.728340+0000","last_clean":"2024-04-23T19:10:47.728340+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:10:47.728340+0000","last_undegraded":"2024-04-23T19:10:47.728340+0000","last_fullsized":"2024-04-23T19:10:47.728340+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:36.438306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399875+0000","last_change":"2024-04-23T19:10:09.491571+0000","last_active":"2024-04-23T19:13:16.399875+0000","last_peered":"2024-04-23T19:13:16.399875+0000","last_clean":"2024-04-23T19:13:16.399875+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:16.399875+0000","last_undegraded":"2024-04-23T19:13:16.399875+0000","last_fullsized":"2024-04-23T19:13:16.399875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724424+0000","last_change":"2024-04-23T19:10:07.500769+0000","last_active":"2024-04-23T19:10:47.724424+0000","last_peered":"2024-04-23T19:10:47.724424+0000","last_clean":"2024-04-23T19:10:47.724424+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:10:47.724424+0000","last_undegraded":"2024-04-23T19:10:47.724424+0000","last_fullsized":"2024-04-23T19:10:47.724424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718451+0000","last_change":"2024-04-23T19:10:11.493115+0000","last_active":"2024-04-23T19:10:47.718451+0000","last_peered":"2024-04-23T19:10:47.718451+0000","last_clean":"2024-04-23T19:10:47.718451+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:10:47.718451+0000","last_undegraded":"2024-04-23T19:10:47.718451+0000","last_fullsized":"2024-04-23T19:10:47.718451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:51.288167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'40","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.848055+0000","last_change":"2024-04-23T19:10:13.491151+0000","last_active":"2024-04-23T19:13:15.848055+0000","last_peered":"2024-04-23T19:13:15.848055+0000","last_clean":"2024-04-23T19:13:15.848055+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:15.848055+0000","last_undegraded":"2024-04-23T19:13:15.848055+0000","last_fullsized":"2024-04-23T19:13:15.848055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:56.656293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717052+0000","last_change":"2024-04-23T19:10:15.497695+0000","last_active":"2024-04-23T19:10:47.717052+0000","last_peered":"2024-04-23T19:10:47.717052+0000","last_clean":"2024-04-23T19:10:47.717052+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:10:47.717052+0000","last_undegraded":"2024-04-23T19:10:47.717052+0000","last_fullsized":"2024-04-23T19:10:47.717052+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:40.709841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.164233+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:10:50.164233+0000","last_peered":"2024-04-23T19:10:50.164233+0000","last_clean":"2024-04-23T19:10:50.164233+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:10:50.164233+0000","last_undegraded":"2024-04-23T19:10:50.164233+0000","last_fullsized":"2024-04-23T19:10:50.164233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:38:04.095526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'9","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165987+0000","last_change":"2024-04-23T19:10:09.498393+0000","last_active":"2024-04-23T19:13:16.165987+0000","last_peered":"2024-04-23T19:13:16.165987+0000","last_clean":"2024-04-23T19:13:16.165987+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:13:16.165987+0000","last_undegraded":"2024-04-23T19:13:16.165987+0000","last_fullsized":"2024-04-23T19:13:16.165987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:24.946389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722073+0000","last_change":"2024-04-23T19:10:07.496451+0000","last_active":"2024-04-23T19:10:47.722073+0000","last_peered":"2024-04-23T19:10:47.722073+0000","last_clean":"2024-04-23T19:10:47.722073+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:10:47.722073+0000","last_undegraded":"2024-04-23T19:10:47.722073+0000","last_fullsized":"2024-04-23T19:10:47.722073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724117+0000","last_change":"2024-04-23T19:10:11.490728+0000","last_active":"2024-04-23T19:10:47.724117+0000","last_peered":"2024-04-23T19:10:47.724117+0000","last_clean":"2024-04-23T19:10:47.724117+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:10:47.724117+0000","last_undegraded":"2024-04-23T19:10:47.724117+0000","last_fullsized":"2024-04-23T19:10:47.724117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:55.751511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'32","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.302845+0000","last_change":"2024-04-23T19:10:13.493330+0000","last_active":"2024-04-23T19:12:14.302845+0000","last_peered":"2024-04-23T19:12:14.302845+0000","last_clean":"2024-04-23T19:12:14.302845+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:12:14.302845+0000","last_undegraded":"2024-04-23T19:12:14.302845+0000","last_fullsized":"2024-04-23T19:12:14.302845+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:48.481253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":44,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719061+0000","last_change":"2024-04-23T19:10:15.508565+0000","last_active":"2024-04-23T19:10:47.719061+0000","last_peered":"2024-04-23T19:10:47.719061+0000","last_clean":"2024-04-23T19:10:47.719061+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:10:47.719061+0000","last_undegraded":"2024-04-23T19:10:47.719061+0000","last_fullsized":"2024-04-23T19:10:47.719061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:02.695164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703848+0000","last_change":"2024-04-23T19:10:47.728408+0000","last_active":"2024-04-23T19:10:50.703848+0000","last_peered":"2024-04-23T19:10:50.703848+0000","last_clean":"2024-04-23T19:10:50.703848+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:10:50.703848+0000","last_undegraded":"2024-04-23T19:10:50.703848+0000","last_fullsized":"2024-04-23T19:10:50.703848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:36.054291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.720730+0000","last_change":"2024-04-23T19:10:07.503907+0000","last_active":"2024-04-23T19:12:15.720730+0000","last_peered":"2024-04-23T19:12:15.720730+0000","last_clean":"2024-04-23T19:12:15.720730+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:12:15.720730+0000","last_undegraded":"2024-04-23T19:12:15.720730+0000","last_fullsized":"2024-04-23T19:12:15.720730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:52.469809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730560+0000","last_change":"2024-04-23T19:10:11.490872+0000","last_active":"2024-04-23T19:10:47.730560+0000","last_peered":"2024-04-23T19:10:47.730560+0000","last_clean":"2024-04-23T19:10:47.730560+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:10:47.730560+0000","last_undegraded":"2024-04-23T19:10:47.730560+0000","last_fullsized":"2024-04-23T19:10:47.730560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'54","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.323713+0000","last_change":"2024-04-23T19:10:13.494268+0000","last_active":"2024-04-23T19:12:14.323713+0000","last_peered":"2024-04-23T19:12:14.323713+0000","last_clean":"2024-04-23T19:12:14.323713+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:12:14.323713+0000","last_undegraded":"2024-04-23T19:12:14.323713+0000","last_fullsized":"2024-04-23T19:12:14.323713+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:00.688062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719088+0000","last_change":"2024-04-23T19:10:15.505820+0000","last_active":"2024-04-23T19:10:47.719088+0000","last_peered":"2024-04-23T19:10:47.719088+0000","last_clean":"2024-04-23T19:10:47.719088+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:10:47.719088+0000","last_undegraded":"2024-04-23T19:10:47.719088+0000","last_fullsized":"2024-04-23T19:10:47.719088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:23.619996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.704248+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:15.704248+0000","last_peered":"2024-04-23T19:12:15.704248+0000","last_clean":"2024-04-23T19:12:15.704248+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:12:15.704248+0000","last_undegraded":"2024-04-23T19:12:15.704248+0000","last_fullsized":"2024-04-23T19:12:15.704248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:58.312384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482427+0000","last_change":"2024-04-23T19:10:09.482621+0000","last_active":"2024-04-23T19:13:16.482427+0000","last_peered":"2024-04-23T19:13:16.482427+0000","last_clean":"2024-04-23T19:13:16.482427+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:13:16.482427+0000","last_undegraded":"2024-04-23T19:13:16.482427+0000","last_fullsized":"2024-04-23T19:13:16.482427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720227+0000","last_change":"2024-04-23T19:10:07.493942+0000","last_active":"2024-04-23T19:12:31.720227+0000","last_peered":"2024-04-23T19:12:31.720227+0000","last_clean":"2024-04-23T19:12:31.720227+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:12:31.720227+0000","last_undegraded":"2024-04-23T19:12:31.720227+0000","last_fullsized":"2024-04-23T19:12:31.720227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721926+0000","last_change":"2024-04-23T19:10:11.486920+0000","last_active":"2024-04-23T19:10:47.721926+0000","last_peered":"2024-04-23T19:10:47.721926+0000","last_clean":"2024-04-23T19:10:47.721926+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:10:47.721926+0000","last_undegraded":"2024-04-23T19:10:47.721926+0000","last_fullsized":"2024-04-23T19:10:47.721926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:56.355675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.303836+0000","last_change":"2024-04-23T19:10:13.502855+0000","last_active":"2024-04-23T19:12:14.303836+0000","last_peered":"2024-04-23T19:12:14.303836+0000","last_clean":"2024-04-23T19:12:14.303836+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:12:14.303836+0000","last_undegraded":"2024-04-23T19:12:14.303836+0000","last_fullsized":"2024-04-23T19:12:14.303836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:09.770521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717958+0000","last_change":"2024-04-23T19:10:15.510094+0000","last_active":"2024-04-23T19:10:47.717958+0000","last_peered":"2024-04-23T19:10:47.717958+0000","last_clean":"2024-04-23T19:10:47.717958+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:10:47.717958+0000","last_undegraded":"2024-04-23T19:10:47.717958+0000","last_fullsized":"2024-04-23T19:10:47.717958+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.609469+0000","last_change":"2024-04-23T19:10:17.517391+0000","last_active":"2024-04-23T19:12:12.609469+0000","last_peered":"2024-04-23T19:12:12.609469+0000","last_clean":"2024-04-23T19:12:12.609469+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:12:12.609469+0000","last_undegraded":"2024-04-23T19:12:12.609469+0000","last_fullsized":"2024-04-23T19:12:12.609469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:35:39.248022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703799+0000","last_change":"2024-04-23T19:10:47.733611+0000","last_active":"2024-04-23T19:10:50.703799+0000","last_peered":"2024-04-23T19:10:50.703799+0000","last_clean":"2024-04-23T19:10:50.703799+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:10:50.703799+0000","last_undegraded":"2024-04-23T19:10:50.703799+0000","last_fullsized":"2024-04-23T19:10:50.703799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:04.349258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717131+0000","last_change":"2024-04-23T19:10:07.502477+0000","last_active":"2024-04-23T19:10:47.717131+0000","last_peered":"2024-04-23T19:10:47.717131+0000","last_clean":"2024-04-23T19:10:47.717131+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:10:47.717131+0000","last_undegraded":"2024-04-23T19:10:47.717131+0000","last_fullsized":"2024-04-23T19:10:47.717131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719460+0000","last_change":"2024-04-23T19:10:11.487445+0000","last_active":"2024-04-23T19:10:47.719460+0000","last_peered":"2024-04-23T19:10:47.719460+0000","last_clean":"2024-04-23T19:10:47.719460+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:10:47.719460+0000","last_undegraded":"2024-04-23T19:10:47.719460+0000","last_fullsized":"2024-04-23T19:10:47.719460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:56.771240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.321490+0000","last_change":"2024-04-23T19:10:13.498101+0000","last_active":"2024-04-23T19:12:14.321490+0000","last_peered":"2024-04-23T19:12:14.321490+0000","last_clean":"2024-04-23T19:12:14.321490+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:12:14.321490+0000","last_undegraded":"2024-04-23T19:12:14.321490+0000","last_fullsized":"2024-04-23T19:12:14.321490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729093+0000","last_change":"2024-04-23T19:10:15.504279+0000","last_active":"2024-04-23T19:10:47.729093+0000","last_peered":"2024-04-23T19:10:47.729093+0000","last_clean":"2024-04-23T19:10:47.729093+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:10:47.729093+0000","last_undegraded":"2024-04-23T19:10:47.729093+0000","last_fullsized":"2024-04-23T19:10:47.729093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:05.168473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.451981+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.451981+0000","last_peered":"2024-04-23T19:13:16.451981+0000","last_clean":"2024-04-23T19:13:16.451981+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.451981+0000","last_undegraded":"2024-04-23T19:13:16.451981+0000","last_fullsized":"2024-04-23T19:13:16.451981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:52.410520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399955+0000","last_change":"2024-04-23T19:10:09.477657+0000","last_active":"2024-04-23T19:13:16.399955+0000","last_peered":"2024-04-23T19:13:16.399955+0000","last_clean":"2024-04-23T19:13:16.399955+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:16.399955+0000","last_undegraded":"2024-04-23T19:13:16.399955+0000","last_fullsized":"2024-04-23T19:13:16.399955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.117950+0000","last_change":"2024-04-23T19:10:07.504027+0000","last_active":"2024-04-23T19:13:16.117950+0000","last_peered":"2024-04-23T19:13:16.117950+0000","last_clean":"2024-04-23T19:13:16.117950+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:13:16.117950+0000","last_undegraded":"2024-04-23T19:13:16.117950+0000","last_fullsized":"2024-04-23T19:13:16.117950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:34.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'4","reported_seq":226,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.374959+0000","last_change":"2024-04-23T19:10:11.485789+0000","last_active":"2024-04-23T19:12:15.374959+0000","last_peered":"2024-04-23T19:12:15.374959+0000","last_clean":"2024-04-23T19:12:15.374959+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:12:15.374959+0000","last_undegraded":"2024-04-23T19:12:15.374959+0000","last_fullsized":"2024-04-23T19:12:15.374959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:54.297420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728532+0000","last_change":"2024-04-23T19:10:13.502626+0000","last_active":"2024-04-23T19:10:47.728532+0000","last_peered":"2024-04-23T19:10:47.728532+0000","last_clean":"2024-04-23T19:10:47.728532+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:10:47.728532+0000","last_undegraded":"2024-04-23T19:10:47.728532+0000","last_fullsized":"2024-04-23T19:10:47.728532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:27.703823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'282","reported_seq":417,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575740+0000","last_change":"2024-04-23T19:10:15.510611+0000","last_active":"2024-04-23T19:13:16.575740+0000","last_peered":"2024-04-23T19:13:16.575740+0000","last_clean":"2024-04-23T19:13:16.575740+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:16.575740+0000","last_undegraded":"2024-04-23T19:13:16.575740+0000","last_fullsized":"2024-04-23T19:13:16.575740+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:00.157929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725003+0000","last_change":"2024-04-23T19:10:17.513830+0000","last_active":"2024-04-23T19:10:47.725003+0000","last_peered":"2024-04-23T19:10:47.725003+0000","last_clean":"2024-04-23T19:10:47.725003+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:10:47.725003+0000","last_undegraded":"2024-04-23T19:10:47.725003+0000","last_fullsized":"2024-04-23T19:10:47.725003+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.594849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.150535+0000","last_change":"2024-04-23T19:10:09.476601+0000","last_active":"2024-04-23T19:11:32.150535+0000","last_peered":"2024-04-23T19:11:32.150535+0000","last_clean":"2024-04-23T19:11:32.150535+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:11:32.150535+0000","last_undegraded":"2024-04-23T19:11:32.150535+0000","last_fullsized":"2024-04-23T19:11:32.150535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:10.224761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718929+0000","last_change":"2024-04-23T19:10:07.497992+0000","last_active":"2024-04-23T19:10:47.718929+0000","last_peered":"2024-04-23T19:10:47.718929+0000","last_clean":"2024-04-23T19:10:47.718929+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:10:47.718929+0000","last_undegraded":"2024-04-23T19:10:47.718929+0000","last_fullsized":"2024-04-23T19:10:47.718929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720669+0000","last_change":"2024-04-23T19:10:11.489334+0000","last_active":"2024-04-23T19:10:47.720669+0000","last_peered":"2024-04-23T19:10:47.720669+0000","last_clean":"2024-04-23T19:10:47.720669+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:10:47.720669+0000","last_undegraded":"2024-04-23T19:10:47.720669+0000","last_fullsized":"2024-04-23T19:10:47.720669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'37","reported_seq":82,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.294646+0000","last_change":"2024-04-23T19:10:13.492238+0000","last_active":"2024-04-23T19:12:14.294646+0000","last_peered":"2024-04-23T19:12:14.294646+0000","last_clean":"2024-04-23T19:12:14.294646+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:12:14.294646+0000","last_undegraded":"2024-04-23T19:12:14.294646+0000","last_fullsized":"2024-04-23T19:12:14.294646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726624+0000","last_change":"2024-04-23T19:10:15.502657+0000","last_active":"2024-04-23T19:10:47.726624+0000","last_peered":"2024-04-23T19:10:47.726624+0000","last_clean":"2024-04-23T19:10:47.726624+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:10:47.726624+0000","last_undegraded":"2024-04-23T19:10:47.726624+0000","last_fullsized":"2024-04-23T19:10:47.726624+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722842+0000","last_change":"2024-04-23T19:10:17.518482+0000","last_active":"2024-04-23T19:10:47.722842+0000","last_peered":"2024-04-23T19:10:47.722842+0000","last_clean":"2024-04-23T19:10:47.722842+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:10:47.722842+0000","last_undegraded":"2024-04-23T19:10:47.722842+0000","last_fullsized":"2024-04-23T19:10:47.722842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'3","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.405706+0000","last_change":"2024-04-23T19:10:09.477644+0000","last_active":"2024-04-23T19:13:16.405706+0000","last_peered":"2024-04-23T19:13:16.405706+0000","last_clean":"2024-04-23T19:13:16.405706+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:16.405706+0000","last_undegraded":"2024-04-23T19:13:16.405706+0000","last_fullsized":"2024-04-23T19:13:16.405706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728885+0000","last_change":"2024-04-23T19:10:07.503802+0000","last_active":"2024-04-23T19:10:47.728885+0000","last_peered":"2024-04-23T19:10:47.728885+0000","last_clean":"2024-04-23T19:10:47.728885+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:10:47.728885+0000","last_undegraded":"2024-04-23T19:10:47.728885+0000","last_fullsized":"2024-04-23T19:10:47.728885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:03.446904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720608+0000","last_change":"2024-04-23T19:10:11.489940+0000","last_active":"2024-04-23T19:10:47.720608+0000","last_peered":"2024-04-23T19:10:47.720608+0000","last_clean":"2024-04-23T19:10:47.720608+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:10:47.720608+0000","last_undegraded":"2024-04-23T19:10:47.720608+0000","last_fullsized":"2024-04-23T19:10:47.720608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:35.423825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.295694+0000","last_change":"2024-04-23T19:10:13.505761+0000","last_active":"2024-04-23T19:12:14.295694+0000","last_peered":"2024-04-23T19:12:14.295694+0000","last_clean":"2024-04-23T19:12:14.295694+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:12:14.295694+0000","last_undegraded":"2024-04-23T19:12:14.295694+0000","last_fullsized":"2024-04-23T19:12:14.295694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:28.957267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.723783+0000","last_change":"2024-04-23T19:10:15.514687+0000","last_active":"2024-04-23T19:10:47.723783+0000","last_peered":"2024-04-23T19:10:47.723783+0000","last_clean":"2024-04-23T19:10:47.723783+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:10:47.723783+0000","last_undegraded":"2024-04-23T19:10:47.723783+0000","last_fullsized":"2024-04-23T19:10:47.723783+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:29.022733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.768829+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:15.768829+0000","last_peered":"2024-04-23T19:12:15.768829+0000","last_clean":"2024-04-23T19:12:15.768829+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:15.768829+0000","last_undegraded":"2024-04-23T19:12:15.768829+0000","last_fullsized":"2024-04-23T19:12:15.768829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:42.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'11","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165914+0000","last_change":"2024-04-23T19:10:09.490516+0000","last_active":"2024-04-23T19:13:16.165914+0000","last_peered":"2024-04-23T19:13:16.165914+0000","last_clean":"2024-04-23T19:13:16.165914+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:13:16.165914+0000","last_undegraded":"2024-04-23T19:13:16.165914+0000","last_fullsized":"2024-04-23T19:13:16.165914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:54.586490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729712+0000","last_change":"2024-04-23T19:10:07.486770+0000","last_active":"2024-04-23T19:10:47.729712+0000","last_peered":"2024-04-23T19:10:47.729712+0000","last_clean":"2024-04-23T19:10:47.729712+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:10:47.729712+0000","last_undegraded":"2024-04-23T19:10:47.729712+0000","last_fullsized":"2024-04-23T19:10:47.729712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:56:06.039657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717835+0000","last_change":"2024-04-23T19:10:11.491433+0000","last_active":"2024-04-23T19:10:47.717835+0000","last_peered":"2024-04-23T19:10:47.717835+0000","last_clean":"2024-04-23T19:10:47.717835+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:10:47.717835+0000","last_undegraded":"2024-04-23T19:10:47.717835+0000","last_fullsized":"2024-04-23T19:10:47.717835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'26","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.319314+0000","last_change":"2024-04-23T19:10:13.503113+0000","last_active":"2024-04-23T19:12:14.319314+0000","last_peered":"2024-04-23T19:12:14.319314+0000","last_clean":"2024-04-23T19:12:14.319314+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:12:14.319314+0000","last_undegraded":"2024-04-23T19:12:14.319314+0000","last_fullsized":"2024-04-23T19:12:14.319314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:25.616871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726203+0000","last_change":"2024-04-23T19:10:15.512937+0000","last_active":"2024-04-23T19:10:47.726203+0000","last_peered":"2024-04-23T19:10:47.726203+0000","last_clean":"2024-04-23T19:10:47.726203+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:10:47.726203+0000","last_undegraded":"2024-04-23T19:10:47.726203+0000","last_fullsized":"2024-04-23T19:10:47.726203+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:07.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.329498+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:14.329498+0000","last_peered":"2024-04-23T19:12:14.329498+0000","last_clean":"2024-04-23T19:12:14.329498+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:14.329498+0000","last_undegraded":"2024-04-23T19:12:14.329498+0000","last_fullsized":"2024-04-23T19:12:14.329498+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:30.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482420+0000","last_change":"2024-04-23T19:10:09.476153+0000","last_active":"2024-04-23T19:13:16.482420+0000","last_peered":"2024-04-23T19:13:16.482420+0000","last_clean":"2024-04-23T19:13:16.482420+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:16.482420+0000","last_undegraded":"2024-04-23T19:13:16.482420+0000","last_fullsized":"2024-04-23T19:13:16.482420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:07.680114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.716957+0000","last_change":"2024-04-23T19:10:07.501211+0000","last_active":"2024-04-23T19:12:11.716957+0000","last_peered":"2024-04-23T19:12:11.716957+0000","last_clean":"2024-04-23T19:12:11.716957+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:12:11.716957+0000","last_undegraded":"2024-04-23T19:12:11.716957+0000","last_fullsized":"2024-04-23T19:12:11.716957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'2","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.979180+0000","last_change":"2024-04-23T19:10:11.491220+0000","last_active":"2024-04-23T19:12:12.979180+0000","last_peered":"2024-04-23T19:12:12.979180+0000","last_clean":"2024-04-23T19:12:12.979180+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:12:12.979180+0000","last_undegraded":"2024-04-23T19:12:12.979180+0000","last_fullsized":"2024-04-23T19:12:12.979180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:56.767918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'32","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789943+0000","last_change":"2024-04-23T19:10:13.502743+0000","last_active":"2024-04-23T19:13:15.789943+0000","last_peered":"2024-04-23T19:13:15.789943+0000","last_clean":"2024-04-23T19:13:15.789943+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:15.789943+0000","last_undegraded":"2024-04-23T19:13:15.789943+0000","last_fullsized":"2024-04-23T19:13:15.789943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:13.230238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727977+0000","last_change":"2024-04-23T19:10:15.502314+0000","last_active":"2024-04-23T19:10:47.727977+0000","last_peered":"2024-04-23T19:10:47.727977+0000","last_clean":"2024-04-23T19:10:47.727977+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:10:47.727977+0000","last_undegraded":"2024-04-23T19:10:47.727977+0000","last_fullsized":"2024-04-23T19:10:47.727977+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:26.867666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.434484+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.434484+0000","last_peered":"2024-04-23T19:13:16.434484+0000","last_clean":"2024-04-23T19:13:16.434484+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.434484+0000","last_undegraded":"2024-04-23T19:13:16.434484+0000","last_fullsized":"2024-04-23T19:13:16.434484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:47.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489162+0000","last_change":"2024-04-23T19:10:09.494269+0000","last_active":"2024-04-23T19:13:16.489162+0000","last_peered":"2024-04-23T19:13:16.489162+0000","last_clean":"2024-04-23T19:13:16.489162+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:16.489162+0000","last_undegraded":"2024-04-23T19:13:16.489162+0000","last_fullsized":"2024-04-23T19:13:16.489162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:25.844962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.477833+0000","last_change":"2024-04-23T19:10:07.492825+0000","last_active":"2024-04-23T19:13:16.477833+0000","last_peered":"2024-04-23T19:13:16.477833+0000","last_clean":"2024-04-23T19:13:16.477833+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:13:16.477833+0000","last_undegraded":"2024-04-23T19:13:16.477833+0000","last_fullsized":"2024-04-23T19:13:16.477833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771507+0000","last_change":"2024-04-23T19:10:09.478781+0000","last_active":"2024-04-23T19:12:15.771507+0000","last_peered":"2024-04-23T19:12:15.771507+0000","last_clean":"2024-04-23T19:12:15.771507+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:12:15.771507+0000","last_undegraded":"2024-04-23T19:12:15.771507+0000","last_fullsized":"2024-04-23T19:12:15.771507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710865+0000","last_change":"2024-04-23T19:10:07.489213+0000","last_active":"2024-04-23T19:12:15.710865+0000","last_peered":"2024-04-23T19:12:15.710865+0000","last_clean":"2024-04-23T19:12:15.710865+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:12:15.710865+0000","last_undegraded":"2024-04-23T19:12:15.710865+0000","last_fullsized":"2024-04-23T19:12:15.710865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.495393+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:17.602017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726729+0000","last_change":"2024-04-23T19:10:07.506697+0000","last_active":"2024-04-23T19:10:47.726729+0000","last_peered":"2024-04-23T19:10:47.726729+0000","last_clean":"2024-04-23T19:10:47.726729+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:10:47.726729+0000","last_undegraded":"2024-04-23T19:10:47.726729+0000","last_fullsized":"2024-04-23T19:10:47.726729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:36.637883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.158375+0000","last_change":"2024-04-23T19:10:09.478697+0000","last_active":"2024-04-23T19:13:16.158375+0000","last_peered":"2024-04-23T19:13:16.158375+0000","last_clean":"2024-04-23T19:13:16.158375+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:13:16.158375+0000","last_undegraded":"2024-04-23T19:13:16.158375+0000","last_fullsized":"2024-04-23T19:13:16.158375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'3","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710919+0000","last_change":"2024-04-23T19:10:07.490623+0000","last_active":"2024-04-23T19:12:15.710919+0000","last_peered":"2024-04-23T19:12:15.710919+0000","last_clean":"2024-04-23T19:12:15.710919+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:12:15.710919+0000","last_undegraded":"2024-04-23T19:12:15.710919+0000","last_fullsized":"2024-04-23T19:12:15.710919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698550+0000","last_change":"2024-04-23T19:10:09.491248+0000","last_active":"2024-04-23T19:10:49.698550+0000","last_peered":"2024-04-23T19:10:49.698550+0000","last_clean":"2024-04-23T19:10:49.698550+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:10:49.698550+0000","last_undegraded":"2024-04-23T19:10:49.698550+0000","last_fullsized":"2024-04-23T19:10:49.698550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729029+0000","last_change":"2024-04-23T19:10:07.490715+0000","last_active":"2024-04-23T19:10:47.729029+0000","last_peered":"2024-04-23T19:10:47.729029+0000","last_clean":"2024-04-23T19:10:47.729029+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:10:47.729029+0000","last_undegraded":"2024-04-23T19:10:47.729029+0000","last_fullsized":"2024-04-23T19:10:47.729029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:03.712798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447089+0000","last_change":"2024-04-23T19:10:09.494954+0000","last_active":"2024-04-23T19:13:16.447089+0000","last_peered":"2024-04-23T19:13:16.447089+0000","last_clean":"2024-04-23T19:13:16.447089+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:16.447089+0000","last_undegraded":"2024-04-23T19:13:16.447089+0000","last_fullsized":"2024-04-23T19:13:16.447089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.351485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.548131+0000","last_change":"2024-04-23T19:10:07.490753+0000","last_active":"2024-04-23T19:10:48.548131+0000","last_peered":"2024-04-23T19:10:48.548131+0000","last_clean":"2024-04-23T19:10:48.548131+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:10:48.548131+0000","last_undegraded":"2024-04-23T19:10:48.548131+0000","last_fullsized":"2024-04-23T19:10:48.548131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.157844+0000","last_change":"2024-04-23T19:10:09.482692+0000","last_active":"2024-04-23T19:13:16.157844+0000","last_peered":"2024-04-23T19:13:16.157844+0000","last_clean":"2024-04-23T19:13:16.157844+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:13:16.157844+0000","last_undegraded":"2024-04-23T19:13:16.157844+0000","last_fullsized":"2024-04-23T19:13:16.157844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.380363+0000","last_change":"2024-04-23T19:10:07.500499+0000","last_active":"2024-04-23T19:13:16.380363+0000","last_peered":"2024-04-23T19:13:16.380363+0000","last_clean":"2024-04-23T19:13:16.380363+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:16.380363+0000","last_undegraded":"2024-04-23T19:13:16.380363+0000","last_fullsized":"2024-04-23T19:13:16.380363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258229+0000","last_change":"2024-04-23T19:10:09.484981+0000","last_active":"2024-04-23T19:13:16.258229+0000","last_peered":"2024-04-23T19:13:16.258229+0000","last_clean":"2024-04-23T19:13:16.258229+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:13:16.258229+0000","last_undegraded":"2024-04-23T19:13:16.258229+0000","last_fullsized":"2024-04-23T19:13:16.258229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.951884+0000","last_change":"2024-04-23T19:10:07.490970+0000","last_active":"2024-04-23T19:11:31.951884+0000","last_peered":"2024-04-23T19:11:31.951884+0000","last_clean":"2024-04-23T19:11:31.951884+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:11:31.951884+0000","last_undegraded":"2024-04-23T19:11:31.951884+0000","last_fullsized":"2024-04-23T19:11:31.951884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:19.094583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698565+0000","last_change":"2024-04-23T19:10:09.484268+0000","last_active":"2024-04-23T19:10:49.698565+0000","last_peered":"2024-04-23T19:10:49.698565+0000","last_clean":"2024-04-23T19:10:49.698565+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:10:49.698565+0000","last_undegraded":"2024-04-23T19:10:49.698565+0000","last_fullsized":"2024-04-23T19:10:49.698565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:46.058996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726273+0000","last_change":"2024-04-23T19:10:07.507068+0000","last_active":"2024-04-23T19:10:47.726273+0000","last_peered":"2024-04-23T19:10:47.726273+0000","last_clean":"2024-04-23T19:10:47.726273+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:10:47.726273+0000","last_undegraded":"2024-04-23T19:10:47.726273+0000","last_fullsized":"2024-04-23T19:10:47.726273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.361176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489138+0000","last_change":"2024-04-23T19:10:09.498436+0000","last_active":"2024-04-23T19:13:16.489138+0000","last_peered":"2024-04-23T19:13:16.489138+0000","last_clean":"2024-04-23T19:13:16.489138+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:13:16.489138+0000","last_undegraded":"2024-04-23T19:13:16.489138+0000","last_fullsized":"2024-04-23T19:13:16.489138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:55.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.685736+0000","last_change":"2024-04-23T19:10:07.489441+0000","last_active":"2024-04-23T19:10:49.685736+0000","last_peered":"2024-04-23T19:10:49.685736+0000","last_clean":"2024-04-23T19:10:49.685736+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:10:49.685736+0000","last_undegraded":"2024-04-23T19:10:49.685736+0000","last_fullsized":"2024-04-23T19:10:49.685736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'5","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712408+0000","last_change":"2024-04-23T19:10:09.492013+0000","last_active":"2024-04-23T19:12:25.712408+0000","last_peered":"2024-04-23T19:12:25.712408+0000","last_clean":"2024-04-23T19:12:25.712408+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:12:25.712408+0000","last_undegraded":"2024-04-23T19:12:25.712408+0000","last_fullsized":"2024-04-23T19:12:25.712408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717325+0000","last_change":"2024-04-23T19:10:07.490880+0000","last_active":"2024-04-23T19:10:47.717325+0000","last_peered":"2024-04-23T19:10:47.717325+0000","last_clean":"2024-04-23T19:10:47.717325+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:10:47.717325+0000","last_undegraded":"2024-04-23T19:10:47.717325+0000","last_fullsized":"2024-04-23T19:10:47.717325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.697786+0000","last_change":"2024-04-23T19:10:09.482277+0000","last_active":"2024-04-23T19:10:49.697786+0000","last_peered":"2024-04-23T19:10:49.697786+0000","last_clean":"2024-04-23T19:10:49.697786+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:10:49.697786+0000","last_undegraded":"2024-04-23T19:10:49.697786+0000","last_fullsized":"2024-04-23T19:10:49.697786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719034+0000","last_change":"2024-04-23T19:10:07.502896+0000","last_active":"2024-04-23T19:10:47.719034+0000","last_peered":"2024-04-23T19:10:47.719034+0000","last_clean":"2024-04-23T19:10:47.719034+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:10:47.719034+0000","last_undegraded":"2024-04-23T19:10:47.719034+0000","last_fullsized":"2024-04-23T19:10:47.719034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.964394+0000","last_change":"2024-04-23T19:10:09.492153+0000","last_active":"2024-04-23T19:11:31.964394+0000","last_peered":"2024-04-23T19:11:31.964394+0000","last_clean":"2024-04-23T19:11:31.964394+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:11:31.964394+0000","last_undegraded":"2024-04-23T19:11:31.964394+0000","last_fullsized":"2024-04-23T19:11:31.964394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.490658+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.568491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720741+0000","last_change":"2024-04-23T19:10:09.483025+0000","last_active":"2024-04-23T19:10:47.720741+0000","last_peered":"2024-04-23T19:10:47.720741+0000","last_clean":"2024-04-23T19:10:47.720741+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:10:47.720741+0000","last_undegraded":"2024-04-23T19:10:47.720741+0000","last_fullsized":"2024-04-23T19:10:47.720741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:47.851863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729169+0000","last_change":"2024-04-23T19:10:07.502808+0000","last_active":"2024-04-23T19:10:47.729169+0000","last_peered":"2024-04-23T19:10:47.729169+0000","last_clean":"2024-04-23T19:10:47.729169+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:10:47.729169+0000","last_undegraded":"2024-04-23T19:10:47.729169+0000","last_fullsized":"2024-04-23T19:10:47.729169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:11.223317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'5","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719983+0000","last_change":"2024-04-23T19:10:09.495999+0000","last_active":"2024-04-23T19:12:26.719983+0000","last_peered":"2024-04-23T19:12:26.719983+0000","last_clean":"2024-04-23T19:12:26.719983+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:12:26.719983+0000","last_undegraded":"2024-04-23T19:12:26.719983+0000","last_fullsized":"2024-04-23T19:12:26.719983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:49.818415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725442+0000","last_change":"2024-04-23T19:10:07.500476+0000","last_active":"2024-04-23T19:10:47.725442+0000","last_peered":"2024-04-23T19:10:47.725442+0000","last_clean":"2024-04-23T19:10:47.725442+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:10:47.725442+0000","last_undegraded":"2024-04-23T19:10:47.725442+0000","last_fullsized":"2024-04-23T19:10:47.725442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258427+0000","last_change":"2024-04-23T19:10:09.489995+0000","last_active":"2024-04-23T19:13:16.258427+0000","last_peered":"2024-04-23T19:13:16.258427+0000","last_clean":"2024-04-23T19:13:16.258427+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:13:16.258427+0000","last_undegraded":"2024-04-23T19:13:16.258427+0000","last_fullsized":"2024-04-23T19:13:16.258427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728417+0000","last_change":"2024-04-23T19:10:07.507910+0000","last_active":"2024-04-23T19:10:47.728417+0000","last_peered":"2024-04-23T19:10:47.728417+0000","last_clean":"2024-04-23T19:10:47.728417+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:10:47.728417+0000","last_undegraded":"2024-04-23T19:10:47.728417+0000","last_fullsized":"2024-04-23T19:10:47.728417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:27.025595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.503476+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:39.006602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399931+0000","last_change":"2024-04-23T19:10:09.487033+0000","last_active":"2024-04-23T19:13:16.399931+0000","last_peered":"2024-04-23T19:13:16.399931+0000","last_clean":"2024-04-23T19:13:16.399931+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:13:16.399931+0000","last_undegraded":"2024-04-23T19:13:16.399931+0000","last_fullsized":"2024-04-23T19:13:16.399931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:36.162060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.392374+0000","last_change":"2024-04-23T19:10:07.486697+0000","last_active":"2024-04-23T19:13:16.392374+0000","last_peered":"2024-04-23T19:13:16.392374+0000","last_clean":"2024-04-23T19:13:16.392374+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:13:16.392374+0000","last_undegraded":"2024-04-23T19:13:16.392374+0000","last_fullsized":"2024-04-23T19:13:16.392374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.159090+0000","last_change":"2024-04-23T19:10:09.491506+0000","last_active":"2024-04-23T19:13:16.159090+0000","last_peered":"2024-04-23T19:13:16.159090+0000","last_clean":"2024-04-23T19:13:16.159090+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:16.159090+0000","last_undegraded":"2024-04-23T19:13:16.159090+0000","last_fullsized":"2024-04-23T19:13:16.159090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13384,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":38719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673007,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46348,"kb_used_data":11012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325492,"statfs":{"total":96636764160,"available":96589303808,"internally_reserved":0,"allocated":11276288,"data_stored":10955671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673006,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44844,"kb_used_data":9508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326996,"statfs":{"total":96636764160,"available":96590843904,"internally_reserved":0,"allocated":9736192,"data_stored":9446289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46552,"kb_used_data":11216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325288,"statfs":{"total":96636764160,"available":96589094912,"internally_reserved":0,"allocated":11485184,"data_stored":11220394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40100,"kb_used_data":8924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331740,"statfs":{"total":96636764160,"available":96595701760,"internally_reserved":0,"allocated":9138176,"data_stored":8856588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10809344,"data_stored":10809344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:13:24.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T19:13:24.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.407+0000 7f8b4e3f7640 1 Processor -- start 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.407+0000 7f8b4e3f7640 1 -- start start 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b4e3f7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b4812b7a0 0x7f8b4812bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b4e3f7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4809aa00 con 0x7f8b4812c0c0 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b4e3f7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8b480a20d0 con 0x7f8b4812b7a0 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b47fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b4812b7a0 0x7f8b4812bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b477fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8b4812c0c0 0x7f8b4812eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38546/0 (socket says 172.21.15.16:38546) 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b477fe640 1 -- 172.21.15.16:0/1396691342 learned_addr learned my addr 172.21.15.16:0/1396691342 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b47fff640 1 -- 172.21.15.16:0/1396691342 >> v1:172.21.15.16:6789/0 conn(0x7f8b4812c0c0 legacy=0x7f8b4812eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.408+0000 7f8b47fff640 1 -- 172.21.15.16:0/1396691342 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4809af50 con 0x7f8b4812b7a0 2024-04-23T19:13:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.409+0000 7f8b47fff640 1 --2- 172.21.15.16:0/1396691342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b4812bb80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009720 tx=0x7f8b3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5632293095cf4ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.409+0000 7f8b46ffd640 1 -- 172.21.15.16:0/1396691342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b380316d0 con 0x7f8b4812b7a0 2024-04-23T19:13:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.409+0000 7f8b46ffd640 1 -- 172.21.15.16:0/1396691342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b38037070 con 0x7f8b4812b7a0 2024-04-23T19:13:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.409+0000 7f8b46ffd640 1 -- 172.21.15.16:0/1396691342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b380319c0 con 0x7f8b4812b7a0 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.410+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1396691342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 msgr2=0x7f8b4812bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.410+0000 7f8b4e3f7640 1 --2- 172.21.15.16:0/1396691342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b4812bb80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009720 tx=0x7f8b3802b840 comp rx=0 tx=0).stop 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.410+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1396691342 shutdown_connections 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.410+0000 7f8b4e3f7640 1 --2- 172.21.15.16:0/1396691342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b4812bb80 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.410+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1396691342 >> 172.21.15.16:0/1396691342 conn(0x7f8b480a15b0 msgr2=0x7f8b480a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.410+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1396691342 shutdown_connections 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.410+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1396691342 wait complete. 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.411+0000 7f8b4e3f7640 1 Processor -- start 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.411+0000 7f8b4e3f7640 1 -- start start 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.411+0000 7f8b4e3f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b48070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.411+0000 7f8b4e3f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4812c500 con 0x7f8b4812b7a0 2024-04-23T19:13:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.411+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b48070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.411+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b48070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39010/0 (socket says 172.21.15.16:39010) 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.411+0000 7f8b47fff640 1 -- 172.21.15.16:0/1933699146 learned_addr learned my addr 172.21.15.16:0/1933699146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.412+0000 7f8b47fff640 1 -- 172.21.15.16:0/1933699146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b48071e40 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.412+0000 7f8b47fff640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b48070410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009720 tx=0x7f8b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.412+0000 7f8b44ff9640 1 -- 172.21.15.16:0/1933699146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b38032770 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.412+0000 7f8b44ff9640 1 -- 172.21.15.16:0/1933699146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b38032910 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.412+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4808fad0 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.412+0000 7f8b44ff9640 1 -- 172.21.15.16:0/1933699146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b38032bf0 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.412+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b48070950 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.413+0000 7f8b44ff9640 1 -- 172.21.15.16:0/1933699146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f8b3804c060 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.413+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b48063660 con 0x7f8b4812b7a0 2024-04-23T19:13:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.413+0000 7f8b44ff9640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b280385c0 0x7f8b2803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.413+0000 7f8b44ff9640 1 -- 172.21.15.16:0/1933699146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f8b3802f050 con 0x7f8b4812b7a0 2024-04-23T19:13:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.413+0000 7f8b477fe640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b280385c0 0x7f8b2803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.414+0000 7f8b477fe640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b280385c0 0x7f8b2803aa80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b30007fd0 tx=0x7f8b30007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.418+0000 7f8b44ff9640 1 -- 172.21.15.16:0/1933699146 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b38040070 con 0x7f8b4812b7a0 2024-04-23T19:13:24.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.602+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b48067f20 con 0x7f8b4812b7a0 2024-04-23T19:13:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.604+0000 7f8b44ff9640 1 -- 172.21.15.16:0/1933699146 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+20925 (secure 0 0 0) 0x7f8b38044d70 con 0x7f8b4812b7a0 2024-04-23T19:13:24.607 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:13:24.607 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"f80ae34c-e526-409e-90c3-45f2fcaa83f6","created":"2024-04-23T19:09:42.236475+0000","modified":"2024-04-23T19:10:47.711440+0000","last_up_change":"2024-04-23T19:09:51.428209+0000","last_in_change":"2024-04-23T19:09:43.346338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T19:09:50.651043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T19:09:57.875436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-04-23T19:10:05.460594+0000","flags":1,"flags_names":"hashpspool","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"client.0","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-04-23T19:10:07.803905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-04-23T19:10:09.750803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-04-23T19:10:11.560391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-04-23T19:10:13.508907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-04-23T19:10:15.580169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8925ca3-b81e-4e90-9554-7a26a6b9ee13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6801","nonce":2262294208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6803","nonce":2262294208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6807","nonce":2262294208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2262294208},{"type":"v1","addr":"172.21.15.16:6805","nonce":2262294208}]},"public_addr":"172.21.15.16:6801/2262294208","cluster_addr":"172.21.15.16:6803/2262294208","heartbeat_back_addr":"172.21.15.16:6807/2262294208","heartbeat_front_addr":"172.21.15.16:6805/2262294208","state":["exists","up"]},{"osd":1,"uuid":"6bf01767-14ca-4771-86ec-2369b6eefd4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6801","nonce":3216879695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6803","nonce":3216879695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6807","nonce":3216879695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3216879695},{"type":"v1","addr":"172.21.15.63:6805","nonce":3216879695}]},"public_addr":"172.21.15.63:6801/3216879695","cluster_addr":"172.21.15.63:6803/3216879695","heartbeat_back_addr":"172.21.15.63:6807/3216879695","heartbeat_front_addr":"172.21.15.63:6805/3216879695","state":["exists","up"]},{"osd":2,"uuid":"3a1dcf6d-e482-4751-babc-9b0c896a3b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6809","nonce":753864785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6811","nonce":753864785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6815","nonce":753864785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":753864785},{"type":"v1","addr":"172.21.15.63:6813","nonce":753864785}]},"public_addr":"172.21.15.63:6809/753864785","cluster_addr":"172.21.15.63:6811/753864785","heartbeat_back_addr":"172.21.15.63:6815/753864785","heartbeat_front_addr":"172.21.15.63:6813/753864785","state":["exists","up"]},{"osd":3,"uuid":"534b421f-9079-4429-ac83-791f6222a368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6817","nonce":1289558434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6819","nonce":1289558434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6823","nonce":1289558434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1289558434},{"type":"v1","addr":"172.21.15.63:6821","nonce":1289558434}]},"public_addr":"172.21.15.63:6817/1289558434","cluster_addr":"172.21.15.63:6819/1289558434","heartbeat_back_addr":"172.21.15.63:6823/1289558434","heartbeat_front_addr":"172.21.15.63:6821/1289558434","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.387629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.445168+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.449933+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T19:09:48.519415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":3,"to":0}]},{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"5.3","mappings":[{"from":0,"to":2}]},{"pgid":"5.6","mappings":[{"from":0,"to":1}]},{"pgid":"5.12","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"client.0":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b280385c0 msgr2=0x7f8b2803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b280385c0 0x7f8b2803aa80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b30007fd0 tx=0x7f8b30007ac0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 msgr2=0x7f8b48070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b48070410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009720 tx=0x7f8b380046c0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 shutdown_connections 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f8b280385c0 0x7f8b2803aa80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 --2- 172.21.15.16:0/1933699146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b4812b7a0 0x7f8b48070410 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:24.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 >> 172.21.15.16:0/1933699146 conn(0x7f8b480a15b0 msgr2=0x7f8b4809bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:24.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 shutdown_connections 2024-04-23T19:13:24.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:24.606+0000 7f8b4e3f7640 1 -- 172.21.15.16:0/1933699146 wait complete. 2024-04-23T19:13:25.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-04-23T19:13:25.621 INFO:tasks.ceph:Scrubbing osd.0 2024-04-23T19:13:25.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-23T19:13:25.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.736+0000 7f65c848c640 1 Processor -- start 2024-04-23T19:13:25.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.736+0000 7f65c848c640 1 -- start start 2024-04-23T19:13:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c848c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65c0099810 0x7f65c01343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c848c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c009a690 con 0x7f65c0098d20 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c848c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f65c00982e0 con 0x7f65c0099810 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c5a27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65c0099810 0x7f65c01343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c6228640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f65c0098d20 0x7f65c0099100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57930/0 (socket says 172.21.15.16:57930) 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c6228640 1 -- 172.21.15.16:0/680447931 learned_addr learned my addr 172.21.15.16:0/680447931 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c5a27640 1 -- 172.21.15.16:0/680447931 >> v1:172.21.15.16:6789/0 conn(0x7f65c0098d20 legacy=0x7f65c0099100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c5a27640 1 -- 172.21.15.16:0/680447931 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c0099c60 con 0x7f65c0099810 2024-04-23T19:13:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.737+0000 7f65c5226640 1 -- 172.21.15.16:0/680447931 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2921517196 0 0) 0x7f65c009a690 con 0x7f65c0098d20 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.738+0000 7f65c5a27640 1 --2- 172.21.15.16:0/680447931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0099810 0x7f65c01343a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009720 tx=0x7f65b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec96ab397d5b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.738+0000 7f65c5226640 1 -- 172.21.15.16:0/680447931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b00316d0 con 0x7f65c0099810 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.738+0000 7f65c5226640 1 -- 172.21.15.16:0/680447931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b0037070 con 0x7f65c0099810 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.738+0000 7f65c5226640 1 -- 172.21.15.16:0/680447931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65b00319c0 con 0x7f65c0099810 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.739+0000 7f65c848c640 1 -- 172.21.15.16:0/680447931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0099810 msgr2=0x7f65c01343a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.739+0000 7f65c848c640 1 --2- 172.21.15.16:0/680447931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0099810 0x7f65c01343a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009720 tx=0x7f65b002b840 comp rx=0 tx=0).stop 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.739+0000 7f65c848c640 1 -- 172.21.15.16:0/680447931 shutdown_connections 2024-04-23T19:13:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.739+0000 7f65c848c640 1 --2- 172.21.15.16:0/680447931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0099810 0x7f65c01343a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.739+0000 7f65c848c640 1 -- 172.21.15.16:0/680447931 >> 172.21.15.16:0/680447931 conn(0x7f65c009f300 msgr2=0x7f65c009f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.739+0000 7f65c848c640 1 -- 172.21.15.16:0/680447931 shutdown_connections 2024-04-23T19:13:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.739+0000 7f65c848c640 1 -- 172.21.15.16:0/680447931 wait complete. 2024-04-23T19:13:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.740+0000 7f65c848c640 1 Processor -- start 2024-04-23T19:13:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.740+0000 7f65c848c640 1 -- start start 2024-04-23T19:13:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.740+0000 7f65c848c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0098d20 0x7f65c0094290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.740+0000 7f65c848c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0134c70 con 0x7f65c0098d20 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.740+0000 7f65c6228640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0098d20 0x7f65c0094290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.740+0000 7f65c6228640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0098d20 0x7f65c0094290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42152/0 (socket says 172.21.15.16:42152) 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.740+0000 7f65c6228640 1 -- 172.21.15.16:0/3401926593 learned_addr learned my addr 172.21.15.16:0/3401926593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.741+0000 7f65c6228640 1 -- 172.21.15.16:0/3401926593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c0096230 con 0x7f65c0098d20 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.741+0000 7f65c6228640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0098d20 0x7f65c0094290 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c420 tx=0x7f65bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.741+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65bc018070 con 0x7f65c0098d20 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.741+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65bc0074d0 con 0x7f65c0098d20 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.741+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65bc004950 con 0x7f65c0098d20 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.741+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c0096f60 con 0x7f65c0098d20 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.741+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c00949b0 con 0x7f65c0098d20 2024-04-23T19:13:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f65bc01b070 con 0x7f65c0098d20 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65c0095b50 con 0x7f65c0098d20 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f65a40385c0 0x7f65a403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f65bc04b450 con 0x7f65c0098d20 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f65a40449a0 0x7f65a4046e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65bc017ec0 con 0x7f65a40449a0 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65bc052050 con 0x7f65c0098d20 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65c6a29640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f65a40449a0 0x7f65a4046e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.742+0000 7f65c5a27640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f65a40385c0 0x7f65a403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.743+0000 7f65c6a29640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f65a40449a0 0x7f65a4046e40 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.743+0000 7f65c5a27640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f65a40385c0 0x7f65a403aa80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65b0036000 tx=0x7f65b0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.743+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== osd.0 v2:172.21.15.16:6800/2262294208 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65bc017ec0 con 0x7f65a40449a0 2024-04-23T19:13:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 --> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f65c0126610 con 0x7f65a40449a0 2024-04-23T19:13:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65b6ffd640 1 -- 172.21.15.16:0/3401926593 <== osd.0 v2:172.21.15.16:6800/2262294208 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f65c0126610 con 0x7f65a40449a0 2024-04-23T19:13:25.759 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-23T19:13:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T19:13:25.760 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f65a40449a0 msgr2=0x7f65a4046e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65c848c640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f65a40449a0 0x7f65a4046e40 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f65a40385c0 msgr2=0x7f65a403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65c848c640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f65a40385c0 0x7f65a403aa80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65b0036000 tx=0x7f65b0008030 comp rx=0 tx=0).stop 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0098d20 msgr2=0x7f65c0094290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.757+0000 7f65c848c640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0098d20 0x7f65c0094290 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c420 tx=0x7f65bc00c8e0 comp rx=0 tx=0).stop 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.758+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 shutdown_connections 2024-04-23T19:13:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.758+0000 7f65c848c640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:6800/2262294208,v1:172.21.15.16:6801/2262294208] conn(0x7f65a40449a0 0x7f65a4046e40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.758+0000 7f65c848c640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f65a40385c0 0x7f65a403aa80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.758+0000 7f65c848c640 1 --2- 172.21.15.16:0/3401926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c0098d20 0x7f65c0094290 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.758+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 >> 172.21.15.16:0/3401926593 conn(0x7f65c009f300 msgr2=0x7f65c009bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.758+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 shutdown_connections 2024-04-23T19:13:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.758+0000 7f65c848c640 1 -- 172.21.15.16:0/3401926593 wait complete. 2024-04-23T19:13:25.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-23T19:13:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f2101b11640 1 Processor -- start 2024-04-23T19:13:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f2101b11640 1 -- start start 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f2101b11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20fc093ec0 0x7f20fc094b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f2101b11640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20fc09d070 con 0x7f20fc0950d0 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f2101b11640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f20fc096cf0 con 0x7f20fc093ec0 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f20fb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20fc093ec0 0x7f20fc094b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f20faffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20fc0950d0 0x7f20fc133400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57940/0 (socket says 172.21.15.16:57940) 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.869+0000 7f20faffd640 1 -- 172.21.15.16:0/2852728619 learned_addr learned my addr 172.21.15.16:0/2852728619 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.870+0000 7f20fb7fe640 1 -- 172.21.15.16:0/2852728619 >> v1:172.21.15.16:6789/0 conn(0x7f20fc0950d0 legacy=0x7f20fc133400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.870+0000 7f20fb7fe640 1 -- 172.21.15.16:0/2852728619 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc096f20 con 0x7f20fc093ec0 2024-04-23T19:13:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.870+0000 7f20fa7fc640 1 -- 172.21.15.16:0/2852728619 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904531511 0 0) 0x7f20f0001110 con 0x7f20fc0950d0 2024-04-23T19:13:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.870+0000 7f20fb7fe640 1 --2- 172.21.15.16:0/2852728619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc094b90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009720 tx=0x7f20ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3838a436bc6859de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f20fa7fc640 1 -- 172.21.15.16:0/2852728619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec034070 con 0x7f20fc093ec0 2024-04-23T19:13:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f20fa7fc640 1 -- 172.21.15.16:0/2852728619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec030540 con 0x7f20fc093ec0 2024-04-23T19:13:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f20fa7fc640 1 -- 172.21.15.16:0/2852728619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec030840 con 0x7f20fc093ec0 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f2101b11640 1 -- 172.21.15.16:0/2852728619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 msgr2=0x7f20fc094b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f2101b11640 1 --2- 172.21.15.16:0/2852728619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc094b90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009720 tx=0x7f20ec02b960 comp rx=0 tx=0).stop 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f2101b11640 1 -- 172.21.15.16:0/2852728619 shutdown_connections 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f2101b11640 1 --2- 172.21.15.16:0/2852728619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc094b90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f2101b11640 1 -- 172.21.15.16:0/2852728619 >> 172.21.15.16:0/2852728619 conn(0x7f20fc09b540 msgr2=0x7f20fc09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f2101b11640 1 -- 172.21.15.16:0/2852728619 shutdown_connections 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.871+0000 7f2101b11640 1 -- 172.21.15.16:0/2852728619 wait complete. 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.872+0000 7f2101b11640 1 Processor -- start 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.872+0000 7f2101b11640 1 -- start start 2024-04-23T19:13:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.872+0000 7f2101b11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc1c0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.872+0000 7f2101b11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc095870 con 0x7f20fc093ec0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc1c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc1c0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42178/0 (socket says 172.21.15.16:42178) 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20fb7fe640 1 -- 172.21.15.16:0/2635442027 learned_addr learned my addr 172.21.15.16:0/2635442027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20fb7fe640 1 -- 172.21.15.16:0/2635442027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc1ab4e0 con 0x7f20fc093ec0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20fb7fe640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc1c0250 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009720 tx=0x7f20ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20e3fff640 1 -- 172.21.15.16:0/2635442027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec049070 con 0x7f20fc093ec0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20e3fff640 1 -- 172.21.15.16:0/2635442027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec039910 con 0x7f20fc093ec0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc1a9aa0 con 0x7f20fc093ec0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f20e3fff640 1 -- 172.21.15.16:0/2635442027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec034040 con 0x7f20fc093ec0 2024-04-23T19:13:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.873+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc1ab780 con 0x7f20fc093ec0 2024-04-23T19:13:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.874+0000 7f20e3fff640 1 -- 172.21.15.16:0/2635442027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f20ec04c070 con 0x7f20fc093ec0 2024-04-23T19:13:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.874+0000 7f20e3fff640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f20e80385c0 0x7f20e803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.874+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20fc063660 con 0x7f20fc093ec0 2024-04-23T19:13:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.874+0000 7f20e3fff640 1 -- 172.21.15.16:0/2635442027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f20ec03f030 con 0x7f20fc093ec0 2024-04-23T19:13:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.875+0000 7f20faffd640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f20e80385c0 0x7f20e803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.875+0000 7f20faffd640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f20e80385c0 0x7f20e803aa80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20f0000c00 tx=0x7f20f0001250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:25.879+0000 7f20e3fff640 1 -- 172.21.15.16:0/2635442027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20ec071070 con 0x7f20fc093ec0 2024-04-23T19:13:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.063+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f20fc067f20 con 0x7f20e80385c0 2024-04-23T19:13:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.065+0000 7f20e3fff640 1 -- 172.21.15.16:0/2635442027 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f20fc067f20 con 0x7f20e80385c0 2024-04-23T19:13:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2024-04-23T19:13:26.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f20e80385c0 msgr2=0x7f20e803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f20e80385c0 0x7f20e803aa80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20f0000c00 tx=0x7f20f0001250 comp rx=0 tx=0).stop 2024-04-23T19:13:26.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 msgr2=0x7f20fc1c0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc1c0250 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009720 tx=0x7f20ec0046c0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 shutdown_connections 2024-04-23T19:13:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f20e80385c0 0x7f20e803aa80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 --2- 172.21.15.16:0/2635442027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20fc093ec0 0x7f20fc1c0250 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 >> 172.21.15.16:0/2635442027 conn(0x7f20fc09b540 msgr2=0x7f20fc06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 shutdown_connections 2024-04-23T19:13:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.067+0000 7f2101b11640 1 -- 172.21.15.16:0/2635442027 wait complete. 2024-04-23T19:13:26.079 INFO:tasks.ceph:Scrubbing osd.1 2024-04-23T19:13:26.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-23T19:13:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.181+0000 7fed96396640 1 Processor -- start 2024-04-23T19:13:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.181+0000 7fed96396640 1 -- start start 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed96396640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed90098d20 0x7fed90099100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed96396640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed9009a690 con 0x7fed90099640 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed96396640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fed900982e0 con 0x7fed90098d20 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed8ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed90098d20 0x7fed90099100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed8f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed90099640 0x7fed90133390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57948/0 (socket says 172.21.15.16:57948) 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed8f7fe640 1 -- 172.21.15.16:0/3518160092 learned_addr learned my addr 172.21.15.16:0/3518160092 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed8ffff640 1 -- 172.21.15.16:0/3518160092 >> v1:172.21.15.16:6789/0 conn(0x7fed90099640 legacy=0x7fed90133390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.182+0000 7fed8ffff640 1 -- 172.21.15.16:0/3518160092 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed90099a90 con 0x7fed90098d20 2024-04-23T19:13:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.183+0000 7fed8ffff640 1 --2- 172.21.15.16:0/3518160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed90099100 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fed80009720 tx=0x7fed8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19a304a3d4ad867 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.183+0000 7fed8effd640 1 -- 172.21.15.16:0/3518160092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed800316d0 con 0x7fed90098d20 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.183+0000 7fed8effd640 1 -- 172.21.15.16:0/3518160092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed80037070 con 0x7fed90098d20 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.183+0000 7fed8effd640 1 -- 172.21.15.16:0/3518160092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed800319c0 con 0x7fed90098d20 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.184+0000 7fed96396640 1 -- 172.21.15.16:0/3518160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 msgr2=0x7fed90099100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.184+0000 7fed96396640 1 --2- 172.21.15.16:0/3518160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed90099100 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fed80009720 tx=0x7fed8002b840 comp rx=0 tx=0).stop 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.184+0000 7fed96396640 1 -- 172.21.15.16:0/3518160092 shutdown_connections 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.184+0000 7fed96396640 1 --2- 172.21.15.16:0/3518160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed90099100 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.184+0000 7fed96396640 1 -- 172.21.15.16:0/3518160092 >> 172.21.15.16:0/3518160092 conn(0x7fed9009f300 msgr2=0x7fed9009f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.184+0000 7fed96396640 1 -- 172.21.15.16:0/3518160092 shutdown_connections 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.184+0000 7fed96396640 1 -- 172.21.15.16:0/3518160092 wait complete. 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed96396640 1 Processor -- start 2024-04-23T19:13:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed96396640 1 -- start start 2024-04-23T19:13:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed96396640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed901c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed96396640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed90134c30 con 0x7fed90098d20 2024-04-23T19:13:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed901c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed901c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42200/0 (socket says 172.21.15.16:42200) 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed8ffff640 1 -- 172.21.15.16:0/3995266139 learned_addr learned my addr 172.21.15.16:0/3995266139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.185+0000 7fed8ffff640 1 -- 172.21.15.16:0/3995266139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed90072d90 con 0x7fed90098d20 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.186+0000 7fed8ffff640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed901c3f50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fed80034040 tx=0x7fed800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.186+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed80032640 con 0x7fed90098d20 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.186+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed800327e0 con 0x7fed90098d20 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.186+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed900963b0 con 0x7fed90098d20 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.186+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed80032ae0 con 0x7fed90098d20 2024-04-23T19:13:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.186+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed90097260 con 0x7fed90098d20 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed90063660 con 0x7fed90098d20 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fed8004c060 con 0x7fed90098d20 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed8cff9640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fed7c0385c0 0x7fed7c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fed8002f050 con 0x7fed90098d20 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed8cff9640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fed7c0449c0 0x7fed7c046e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed80039e50 con 0x7fed7c0449c0 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed80070270 con 0x7fed90098d20 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed8f7fe640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fed7c0385c0 0x7fed7c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.187+0000 7fed94933640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fed7c0449c0 0x7fed7c046e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.188+0000 7fed8f7fe640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fed7c0385c0 0x7fed7c03aa80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fed84007fd0 tx=0x7fed84007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.188+0000 7fed94933640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fed7c0449c0 0x7fed7c046e60 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.189+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== osd.1 v2:172.21.15.63:6800/3216879695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed80039e50 con 0x7fed7c0449c0 2024-04-23T19:13:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.202+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 --> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fed90067f20 con 0x7fed7c0449c0 2024-04-23T19:13:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.202+0000 7fed8cff9640 1 -- 172.21.15.16:0/3995266139 <== osd.1 v2:172.21.15.63:6800/3216879695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fed90067f20 con 0x7fed7c0449c0 2024-04-23T19:13:26.204 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-23T19:13:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T19:13:26.205 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-23T19:13:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fed7c0449c0 msgr2=0x7fed7c046e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fed7c0449c0 0x7fed7c046e60 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fed7c0385c0 msgr2=0x7fed7c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fed7c0385c0 0x7fed7c03aa80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fed84007fd0 tx=0x7fed84007ac0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 msgr2=0x7fed901c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed901c3f50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fed80034040 tx=0x7fed800046c0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 shutdown_connections 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6800/3216879695,v1:172.21.15.63:6801/3216879695] conn(0x7fed7c0449c0 0x7fed7c046e60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fed7c0385c0 0x7fed7c03aa80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 --2- 172.21.15.16:0/3995266139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90098d20 0x7fed901c3f50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 >> 172.21.15.16:0/3995266139 conn(0x7fed9009f300 msgr2=0x7fed90070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 shutdown_connections 2024-04-23T19:13:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.203+0000 7fed96396640 1 -- 172.21.15.16:0/3995266139 wait complete. 2024-04-23T19:13:26.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-23T19:13:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.313+0000 7f2b0f92b640 1 Processor -- start 2024-04-23T19:13:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.313+0000 7f2b0f92b640 1 -- start start 2024-04-23T19:13:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0f92b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b0806e580 0x7f2b08134120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0f92b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b0809f570 con 0x7f2b08072800 2024-04-23T19:13:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0f92b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2b08072ee0 con 0x7f2b0806e580 2024-04-23T19:13:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0cec6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b0806e580 0x7f2b08134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0d6c7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2b08072800 0x7f2b0806de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57958/0 (socket says 172.21.15.16:57958) 2024-04-23T19:13:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0d6c7640 1 -- 172.21.15.16:0/385191628 learned_addr learned my addr 172.21.15.16:0/385191628 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0cec6640 1 -- 172.21.15.16:0/385191628 >> v1:172.21.15.16:6789/0 conn(0x7f2b08072800 legacy=0x7f2b0806de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.314+0000 7f2b0cec6640 1 -- 172.21.15.16:0/385191628 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0806fab0 con 0x7f2b0806e580 2024-04-23T19:13:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.315+0000 7f2b0cec6640 1 --2- 172.21.15.16:0/385191628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b08134120 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009750 tx=0x7f2afc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2e089629b5459a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.315+0000 7f2b07fff640 1 -- 172.21.15.16:0/385191628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc0316d0 con 0x7f2b0806e580 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.315+0000 7f2b07fff640 1 -- 172.21.15.16:0/385191628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2afc037070 con 0x7f2b0806e580 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.315+0000 7f2b07fff640 1 -- 172.21.15.16:0/385191628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2afc0319c0 con 0x7f2b0806e580 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.316+0000 7f2b0f92b640 1 -- 172.21.15.16:0/385191628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 msgr2=0x7f2b08134120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.316+0000 7f2b0f92b640 1 --2- 172.21.15.16:0/385191628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b08134120 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009750 tx=0x7f2afc02b840 comp rx=0 tx=0).stop 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.316+0000 7f2b0f92b640 1 -- 172.21.15.16:0/385191628 shutdown_connections 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.316+0000 7f2b0f92b640 1 --2- 172.21.15.16:0/385191628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b08134120 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.316+0000 7f2b0f92b640 1 -- 172.21.15.16:0/385191628 >> 172.21.15.16:0/385191628 conn(0x7f2b08071820 msgr2=0x7f2b08071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.316+0000 7f2b0f92b640 1 -- 172.21.15.16:0/385191628 shutdown_connections 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.316+0000 7f2b0f92b640 1 -- 172.21.15.16:0/385191628 wait complete. 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0f92b640 1 Processor -- start 2024-04-23T19:13:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0f92b640 1 -- start start 2024-04-23T19:13:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0f92b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b0811fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0f92b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0806eb10 con 0x7f2b0806e580 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0d6c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b0811fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0d6c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b0811fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42216/0 (socket says 172.21.15.16:42216) 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0d6c7640 1 -- 172.21.15.16:0/3014514656 learned_addr learned my addr 172.21.15.16:0/3014514656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0d6c7640 1 -- 172.21.15.16:0/3014514656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b08122e40 con 0x7f2b0806e580 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.317+0000 7f2b0d6c7640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b0811fc00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000cb20 tx=0x7f2b0000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.318+0000 7f2b05ffb640 1 -- 172.21.15.16:0/3014514656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b00017070 con 0x7f2b0806e580 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.318+0000 7f2b05ffb640 1 -- 172.21.15.16:0/3014514656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0000abf0 con 0x7f2b0806e580 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.318+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b08122210 con 0x7f2b0806e580 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.318+0000 7f2b05ffb640 1 -- 172.21.15.16:0/3014514656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b00012400 con 0x7f2b0806e580 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.318+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b08120140 con 0x7f2b0806e580 2024-04-23T19:13:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.318+0000 7f2b05ffb640 1 -- 172.21.15.16:0/3014514656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f2b0001a070 con 0x7f2b0806e580 2024-04-23T19:13:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.319+0000 7f2b05ffb640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2af40385c0 0x7f2af403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.319+0000 7f2b05ffb640 1 -- 172.21.15.16:0/3014514656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f2b0004be60 con 0x7f2b0806e580 2024-04-23T19:13:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.319+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b081215a0 con 0x7f2b0806e580 2024-04-23T19:13:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.319+0000 7f2b0cec6640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2af40385c0 0x7f2af403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.320+0000 7f2b0cec6640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2af40385c0 0x7f2af403aa80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2afc036000 tx=0x7f2afc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.323+0000 7f2b05ffb640 1 -- 172.21.15.16:0/3014514656 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b00012df0 con 0x7f2b0806e580 2024-04-23T19:13:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.511+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2b08122590 con 0x7f2af40385c0 2024-04-23T19:13:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.511+0000 7f2b05ffb640 1 -- 172.21.15.16:0/3014514656 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2b08122590 con 0x7f2af40385c0 2024-04-23T19:13:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2024-04-23T19:13:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2af40385c0 msgr2=0x7f2af403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2af40385c0 0x7f2af403aa80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2afc036000 tx=0x7f2afc008030 comp rx=0 tx=0).stop 2024-04-23T19:13:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 msgr2=0x7f2b0811fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b0811fc00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000cb20 tx=0x7f2b0000a060 comp rx=0 tx=0).stop 2024-04-23T19:13:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 shutdown_connections 2024-04-23T19:13:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f2af40385c0 0x7f2af403aa80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 --2- 172.21.15.16:0/3014514656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b0806e580 0x7f2b0811fc00 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 >> 172.21.15.16:0/3014514656 conn(0x7f2b08071820 msgr2=0x7f2b081b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 shutdown_connections 2024-04-23T19:13:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.514+0000 7f2b0f92b640 1 -- 172.21.15.16:0/3014514656 wait complete. 2024-04-23T19:13:26.526 INFO:tasks.ceph:Scrubbing osd.2 2024-04-23T19:13:26.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-23T19:13:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.628+0000 7f1ef423f640 1 Processor -- start 2024-04-23T19:13:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.628+0000 7f1ef423f640 1 -- start start 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.628+0000 7f1ef423f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1eec12ea80 0x7f1eec130e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.628+0000 7f1ef423f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eec12b820 con 0x7f1eec12bc50 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.628+0000 7f1ef423f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1eec096b30 con 0x7f1eec12ea80 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.628+0000 7f1ef17da640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1eec12ea80 0x7f1eec130e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.629+0000 7f1ef1fdb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1eec12bc50 0x7f1eec12c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57962/0 (socket says 172.21.15.16:57962) 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.629+0000 7f1ef1fdb640 1 -- 172.21.15.16:0/3548048637 learned_addr learned my addr 172.21.15.16:0/3548048637 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.629+0000 7f1ef0fd9640 1 -- 172.21.15.16:0/3548048637 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697449753 0 0) 0x7f1eec12b820 con 0x7f1eec12bc50 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.629+0000 7f1ef0fd9640 1 -- 172.21.15.16:0/3548048637 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ed8003520 con 0x7f1eec12bc50 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.629+0000 7f1ef17da640 1 -- 172.21.15.16:0/3548048637 >> v1:172.21.15.16:6789/0 conn(0x7f1eec12bc50 legacy=0x7f1eec12c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.629+0000 7f1ef17da640 1 -- 172.21.15.16:0/3548048637 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec0991a0 con 0x7f1eec12ea80 2024-04-23T19:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef17da640 1 --2- 172.21.15.16:0/3548048637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12ea80 0x7f1eec130e70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009720 tx=0x7f1edc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa138dc32b5af65 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef0fd9640 1 -- 172.21.15.16:0/3548048637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1edc0316d0 con 0x7f1eec12ea80 2024-04-23T19:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef0fd9640 1 -- 172.21.15.16:0/3548048637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1edc037070 con 0x7f1eec12ea80 2024-04-23T19:13:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef0fd9640 1 -- 172.21.15.16:0/3548048637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1edc0319a0 con 0x7f1eec12ea80 2024-04-23T19:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef423f640 1 -- 172.21.15.16:0/3548048637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12ea80 msgr2=0x7f1eec130e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef423f640 1 --2- 172.21.15.16:0/3548048637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12ea80 0x7f1eec130e70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009720 tx=0x7f1edc02b840 comp rx=0 tx=0).stop 2024-04-23T19:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef423f640 1 -- 172.21.15.16:0/3548048637 shutdown_connections 2024-04-23T19:13:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef423f640 1 --2- 172.21.15.16:0/3548048637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12ea80 0x7f1eec130e70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef423f640 1 -- 172.21.15.16:0/3548048637 >> 172.21.15.16:0/3548048637 conn(0x7f1eec099f40 msgr2=0x7f1eec097810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef423f640 1 -- 172.21.15.16:0/3548048637 shutdown_connections 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.630+0000 7f1ef423f640 1 -- 172.21.15.16:0/3548048637 wait complete. 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.631+0000 7f1ef423f640 1 Processor -- start 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.631+0000 7f1ef423f640 1 -- start start 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef423f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12bc50 0x7f1eec123d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef423f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec06d500 con 0x7f1eec12bc50 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef1fdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12bc50 0x7f1eec123d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef1fdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12bc50 0x7f1eec123d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42234/0 (socket says 172.21.15.16:42234) 2024-04-23T19:13:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef1fdb640 1 -- 172.21.15.16:0/2126871162 learned_addr learned my addr 172.21.15.16:0/2126871162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef1fdb640 1 -- 172.21.15.16:0/2126871162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec06e4c0 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef1fdb640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12bc50 0x7f1eec123d70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c3b0 tx=0x7f1ee800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee8017070 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee8005c10 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec125700 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee80070d0 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.632+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec06eae0 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.633+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1ee801a070 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.633+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1eec06ed70 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.633+0000 7f1ee27fc640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1ed00385c0 0x7f1ed003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.633+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f1ee804b560 con 0x7f1eec12bc50 2024-04-23T19:13:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.633+0000 7f1ee27fc640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1ed00449a0 0x7f1ed0046e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.633+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ee8009e50 con 0x7f1ed00449a0 2024-04-23T19:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.633+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ee8052050 con 0x7f1eec12bc50 2024-04-23T19:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.634+0000 7f1ef17da640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1ed00385c0 0x7f1ed003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.634+0000 7f1ef27dc640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1ed00449a0 0x7f1ed0046e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.634+0000 7f1ef27dc640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1ed00449a0 0x7f1ed0046e40 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.634+0000 7f1ef17da640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1ed00385c0 0x7f1ed003aa80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1edc036000 tx=0x7f1edc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.635+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== osd.2 v2:172.21.15.63:6808/753864785 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ee8009e50 con 0x7f1ed00449a0 2024-04-23T19:13:26.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.648+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 --> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1eec063660 con 0x7f1ed00449a0 2024-04-23T19:13:26.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.648+0000 7f1ee27fc640 1 -- 172.21.15.16:0/2126871162 <== osd.2 v2:172.21.15.63:6808/753864785 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1eec063660 con 0x7f1ed00449a0 2024-04-23T19:13:26.650 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-23T19:13:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T19:13:26.651 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.648+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1ed00449a0 msgr2=0x7f1ed0046e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.648+0000 7f1ef423f640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1ed00449a0 0x7f1ed0046e40 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1ed00385c0 msgr2=0x7f1ed003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1ed00385c0 0x7f1ed003aa80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1edc036000 tx=0x7f1edc008030 comp rx=0 tx=0).stop 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12bc50 msgr2=0x7f1eec123d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12bc50 0x7f1eec123d70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c3b0 tx=0x7f1ee800c870 comp rx=0 tx=0).stop 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 shutdown_connections 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6808/753864785,v1:172.21.15.63:6809/753864785] conn(0x7f1ed00449a0 0x7f1ed0046e40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f1ed00385c0 0x7f1ed003aa80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 --2- 172.21.15.16:0/2126871162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec12bc50 0x7f1eec123d70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 >> 172.21.15.16:0/2126871162 conn(0x7f1eec099f40 msgr2=0x7f1eec129510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 shutdown_connections 2024-04-23T19:13:26.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.649+0000 7f1ef423f640 1 -- 172.21.15.16:0/2126871162 wait complete. 2024-04-23T19:13:26.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f132aa25640 1 Processor -- start 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f132aa25640 1 -- start start 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f132aa25640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13241288d0 0x7f1324128cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f132aa25640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1324128410 con 0x7f1324121e40 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f132aa25640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f13241286b0 con 0x7f13241288d0 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f1323fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13241288d0 0x7f1324128cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f13237fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1324121e40 0x7f1324122280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57966/0 (socket says 172.21.15.16:57966) 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.760+0000 7f13237fe640 1 -- 172.21.15.16:0/501050104 learned_addr learned my addr 172.21.15.16:0/501050104 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.761+0000 7f1323fff640 1 -- 172.21.15.16:0/501050104 >> v1:172.21.15.16:6789/0 conn(0x7f1324121e40 legacy=0x7f1324122280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.761+0000 7f1322ffd640 1 -- 172.21.15.16:0/501050104 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962130269 0 0) 0x7f1324128410 con 0x7f1324121e40 2024-04-23T19:13:26.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.761+0000 7f1323fff640 1 -- 172.21.15.16:0/501050104 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132406de40 con 0x7f13241288d0 2024-04-23T19:13:26.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.761+0000 7f1323fff640 1 --2- 172.21.15.16:0/501050104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13241288d0 0x7f1324128cb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1314009720 tx=0x7f131402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1947a6c3f90cdc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.761+0000 7f1322ffd640 1 -- 172.21.15.16:0/501050104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13140316d0 con 0x7f13241288d0 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f1322ffd640 1 -- 172.21.15.16:0/501050104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1314037070 con 0x7f13241288d0 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f1322ffd640 1 -- 172.21.15.16:0/501050104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13140319c0 con 0x7f13241288d0 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f132aa25640 1 -- 172.21.15.16:0/501050104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13241288d0 msgr2=0x7f1324128cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f132aa25640 1 --2- 172.21.15.16:0/501050104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13241288d0 0x7f1324128cb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1314009720 tx=0x7f131402b840 comp rx=0 tx=0).stop 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f132aa25640 1 -- 172.21.15.16:0/501050104 shutdown_connections 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f132aa25640 1 --2- 172.21.15.16:0/501050104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13241288d0 0x7f1324128cb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f132aa25640 1 -- 172.21.15.16:0/501050104 >> 172.21.15.16:0/501050104 conn(0x7f132409a200 msgr2=0x7f1324097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f132aa25640 1 -- 172.21.15.16:0/501050104 shutdown_connections 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.762+0000 7f132aa25640 1 -- 172.21.15.16:0/501050104 wait complete. 2024-04-23T19:13:26.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.763+0000 7f132aa25640 1 Processor -- start 2024-04-23T19:13:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.763+0000 7f132aa25640 1 -- start start 2024-04-23T19:13:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.763+0000 7f132aa25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324121e40 0x7f1324124380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.763+0000 7f132aa25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13240945f0 con 0x7f1324121e40 2024-04-23T19:13:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.763+0000 7f1323fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324121e40 0x7f1324124380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f1323fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324121e40 0x7f1324124380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42262/0 (socket says 172.21.15.16:42262) 2024-04-23T19:13:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f1323fff640 1 -- 172.21.15.16:0/3249506948 learned_addr learned my addr 172.21.15.16:0/3249506948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f1323fff640 1 -- 172.21.15.16:0/3249506948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1324127370 con 0x7f1324121e40 2024-04-23T19:13:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f1323fff640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324121e40 0x7f1324124380 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1314009720 tx=0x7f13140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f1320ff9640 1 -- 172.21.15.16:0/3249506948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1314004160 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f1320ff9640 1 -- 172.21.15.16:0/3249506948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1314004320 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1324127060 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f1320ff9640 1 -- 172.21.15.16:0/3249506948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1314032640 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.764+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1324128040 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.765+0000 7f1320ff9640 1 -- 172.21.15.16:0/3249506948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f13140327e0 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.765+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1324063660 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.765+0000 7f1320ff9640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f13100385c0 0x7f131003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.765+0000 7f1320ff9640 1 -- 172.21.15.16:0/3249506948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f131402f050 con 0x7f1324121e40 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.766+0000 7f13237fe640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f13100385c0 0x7f131003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:26.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.766+0000 7f13237fe640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f13100385c0 0x7f131003aa80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318007910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:26.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.770+0000 7f1320ff9640 1 -- 172.21.15.16:0/3249506948 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1314041070 con 0x7f1324121e40 2024-04-23T19:13:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.956+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f132409c270 con 0x7f13100385c0 2024-04-23T19:13:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.957+0000 7f1320ff9640 1 -- 172.21.15.16:0/3249506948 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f13180070d0 con 0x7f13100385c0 2024-04-23T19:13:26.959 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2024-04-23T19:13:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f13100385c0 msgr2=0x7f131003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f13100385c0 0x7f131003aa80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318007910 comp rx=0 tx=0).stop 2024-04-23T19:13:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324121e40 msgr2=0x7f1324124380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324121e40 0x7f1324124380 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1314009720 tx=0x7f13140046c0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 shutdown_connections 2024-04-23T19:13:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f13100385c0 0x7f131003aa80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 --2- 172.21.15.16:0/3249506948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324121e40 0x7f1324124380 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 >> 172.21.15.16:0/3249506948 conn(0x7f132409a200 msgr2=0x7f1324097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 shutdown_connections 2024-04-23T19:13:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:26.959+0000 7f132aa25640 1 -- 172.21.15.16:0/3249506948 wait complete. 2024-04-23T19:13:26.971 INFO:tasks.ceph:Scrubbing osd.3 2024-04-23T19:13:26.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-23T19:13:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.073+0000 7fb45e332640 1 Processor -- start 2024-04-23T19:13:27.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb45e332640 1 -- start start 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb45e332640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb458129e00 0x7fb458131e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb45e332640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb458096700 con 0x7fb458129310 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb45e332640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb458096b40 con 0x7fb458129e00 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb4577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb458129e00 0x7fb458131e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb457fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb458129310 0x7fb4581296f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57982/0 (socket says 172.21.15.16:57982) 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb457fff640 1 -- 172.21.15.16:0/3008738848 learned_addr learned my addr 172.21.15.16:0/3008738848 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb4577fe640 1 -- 172.21.15.16:0/3008738848 >> v1:172.21.15.16:6789/0 conn(0x7fb458129310 legacy=0x7fb4581296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.074+0000 7fb4577fe640 1 -- 172.21.15.16:0/3008738848 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4580991a0 con 0x7fb458129e00 2024-04-23T19:13:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.075+0000 7fb4577fe640 1 --2- 172.21.15.16:0/3008738848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129e00 0x7fb458131e10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009720 tx=0x7fb44c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5cf58cce87ea98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.075+0000 7fb456ffd640 1 -- 172.21.15.16:0/3008738848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb44c0316d0 con 0x7fb458129e00 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.075+0000 7fb456ffd640 1 -- 172.21.15.16:0/3008738848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44c037070 con 0x7fb458129e00 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.075+0000 7fb456ffd640 1 -- 172.21.15.16:0/3008738848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb44c0319c0 con 0x7fb458129e00 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.076+0000 7fb45e332640 1 -- 172.21.15.16:0/3008738848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129e00 msgr2=0x7fb458131e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.076+0000 7fb45e332640 1 --2- 172.21.15.16:0/3008738848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129e00 0x7fb458131e10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009720 tx=0x7fb44c02b840 comp rx=0 tx=0).stop 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.076+0000 7fb45e332640 1 -- 172.21.15.16:0/3008738848 shutdown_connections 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.076+0000 7fb45e332640 1 --2- 172.21.15.16:0/3008738848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129e00 0x7fb458131e10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.076+0000 7fb45e332640 1 -- 172.21.15.16:0/3008738848 >> 172.21.15.16:0/3008738848 conn(0x7fb458099f40 msgr2=0x7fb458097810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.076+0000 7fb45e332640 1 -- 172.21.15.16:0/3008738848 shutdown_connections 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.076+0000 7fb45e332640 1 -- 172.21.15.16:0/3008738848 wait complete. 2024-04-23T19:13:27.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.077+0000 7fb45e332640 1 Processor -- start 2024-04-23T19:13:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.077+0000 7fb45e332640 1 -- start start 2024-04-23T19:13:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.077+0000 7fb45e332640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129310 0x7fb4580702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.077+0000 7fb45e332640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb458096210 con 0x7fb458129310 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.077+0000 7fb457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129310 0x7fb4580702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.077+0000 7fb457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129310 0x7fb4580702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42270/0 (socket says 172.21.15.16:42270) 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.077+0000 7fb457fff640 1 -- 172.21.15.16:0/608152146 learned_addr learned my addr 172.21.15.16:0/608152146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.078+0000 7fb457fff640 1 -- 172.21.15.16:0/608152146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb458122120 con 0x7fb458129310 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.078+0000 7fb457fff640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129310 0x7fb4580702d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c380 tx=0x7fb44800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.078+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb448005c10 con 0x7fb458129310 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.078+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb448005db0 con 0x7fb458129310 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.078+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb458073050 con 0x7fb458129310 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.078+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb448004930 con 0x7fb458129310 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.078+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4580709f0 con 0x7fb458129310 2024-04-23T19:13:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4580720e0 con 0x7fb458129310 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb44801a020 con 0x7fb458129310 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb454ff9640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4440385c0 0x7fb44403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fb44804b8d0 con 0x7fb458129310 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb454ff9640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fb4440449c0 0x7fb444046e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb444047530 con 0x7fb4440449c0 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4480502a0 con 0x7fb458129310 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.079+0000 7fb4577fe640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4440385c0 0x7fb44403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.080+0000 7fb45c8cf640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fb4440449c0 0x7fb444046e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.080+0000 7fb4577fe640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4440385c0 0x7fb44403aa80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009720 tx=0x7fb44c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.080+0000 7fb45c8cf640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fb4440449c0 0x7fb444046e60 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.081+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== osd.3 v2:172.21.15.63:6816/1289558434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb444047530 con 0x7fb4440449c0 2024-04-23T19:13:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.094+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 --> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb458063660 con 0x7fb4440449c0 2024-04-23T19:13:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.094+0000 7fb454ff9640 1 -- 172.21.15.16:0/608152146 <== osd.3 v2:172.21.15.63:6816/1289558434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb458063660 con 0x7fb4440449c0 2024-04-23T19:13:27.096 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-23T19:13:27.097 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T19:13:27.097 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fb4440449c0 msgr2=0x7fb444046e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fb4440449c0 0x7fb444046e60 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4440385c0 msgr2=0x7fb44403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4440385c0 0x7fb44403aa80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009720 tx=0x7fb44c008030 comp rx=0 tx=0).stop 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129310 msgr2=0x7fb4580702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129310 0x7fb4580702d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c380 tx=0x7fb44800c840 comp rx=0 tx=0).stop 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 shutdown_connections 2024-04-23T19:13:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6816/1289558434,v1:172.21.15.63:6817/1289558434] conn(0x7fb4440449c0 0x7fb444046e60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb4440385c0 0x7fb44403aa80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 --2- 172.21.15.16:0/608152146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb458129310 0x7fb4580702d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 >> 172.21.15.16:0/608152146 conn(0x7fb458099f40 msgr2=0x7fb45812a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 shutdown_connections 2024-04-23T19:13:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.095+0000 7fb45e332640 1 -- 172.21.15.16:0/608152146 wait complete. 2024-04-23T19:13:27.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-23T19:13:27.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.205+0000 7f280199f640 1 Processor -- start 2024-04-23T19:13:27.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.205+0000 7f280199f640 1 -- start start 2024-04-23T19:13:27.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f280199f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27fc06e0e0 0x7f27fc06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f280199f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27fc06fdd0 con 0x7f27fc06ea00 2024-04-23T19:13:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f280199f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f27fc09e7f0 con 0x7f27fc06e0e0 2024-04-23T19:13:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f280099d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27fc06e0e0 0x7f27fc06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f27fbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f27fc06ea00 0x7f27fc09a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57990/0 (socket says 172.21.15.16:57990) 2024-04-23T19:13:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f27fbfff640 1 -- 172.21.15.16:0/2537028042 learned_addr learned my addr 172.21.15.16:0/2537028042 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f280099d640 1 -- 172.21.15.16:0/2537028042 >> v1:172.21.15.16:6789/0 conn(0x7f27fc06ea00 legacy=0x7f27fc09a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.206+0000 7f280099d640 1 -- 172.21.15.16:0/2537028042 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc06ee30 con 0x7f27fc06e0e0 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.207+0000 7f280099d640 1 --2- 172.21.15.16:0/2537028042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc06e4c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009d90 tx=0x7f27f402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b92fd5e9025ecef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.207+0000 7f27fb7fe640 1 -- 172.21.15.16:0/2537028042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4039070 con 0x7f27fc06e0e0 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.207+0000 7f27fb7fe640 1 -- 172.21.15.16:0/2537028042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f402cb10 con 0x7f27fc06e0e0 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.207+0000 7f27fb7fe640 1 -- 172.21.15.16:0/2537028042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f402ce10 con 0x7f27fc06e0e0 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.207+0000 7f280199f640 1 -- 172.21.15.16:0/2537028042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 msgr2=0x7f27fc06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.207+0000 7f280199f640 1 --2- 172.21.15.16:0/2537028042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc06e4c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009d90 tx=0x7f27f402beb0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.208+0000 7f280199f640 1 -- 172.21.15.16:0/2537028042 shutdown_connections 2024-04-23T19:13:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.208+0000 7f280199f640 1 --2- 172.21.15.16:0/2537028042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc06e4c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.208+0000 7f280199f640 1 -- 172.21.15.16:0/2537028042 >> 172.21.15.16:0/2537028042 conn(0x7f27fc0a1540 msgr2=0x7f27fc09da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.208+0000 7f280199f640 1 -- 172.21.15.16:0/2537028042 shutdown_connections 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.208+0000 7f280199f640 1 -- 172.21.15.16:0/2537028042 wait complete. 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.208+0000 7f280199f640 1 Processor -- start 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.208+0000 7f280199f640 1 -- start start 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.209+0000 7f280199f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc127750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.209+0000 7f280199f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc09af40 con 0x7f27fc06e0e0 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.209+0000 7f280099d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc127750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.209+0000 7f280099d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc127750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42288/0 (socket says 172.21.15.16:42288) 2024-04-23T19:13:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.209+0000 7f280099d640 1 -- 172.21.15.16:0/3922224970 learned_addr learned my addr 172.21.15.16:0/3922224970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:27.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.209+0000 7f280099d640 1 -- 172.21.15.16:0/3922224970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc08f3f0 con 0x7f27fc06e0e0 2024-04-23T19:13:27.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.209+0000 7f280099d640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc127750 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f27f4037040 tx=0x7f27f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.210+0000 7f27f97fa640 1 -- 172.21.15.16:0/3922224970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4044020 con 0x7f27fc06e0e0 2024-04-23T19:13:27.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.210+0000 7f27f97fa640 1 -- 172.21.15.16:0/3922224970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f403f5a0 con 0x7f27fc06e0e0 2024-04-23T19:13:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.210+0000 7f27f97fa640 1 -- 172.21.15.16:0/3922224970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4039040 con 0x7f27fc06e0e0 2024-04-23T19:13:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.210+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc08f870 con 0x7f27fc06e0e0 2024-04-23T19:13:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.210+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc127c90 con 0x7f27fc06e0e0 2024-04-23T19:13:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.211+0000 7f27f97fa640 1 -- 172.21.15.16:0/3922224970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f27f4047020 con 0x7f27fc06e0e0 2024-04-23T19:13:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.211+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc063660 con 0x7f27fc06e0e0 2024-04-23T19:13:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.211+0000 7f27f97fa640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f27e80385c0 0x7f27e803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.211+0000 7f27f97fa640 1 -- 172.21.15.16:0/3922224970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f27f406eef0 con 0x7f27fc06e0e0 2024-04-23T19:13:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.211+0000 7f27fbfff640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f27e80385c0 0x7f27e803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.212+0000 7f27fbfff640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f27e80385c0 0x7f27e803aa80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f27f00015c0 tx=0x7f27f0007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.215+0000 7f27f97fa640 1 -- 172.21.15.16:0/3922224970 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27f404e030 con 0x7f27fc06e0e0 2024-04-23T19:13:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.397+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f27fc09a780 con 0x7f27e80385c0 2024-04-23T19:13:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.398+0000 7f27f97fa640 1 -- 172.21.15.16:0/3922224970 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f27fc09a780 con 0x7f27e80385c0 2024-04-23T19:13:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2024-04-23T19:13:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.399+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f27e80385c0 msgr2=0x7f27e803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.399+0000 7f280199f640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f27e80385c0 0x7f27e803aa80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f27f00015c0 tx=0x7f27f0007a80 comp rx=0 tx=0).stop 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 msgr2=0x7f27fc127750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc127750 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f27f4037040 tx=0x7f27f4004a40 comp rx=0 tx=0).stop 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 shutdown_connections 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f27e80385c0 0x7f27e803aa80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 --2- 172.21.15.16:0/3922224970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc06e0e0 0x7f27fc127750 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 >> 172.21.15.16:0/3922224970 conn(0x7f27fc0a1540 msgr2=0x7f27fc12dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 shutdown_connections 2024-04-23T19:13:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.400+0000 7f280199f640 1 -- 172.21.15.16:0/3922224970 wait complete. 2024-04-23T19:13:27.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:13:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.511+0000 7fc1370d2640 1 Processor -- start 2024-04-23T19:13:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.511+0000 7fc1370d2640 1 -- start start 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc1370d2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc13012c290 0x7fc130133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc1370d2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13009aa00 con 0x7fc13012b7a0 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc1370d2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc1300a20d0 con 0x7fc13012c290 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc127fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc13012c290 0x7fc130133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc134e6e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc13012b7a0 0x7fc13012bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57994/0 (socket says 172.21.15.16:57994) 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc134e6e640 1 -- 172.21.15.16:0/3169403547 learned_addr learned my addr 172.21.15.16:0/3169403547 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc127fff640 1 -- 172.21.15.16:0/3169403547 >> v1:172.21.15.16:6789/0 conn(0x7fc13012b7a0 legacy=0x7fc13012bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.512+0000 7fc127fff640 1 -- 172.21.15.16:0/3169403547 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13009af50 con 0x7fc13012c290 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.513+0000 7fc127fff640 1 --2- 172.21.15.16:0/3169403547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012c290 0x7fc130133fc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc120009720 tx=0x7fc12002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba9f6dd7875128e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.513+0000 7fc1277fe640 1 -- 172.21.15.16:0/3169403547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1200316d0 con 0x7fc13012c290 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.513+0000 7fc1277fe640 1 -- 172.21.15.16:0/3169403547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc120037070 con 0x7fc13012c290 2024-04-23T19:13:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.513+0000 7fc1277fe640 1 -- 172.21.15.16:0/3169403547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1200319c0 con 0x7fc13012c290 2024-04-23T19:13:27.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 -- 172.21.15.16:0/3169403547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012c290 msgr2=0x7fc130133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 --2- 172.21.15.16:0/3169403547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012c290 0x7fc130133fc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc120009720 tx=0x7fc12002b840 comp rx=0 tx=0).stop 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 -- 172.21.15.16:0/3169403547 shutdown_connections 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 --2- 172.21.15.16:0/3169403547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012c290 0x7fc130133fc0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 -- 172.21.15.16:0/3169403547 >> 172.21.15.16:0/3169403547 conn(0x7fc1300a15b0 msgr2=0x7fc1300a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 -- 172.21.15.16:0/3169403547 shutdown_connections 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 -- 172.21.15.16:0/3169403547 wait complete. 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 Processor -- start 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.514+0000 7fc1370d2640 1 -- start start 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.515+0000 7fc1370d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012b7a0 0x7fc1301c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.515+0000 7fc1370d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc130134500 con 0x7fc13012b7a0 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.515+0000 7fc134e6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012b7a0 0x7fc1301c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.515+0000 7fc134e6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012b7a0 0x7fc1301c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42308/0 (socket says 172.21.15.16:42308) 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.515+0000 7fc134e6e640 1 -- 172.21.15.16:0/1639355107 learned_addr learned my addr 172.21.15.16:0/1639355107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.515+0000 7fc134e6e640 1 -- 172.21.15.16:0/1639355107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13006da40 con 0x7fc13012b7a0 2024-04-23T19:13:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.515+0000 7fc134e6e640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012b7a0 0x7fc1301c3950 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00c4a0 tx=0x7fc12c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.516+0000 7fc1257fa640 1 -- 172.21.15.16:0/1639355107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc12c017070 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.516+0000 7fc1257fa640 1 -- 172.21.15.16:0/1639355107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12c005c10 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.516+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc130133690 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.516+0000 7fc1257fa640 1 -- 172.21.15.16:0/1639355107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc12c007070 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.516+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13006e8b0 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.516+0000 7fc1257fa640 1 -- 172.21.15.16:0/1639355107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fc12c01a070 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.517+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13006e6c0 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.517+0000 7fc1257fa640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc11c0385c0 0x7fc11c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.517+0000 7fc1257fa640 1 -- 172.21.15.16:0/1639355107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fc12c04b390 con 0x7fc13012b7a0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.517+0000 7fc127fff640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc11c0385c0 0x7fc11c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.517+0000 7fc127fff640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc11c0385c0 0x7fc11c03aa80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc120036000 tx=0x7fc120008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.521+0000 7fc1257fa640 1 -- 172.21.15.16:0/1639355107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc12c007210 con 0x7fc13012b7a0 2024-04-23T19:13:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.701+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc130063660 con 0x7fc11c0385c0 2024-04-23T19:13:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.717+0000 7fc1257fa640 1 -- 172.21.15.16:0/1639355107 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665601 (secure 0 0 0) 0x7fc130063660 con 0x7fc11c0385c0 2024-04-23T19:13:27.727 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:13:27.733 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:13:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc11c0385c0 msgr2=0x7fc11c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc11c0385c0 0x7fc11c03aa80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc120036000 tx=0x7fc120008030 comp rx=0 tx=0).stop 2024-04-23T19:13:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012b7a0 msgr2=0x7fc1301c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012b7a0 0x7fc1301c3950 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00c4a0 tx=0x7fc12c00c960 comp rx=0 tx=0).stop 2024-04-23T19:13:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 shutdown_connections 2024-04-23T19:13:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fc11c0385c0 0x7fc11c03aa80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 --2- 172.21.15.16:0/1639355107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13012b7a0 0x7fc1301c3950 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 >> 172.21.15.16:0/1639355107 conn(0x7fc1300a15b0 msgr2=0x7fc13012a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 shutdown_connections 2024-04-23T19:13:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:27.722+0000 7fc1370d2640 1 -- 172.21.15.16:0/1639355107 wait complete. 2024-04-23T19:13:27.736 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-04-23T19:13:26.702027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9633863,"num_objects":223,"num_object_clones":0,"num_object_copies":449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":4971,"num_read_kb":4555,"num_write":2138,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4538,"ondisk_log_size":4538,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":159096,"kb_used_data":21908,"kb_used_omap":6,"kb_used_meta":137145,"kb_avail":377328264,"statfs":{"total":386547056640,"available":386384142336,"internally_reserved":0,"allocated":22433792,"data_stored":21279280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":140437271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9032761,"num_objects":5,"num_object_clones":0,"num_object_copies":13,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":592,"num_read_kb":554,"num_write":418,"num_write_kb":29372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003695"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722031+0000","last_change":"2024-04-23T19:10:07.492726+0000","last_active":"2024-04-23T19:10:47.722031+0000","last_peered":"2024-04-23T19:10:47.722031+0000","last_clean":"2024-04-23T19:10:47.722031+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:10:47.722031+0000","last_undegraded":"2024-04-23T19:10:47.722031+0000","last_fullsized":"2024-04-23T19:10:47.722031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771923+0000","last_change":"2024-04-23T19:10:09.485943+0000","last_active":"2024-04-23T19:12:15.771923+0000","last_peered":"2024-04-23T19:12:15.771923+0000","last_clean":"2024-04-23T19:12:15.771923+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:12:15.771923+0000","last_undegraded":"2024-04-23T19:12:15.771923+0000","last_fullsized":"2024-04-23T19:12:15.771923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729774+0000","last_change":"2024-04-23T19:10:07.490929+0000","last_active":"2024-04-23T19:10:47.729774+0000","last_peered":"2024-04-23T19:10:47.729774+0000","last_clean":"2024-04-23T19:10:47.729774+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:10:47.729774+0000","last_undegraded":"2024-04-23T19:10:47.729774+0000","last_fullsized":"2024-04-23T19:10:47.729774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:43.016228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.498665+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:54.763728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729513+0000","last_change":"2024-04-23T19:10:07.508341+0000","last_active":"2024-04-23T19:10:47.729513+0000","last_peered":"2024-04-23T19:10:47.729513+0000","last_clean":"2024-04-23T19:10:47.729513+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:10:47.729513+0000","last_undegraded":"2024-04-23T19:10:47.729513+0000","last_fullsized":"2024-04-23T19:10:47.729513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:50.865140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771899+0000","last_change":"2024-04-23T19:10:09.498971+0000","last_active":"2024-04-23T19:12:15.771899+0000","last_peered":"2024-04-23T19:12:15.771899+0000","last_clean":"2024-04-23T19:12:15.771899+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:12:15.771899+0000","last_undegraded":"2024-04-23T19:12:15.771899+0000","last_fullsized":"2024-04-23T19:12:15.771899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:58:56.677184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719199+0000","last_change":"2024-04-23T19:10:07.490859+0000","last_active":"2024-04-23T19:10:47.719199+0000","last_peered":"2024-04-23T19:10:47.719199+0000","last_clean":"2024-04-23T19:10:47.719199+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:10:47.719199+0000","last_undegraded":"2024-04-23T19:10:47.719199+0000","last_fullsized":"2024-04-23T19:10:47.719199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447391+0000","last_change":"2024-04-23T19:10:09.489556+0000","last_active":"2024-04-23T19:13:16.447391+0000","last_peered":"2024-04-23T19:13:16.447391+0000","last_clean":"2024-04-23T19:13:16.447391+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:13:16.447391+0000","last_undegraded":"2024-04-23T19:13:16.447391+0000","last_fullsized":"2024-04-23T19:13:16.447391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:55.937097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.675532+0000","last_change":"2024-04-23T19:10:07.504507+0000","last_active":"2024-04-23T19:12:15.675532+0000","last_peered":"2024-04-23T19:12:15.675532+0000","last_clean":"2024-04-23T19:12:15.675532+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:12:15.675532+0000","last_undegraded":"2024-04-23T19:12:15.675532+0000","last_fullsized":"2024-04-23T19:12:15.675532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:05.672303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.764033+0000","last_change":"2024-04-23T19:10:09.482365+0000","last_active":"2024-04-23T19:12:15.764033+0000","last_peered":"2024-04-23T19:12:15.764033+0000","last_clean":"2024-04-23T19:12:15.764033+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:12:15.764033+0000","last_undegraded":"2024-04-23T19:12:15.764033+0000","last_fullsized":"2024-04-23T19:12:15.764033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.042125+0000","last_change":"2024-04-23T19:10:07.490892+0000","last_active":"2024-04-23T19:13:16.042125+0000","last_peered":"2024-04-23T19:13:16.042125+0000","last_clean":"2024-04-23T19:13:16.042125+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:13:16.042125+0000","last_undegraded":"2024-04-23T19:13:16.042125+0000","last_fullsized":"2024-04-23T19:13:16.042125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258616+0000","last_change":"2024-04-23T19:10:09.499345+0000","last_active":"2024-04-23T19:13:16.258616+0000","last_peered":"2024-04-23T19:13:16.258616+0000","last_clean":"2024-04-23T19:13:16.258616+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:16.258616+0000","last_undegraded":"2024-04-23T19:13:16.258616+0000","last_fullsized":"2024-04-23T19:13:16.258616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:28.869634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724671+0000","last_change":"2024-04-23T19:10:07.489309+0000","last_active":"2024-04-23T19:10:47.724671+0000","last_peered":"2024-04-23T19:10:47.724671+0000","last_clean":"2024-04-23T19:10:47.724671+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:10:47.724671+0000","last_undegraded":"2024-04-23T19:10:47.724671+0000","last_fullsized":"2024-04-23T19:10:47.724671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:18.611845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'23","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458768+0000","last_change":"2024-04-23T19:10:09.492130+0000","last_active":"2024-04-23T19:13:16.458768+0000","last_peered":"2024-04-23T19:13:16.458768+0000","last_clean":"2024-04-23T19:13:16.458768+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:16.458768+0000","last_undegraded":"2024-04-23T19:13:16.458768+0000","last_fullsized":"2024-04-23T19:13:16.458768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.100026+0000","last_change":"2024-04-23T19:10:07.487170+0000","last_active":"2024-04-23T19:13:16.100026+0000","last_peered":"2024-04-23T19:13:16.100026+0000","last_clean":"2024-04-23T19:13:16.100026+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:13:16.100026+0000","last_undegraded":"2024-04-23T19:13:16.100026+0000","last_fullsized":"2024-04-23T19:13:16.100026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'17","reported_seq":72,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458912+0000","last_change":"2024-04-23T19:10:09.493592+0000","last_active":"2024-04-23T19:13:16.458912+0000","last_peered":"2024-04-23T19:13:16.458912+0000","last_clean":"2024-04-23T19:13:16.458912+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:13:16.458912+0000","last_undegraded":"2024-04-23T19:13:16.458912+0000","last_fullsized":"2024-04-23T19:13:16.458912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727109+0000","last_change":"2024-04-23T19:10:09.499521+0000","last_active":"2024-04-23T19:10:47.727109+0000","last_peered":"2024-04-23T19:10:47.727109+0000","last_clean":"2024-04-23T19:10:47.727109+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:10:47.727109+0000","last_undegraded":"2024-04-23T19:10:47.727109+0000","last_fullsized":"2024-04-23T19:10:47.727109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:05.408478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722404+0000","last_change":"2024-04-23T19:10:07.491914+0000","last_active":"2024-04-23T19:10:47.722404+0000","last_peered":"2024-04-23T19:10:47.722404+0000","last_clean":"2024-04-23T19:10:47.722404+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:10:47.722404+0000","last_undegraded":"2024-04-23T19:10:47.722404+0000","last_fullsized":"2024-04-23T19:10:47.722404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449380+0000","last_change":"2024-04-23T19:10:09.482719+0000","last_active":"2024-04-23T19:13:16.449380+0000","last_peered":"2024-04-23T19:13:16.449380+0000","last_clean":"2024-04-23T19:13:16.449380+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:13:16.449380+0000","last_undegraded":"2024-04-23T19:13:16.449380+0000","last_fullsized":"2024-04-23T19:13:16.449380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725054+0000","last_change":"2024-04-23T19:10:07.489933+0000","last_active":"2024-04-23T19:10:47.725054+0000","last_peered":"2024-04-23T19:10:47.725054+0000","last_clean":"2024-04-23T19:10:47.725054+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:10:47.725054+0000","last_undegraded":"2024-04-23T19:10:47.725054+0000","last_fullsized":"2024-04-23T19:10:47.725054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720219+0000","last_change":"2024-04-23T19:10:09.484959+0000","last_active":"2024-04-23T19:10:47.720219+0000","last_peered":"2024-04-23T19:10:47.720219+0000","last_clean":"2024-04-23T19:10:47.720219+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:10:47.720219+0000","last_undegraded":"2024-04-23T19:10:47.720219+0000","last_fullsized":"2024-04-23T19:10:47.720219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:54.767051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720449+0000","last_change":"2024-04-23T19:10:07.493585+0000","last_active":"2024-04-23T19:12:31.720449+0000","last_peered":"2024-04-23T19:12:31.720449+0000","last_clean":"2024-04-23T19:12:31.720449+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:12:31.720449+0000","last_undegraded":"2024-04-23T19:12:31.720449+0000","last_fullsized":"2024-04-23T19:12:31.720449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408189+0000","last_change":"2024-04-23T19:10:09.491221+0000","last_active":"2024-04-23T19:13:16.408189+0000","last_peered":"2024-04-23T19:13:16.408189+0000","last_clean":"2024-04-23T19:13:16.408189+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:16.408189+0000","last_undegraded":"2024-04-23T19:13:16.408189+0000","last_fullsized":"2024-04-23T19:13:16.408189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718838+0000","last_change":"2024-04-23T19:10:07.501878+0000","last_active":"2024-04-23T19:10:47.718838+0000","last_peered":"2024-04-23T19:10:47.718838+0000","last_clean":"2024-04-23T19:10:47.718838+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:10:47.718838+0000","last_undegraded":"2024-04-23T19:10:47.718838+0000","last_fullsized":"2024-04-23T19:10:47.718838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:28.263492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771581+0000","last_change":"2024-04-23T19:10:09.492916+0000","last_active":"2024-04-23T19:12:15.771581+0000","last_peered":"2024-04-23T19:12:15.771581+0000","last_clean":"2024-04-23T19:12:15.771581+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:12:15.771581+0000","last_undegraded":"2024-04-23T19:12:15.771581+0000","last_fullsized":"2024-04-23T19:12:15.771581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728570+0000","last_change":"2024-04-23T19:10:07.490910+0000","last_active":"2024-04-23T19:10:47.728570+0000","last_peered":"2024-04-23T19:10:47.728570+0000","last_clean":"2024-04-23T19:10:47.728570+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:10:47.728570+0000","last_undegraded":"2024-04-23T19:10:47.728570+0000","last_fullsized":"2024-04-23T19:10:47.728570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482419+0000","last_change":"2024-04-23T19:10:09.482527+0000","last_active":"2024-04-23T19:13:16.482419+0000","last_peered":"2024-04-23T19:13:16.482419+0000","last_clean":"2024-04-23T19:13:16.482419+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:16.482419+0000","last_undegraded":"2024-04-23T19:13:16.482419+0000","last_fullsized":"2024-04-23T19:13:16.482419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:46.899076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727215+0000","last_change":"2024-04-23T19:10:07.493956+0000","last_active":"2024-04-23T19:10:47.727215+0000","last_peered":"2024-04-23T19:10:47.727215+0000","last_clean":"2024-04-23T19:10:47.727215+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:10:47.727215+0000","last_undegraded":"2024-04-23T19:10:47.727215+0000","last_fullsized":"2024-04-23T19:10:47.727215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:22:20.896254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'35","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.422839+0000","last_change":"2024-04-23T19:10:17.514587+0000","last_active":"2024-04-23T19:13:16.422839+0000","last_peered":"2024-04-23T19:13:16.422839+0000","last_clean":"2024-04-23T19:13:16.422839+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:16.422839+0000","last_undegraded":"2024-04-23T19:13:16.422839+0000","last_fullsized":"2024-04-23T19:13:16.422839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:18.256132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.432503+0000","last_change":"2024-04-23T19:10:09.487433+0000","last_active":"2024-04-23T19:10:49.432503+0000","last_peered":"2024-04-23T19:10:49.432503+0000","last_clean":"2024-04-23T19:10:49.432503+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:10:49.432503+0000","last_undegraded":"2024-04-23T19:10:49.432503+0000","last_fullsized":"2024-04-23T19:10:49.432503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:36.758021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728416+0000","last_change":"2024-04-23T19:10:07.507346+0000","last_active":"2024-04-23T19:10:47.728416+0000","last_peered":"2024-04-23T19:10:47.728416+0000","last_clean":"2024-04-23T19:10:47.728416+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:10:47.728416+0000","last_undegraded":"2024-04-23T19:10:47.728416+0000","last_fullsized":"2024-04-23T19:10:47.728416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:42.849025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728325+0000","last_change":"2024-04-23T19:10:11.488429+0000","last_active":"2024-04-23T19:10:47.728325+0000","last_peered":"2024-04-23T19:10:47.728325+0000","last_clean":"2024-04-23T19:10:47.728325+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:10:47.728325+0000","last_undegraded":"2024-04-23T19:10:47.728325+0000","last_fullsized":"2024-04-23T19:10:47.728325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:50.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722656+0000","last_change":"2024-04-23T19:10:13.501588+0000","last_active":"2024-04-23T19:12:41.722656+0000","last_peered":"2024-04-23T19:12:41.722656+0000","last_clean":"2024-04-23T19:12:41.722656+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:12:41.722656+0000","last_undegraded":"2024-04-23T19:12:41.722656+0000","last_fullsized":"2024-04-23T19:12:41.722656+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:49:34.948969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722273+0000","last_change":"2024-04-23T19:10:15.509911+0000","last_active":"2024-04-23T19:10:47.722273+0000","last_peered":"2024-04-23T19:10:47.722273+0000","last_clean":"2024-04-23T19:10:47.722273+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:10:47.722273+0000","last_undegraded":"2024-04-23T19:10:47.722273+0000","last_fullsized":"2024-04-23T19:10:47.722273+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'9","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.539127+0000","last_change":"2024-04-23T19:10:17.514347+0000","last_active":"2024-04-23T19:13:16.539127+0000","last_peered":"2024-04-23T19:13:16.539127+0000","last_clean":"2024-04-23T19:13:16.539127+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:16.539127+0000","last_undegraded":"2024-04-23T19:13:16.539127+0000","last_fullsized":"2024-04-23T19:13:16.539127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:53.299426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399880+0000","last_change":"2024-04-23T19:10:09.496014+0000","last_active":"2024-04-23T19:13:16.399880+0000","last_peered":"2024-04-23T19:13:16.399880+0000","last_clean":"2024-04-23T19:13:16.399880+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:13:16.399880+0000","last_undegraded":"2024-04-23T19:13:16.399880+0000","last_fullsized":"2024-04-23T19:13:16.399880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:32.575973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720263+0000","last_change":"2024-04-23T19:10:07.492730+0000","last_active":"2024-04-23T19:10:47.720263+0000","last_peered":"2024-04-23T19:10:47.720263+0000","last_clean":"2024-04-23T19:10:47.720263+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:10:47.720263+0000","last_undegraded":"2024-04-23T19:10:47.720263+0000","last_fullsized":"2024-04-23T19:10:47.720263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"23'1","reported_seq":220,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:22.995732+0000","last_change":"2024-04-23T19:10:11.489225+0000","last_active":"2024-04-23T19:13:22.995732+0000","last_peered":"2024-04-23T19:13:22.995732+0000","last_clean":"2024-04-23T19:13:22.995732+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:13:22.995732+0000","last_undegraded":"2024-04-23T19:13:22.995732+0000","last_fullsized":"2024-04-23T19:13:22.995732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:01.595709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"30'6","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721580+0000","last_change":"2024-04-23T19:10:13.491509+0000","last_active":"2024-04-23T19:10:47.721580+0000","last_peered":"2024-04-23T19:10:47.721580+0000","last_clean":"2024-04-23T19:10:47.721580+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:10:47.721580+0000","last_undegraded":"2024-04-23T19:10:47.721580+0000","last_fullsized":"2024-04-23T19:10:47.721580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:32:32.880512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720747+0000","last_change":"2024-04-23T19:10:15.506746+0000","last_active":"2024-04-23T19:10:47.720747+0000","last_peered":"2024-04-23T19:10:47.720747+0000","last_clean":"2024-04-23T19:10:47.720747+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:10:47.720747+0000","last_undegraded":"2024-04-23T19:10:47.720747+0000","last_fullsized":"2024-04-23T19:10:47.720747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726990+0000","last_change":"2024-04-23T19:10:17.518249+0000","last_active":"2024-04-23T19:10:47.726990+0000","last_peered":"2024-04-23T19:10:47.726990+0000","last_clean":"2024-04-23T19:10:47.726990+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:10:47.726990+0000","last_undegraded":"2024-04-23T19:10:47.726990+0000","last_fullsized":"2024-04-23T19:10:47.726990+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:58.051610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447281+0000","last_change":"2024-04-23T19:10:09.490234+0000","last_active":"2024-04-23T19:13:16.447281+0000","last_peered":"2024-04-23T19:13:16.447281+0000","last_clean":"2024-04-23T19:13:16.447281+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:13:16.447281+0000","last_undegraded":"2024-04-23T19:13:16.447281+0000","last_fullsized":"2024-04-23T19:13:16.447281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:54.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.389417+0000","last_change":"2024-04-23T19:10:07.504260+0000","last_active":"2024-04-23T19:13:16.389417+0000","last_peered":"2024-04-23T19:13:16.389417+0000","last_clean":"2024-04-23T19:13:16.389417+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:13:16.389417+0000","last_undegraded":"2024-04-23T19:13:16.389417+0000","last_fullsized":"2024-04-23T19:13:16.389417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:46.596307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730636+0000","last_change":"2024-04-23T19:10:11.493566+0000","last_active":"2024-04-23T19:10:47.730636+0000","last_peered":"2024-04-23T19:10:47.730636+0000","last_clean":"2024-04-23T19:10:47.730636+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:10:47.730636+0000","last_undegraded":"2024-04-23T19:10:47.730636+0000","last_fullsized":"2024-04-23T19:10:47.730636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718899+0000","last_change":"2024-04-23T19:10:13.502669+0000","last_active":"2024-04-23T19:12:35.718899+0000","last_peered":"2024-04-23T19:12:35.718899+0000","last_clean":"2024-04-23T19:12:35.718899+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:12:35.718899+0000","last_undegraded":"2024-04-23T19:12:35.718899+0000","last_fullsized":"2024-04-23T19:12:35.718899+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:02.466667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'282","reported_seq":385,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.598276+0000","last_change":"2024-04-23T19:10:15.515398+0000","last_active":"2024-04-23T19:13:16.598276+0000","last_peered":"2024-04-23T19:13:16.598276+0000","last_clean":"2024-04-23T19:13:16.598276+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:13:16.598276+0000","last_undegraded":"2024-04-23T19:13:16.598276+0000","last_fullsized":"2024-04-23T19:13:16.598276+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.433967+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:10:49.433967+0000","last_peered":"2024-04-23T19:10:49.433967+0000","last_clean":"2024-04-23T19:10:49.433967+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:10:49.433967+0000","last_undegraded":"2024-04-23T19:10:49.433967+0000","last_fullsized":"2024-04-23T19:10:49.433967+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:55.634563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482380+0000","last_change":"2024-04-23T19:10:09.487095+0000","last_active":"2024-04-23T19:13:16.482380+0000","last_peered":"2024-04-23T19:13:16.482380+0000","last_clean":"2024-04-23T19:13:16.482380+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:13:16.482380+0000","last_undegraded":"2024-04-23T19:13:16.482380+0000","last_fullsized":"2024-04-23T19:13:16.482380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:05.454715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716719+0000","last_change":"2024-04-23T19:10:07.504154+0000","last_active":"2024-04-23T19:10:47.716719+0000","last_peered":"2024-04-23T19:10:47.716719+0000","last_clean":"2024-04-23T19:10:47.716719+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:10:47.716719+0000","last_undegraded":"2024-04-23T19:10:47.716719+0000","last_fullsized":"2024-04-23T19:10:47.716719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:32:24.070928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728932+0000","last_change":"2024-04-23T19:10:11.488228+0000","last_active":"2024-04-23T19:10:47.728932+0000","last_peered":"2024-04-23T19:10:47.728932+0000","last_clean":"2024-04-23T19:10:47.728932+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:10:47.728932+0000","last_undegraded":"2024-04-23T19:10:47.728932+0000","last_fullsized":"2024-04-23T19:10:47.728932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:01.154683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'34","reported_seq":80,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.088997+0000","last_change":"2024-04-23T19:10:13.505813+0000","last_active":"2024-04-23T19:13:23.088997+0000","last_peered":"2024-04-23T19:13:23.088997+0000","last_clean":"2024-04-23T19:13:23.088997+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:13:23.088997+0000","last_undegraded":"2024-04-23T19:13:23.088997+0000","last_fullsized":"2024-04-23T19:13:23.088997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:39.996179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'285","reported_seq":404,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.028659+0000","last_change":"2024-04-23T19:10:15.500511+0000","last_active":"2024-04-23T19:13:23.028659+0000","last_peered":"2024-04-23T19:13:23.028659+0000","last_clean":"2024-04-23T19:13:23.028659+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:23.028659+0000","last_undegraded":"2024-04-23T19:13:23.028659+0000","last_fullsized":"2024-04-23T19:13:23.028659+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:23.813091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449319+0000","last_change":"2024-04-23T19:10:47.730482+0000","last_active":"2024-04-23T19:13:16.449319+0000","last_peered":"2024-04-23T19:13:16.449319+0000","last_clean":"2024-04-23T19:13:16.449319+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:13:16.449319+0000","last_undegraded":"2024-04-23T19:13:16.449319+0000","last_fullsized":"2024-04-23T19:13:16.449319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:33.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545310+0000","last_change":"2024-04-23T19:10:07.492480+0000","last_active":"2024-04-23T19:10:48.545310+0000","last_peered":"2024-04-23T19:10:48.545310+0000","last_clean":"2024-04-23T19:10:48.545310+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:10:48.545310+0000","last_undegraded":"2024-04-23T19:10:48.545310+0000","last_fullsized":"2024-04-23T19:10:48.545310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716980+0000","last_change":"2024-04-23T19:10:11.490040+0000","last_active":"2024-04-23T19:10:47.716980+0000","last_peered":"2024-04-23T19:10:47.716980+0000","last_clean":"2024-04-23T19:10:47.716980+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:10:47.716980+0000","last_undegraded":"2024-04-23T19:10:47.716980+0000","last_fullsized":"2024-04-23T19:10:47.716980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:53:07.412668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'43","reported_seq":94,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.131417+0000","last_change":"2024-04-23T19:10:13.494728+0000","last_active":"2024-04-23T19:13:23.131417+0000","last_peered":"2024-04-23T19:13:23.131417+0000","last_clean":"2024-04-23T19:13:23.131417+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:13:23.131417+0000","last_undegraded":"2024-04-23T19:13:23.131417+0000","last_fullsized":"2024-04-23T19:13:23.131417+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:56.304108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728369+0000","last_change":"2024-04-23T19:10:15.506637+0000","last_active":"2024-04-23T19:10:47.728369+0000","last_peered":"2024-04-23T19:10:47.728369+0000","last_clean":"2024-04-23T19:10:47.728369+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:10:47.728369+0000","last_undegraded":"2024-04-23T19:10:47.728369+0000","last_fullsized":"2024-04-23T19:10:47.728369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:13.464606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.971818+0000","last_change":"2024-04-23T19:10:17.519710+0000","last_active":"2024-04-23T19:11:31.971818+0000","last_peered":"2024-04-23T19:11:31.971818+0000","last_clean":"2024-04-23T19:11:31.971818+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:11:31.971818+0000","last_undegraded":"2024-04-23T19:11:31.971818+0000","last_fullsized":"2024-04-23T19:11:31.971818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:10.239585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'7","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.488606+0000","last_change":"2024-04-23T19:10:09.482665+0000","last_active":"2024-04-23T19:13:16.488606+0000","last_peered":"2024-04-23T19:13:16.488606+0000","last_clean":"2024-04-23T19:13:16.488606+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:16.488606+0000","last_undegraded":"2024-04-23T19:13:16.488606+0000","last_fullsized":"2024-04-23T19:13:16.488606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724246+0000","last_change":"2024-04-23T19:10:07.500938+0000","last_active":"2024-04-23T19:10:47.724246+0000","last_peered":"2024-04-23T19:10:47.724246+0000","last_clean":"2024-04-23T19:10:47.724246+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:10:47.724246+0000","last_undegraded":"2024-04-23T19:10:47.724246+0000","last_fullsized":"2024-04-23T19:10:47.724246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727117+0000","last_change":"2024-04-23T19:10:11.493617+0000","last_active":"2024-04-23T19:10:47.727117+0000","last_peered":"2024-04-23T19:10:47.727117+0000","last_clean":"2024-04-23T19:10:47.727117+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:10:47.727117+0000","last_undegraded":"2024-04-23T19:10:47.727117+0000","last_fullsized":"2024-04-23T19:10:47.727117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:32.061098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'31","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.296627+0000","last_change":"2024-04-23T19:10:13.501851+0000","last_active":"2024-04-23T19:12:14.296627+0000","last_peered":"2024-04-23T19:12:14.296627+0000","last_clean":"2024-04-23T19:12:14.296627+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:12:14.296627+0000","last_undegraded":"2024-04-23T19:12:14.296627+0000","last_fullsized":"2024-04-23T19:12:14.296627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:30.608779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727229+0000","last_change":"2024-04-23T19:10:15.502245+0000","last_active":"2024-04-23T19:10:47.727229+0000","last_peered":"2024-04-23T19:10:47.727229+0000","last_clean":"2024-04-23T19:10:47.727229+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:10:47.727229+0000","last_undegraded":"2024-04-23T19:10:47.727229+0000","last_fullsized":"2024-04-23T19:10:47.727229+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'5","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.261614+0000","last_change":"2024-04-23T19:10:17.518314+0000","last_active":"2024-04-23T19:13:16.261614+0000","last_peered":"2024-04-23T19:13:16.261614+0000","last_clean":"2024-04-23T19:13:16.261614+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:13:16.261614+0000","last_undegraded":"2024-04-23T19:13:16.261614+0000","last_fullsized":"2024-04-23T19:13:16.261614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:11.577457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.166247+0000","last_change":"2024-04-23T19:10:09.492229+0000","last_active":"2024-04-23T19:13:16.166247+0000","last_peered":"2024-04-23T19:13:16.166247+0000","last_clean":"2024-04-23T19:13:16.166247+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:13:16.166247+0000","last_undegraded":"2024-04-23T19:13:16.166247+0000","last_fullsized":"2024-04-23T19:13:16.166247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:36.861914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729281+0000","last_change":"2024-04-23T19:10:07.492637+0000","last_active":"2024-04-23T19:10:47.729281+0000","last_peered":"2024-04-23T19:10:47.729281+0000","last_clean":"2024-04-23T19:10:47.729281+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:10:47.729281+0000","last_undegraded":"2024-04-23T19:10:47.729281+0000","last_fullsized":"2024-04-23T19:10:47.729281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.154249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729542+0000","last_change":"2024-04-23T19:10:11.491503+0000","last_active":"2024-04-23T19:10:47.729542+0000","last_peered":"2024-04-23T19:10:47.729542+0000","last_clean":"2024-04-23T19:10:47.729542+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:10:47.729542+0000","last_undegraded":"2024-04-23T19:10:47.729542+0000","last_fullsized":"2024-04-23T19:10:47.729542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.318445+0000","last_change":"2024-04-23T19:10:13.508355+0000","last_active":"2024-04-23T19:12:14.318445+0000","last_peered":"2024-04-23T19:12:14.318445+0000","last_clean":"2024-04-23T19:12:14.318445+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:12:14.318445+0000","last_undegraded":"2024-04-23T19:12:14.318445+0000","last_fullsized":"2024-04-23T19:12:14.318445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:19.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'285","reported_seq":351,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.029925+0000","last_change":"2024-04-23T19:10:15.509684+0000","last_active":"2024-04-23T19:13:23.029925+0000","last_peered":"2024-04-23T19:13:23.029925+0000","last_clean":"2024-04-23T19:13:23.029925+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:23.029925+0000","last_undegraded":"2024-04-23T19:13:23.029925+0000","last_fullsized":"2024-04-23T19:13:23.029925+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728722+0000","last_change":"2024-04-23T19:10:17.520791+0000","last_active":"2024-04-23T19:10:47.728722+0000","last_peered":"2024-04-23T19:10:47.728722+0000","last_clean":"2024-04-23T19:10:47.728722+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:10:47.728722+0000","last_undegraded":"2024-04-23T19:10:47.728722+0000","last_fullsized":"2024-04-23T19:10:47.728722+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:33:03.997750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399785+0000","last_change":"2024-04-23T19:10:09.476300+0000","last_active":"2024-04-23T19:13:16.399785+0000","last_peered":"2024-04-23T19:13:16.399785+0000","last_clean":"2024-04-23T19:13:16.399785+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:16.399785+0000","last_undegraded":"2024-04-23T19:13:16.399785+0000","last_fullsized":"2024-04-23T19:13:16.399785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:59.576302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726343+0000","last_change":"2024-04-23T19:10:07.501083+0000","last_active":"2024-04-23T19:10:47.726343+0000","last_peered":"2024-04-23T19:10:47.726343+0000","last_clean":"2024-04-23T19:10:47.726343+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:10:47.726343+0000","last_undegraded":"2024-04-23T19:10:47.726343+0000","last_fullsized":"2024-04-23T19:10:47.726343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:08.925701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727624+0000","last_change":"2024-04-23T19:10:11.493665+0000","last_active":"2024-04-23T19:10:47.727624+0000","last_peered":"2024-04-23T19:10:47.727624+0000","last_clean":"2024-04-23T19:10:47.727624+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:10:47.727624+0000","last_undegraded":"2024-04-23T19:10:47.727624+0000","last_fullsized":"2024-04-23T19:10:47.727624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:29.762026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'54","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.320262+0000","last_change":"2024-04-23T19:10:13.499017+0000","last_active":"2024-04-23T19:12:14.320262+0000","last_peered":"2024-04-23T19:12:14.320262+0000","last_clean":"2024-04-23T19:12:14.320262+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:12:14.320262+0000","last_undegraded":"2024-04-23T19:12:14.320262+0000","last_fullsized":"2024-04-23T19:12:14.320262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:42.419535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726896+0000","last_change":"2024-04-23T19:10:15.503053+0000","last_active":"2024-04-23T19:10:47.726896+0000","last_peered":"2024-04-23T19:10:47.726896+0000","last_clean":"2024-04-23T19:10:47.726896+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:10:47.726896+0000","last_undegraded":"2024-04-23T19:10:47.726896+0000","last_fullsized":"2024-04-23T19:10:47.726896+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:38.685045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'18","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.583903+0000","last_change":"2024-04-23T19:10:17.513990+0000","last_active":"2024-04-23T19:13:16.583903+0000","last_peered":"2024-04-23T19:13:16.583903+0000","last_clean":"2024-04-23T19:13:16.583903+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:13:16.583903+0000","last_undegraded":"2024-04-23T19:13:16.583903+0000","last_fullsized":"2024-04-23T19:13:16.583903+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:51:57.264911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258650+0000","last_change":"2024-04-23T19:10:09.490325+0000","last_active":"2024-04-23T19:13:16.258650+0000","last_peered":"2024-04-23T19:13:16.258650+0000","last_clean":"2024-04-23T19:13:16.258650+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:16.258650+0000","last_undegraded":"2024-04-23T19:13:16.258650+0000","last_fullsized":"2024-04-23T19:13:16.258650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728226+0000","last_change":"2024-04-23T19:10:07.490182+0000","last_active":"2024-04-23T19:10:47.728226+0000","last_peered":"2024-04-23T19:10:47.728226+0000","last_clean":"2024-04-23T19:10:47.728226+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:10:47.728226+0000","last_undegraded":"2024-04-23T19:10:47.728226+0000","last_fullsized":"2024-04-23T19:10:47.728226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.284465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719940+0000","last_change":"2024-04-23T19:10:11.490064+0000","last_active":"2024-04-23T19:10:47.719940+0000","last_peered":"2024-04-23T19:10:47.719940+0000","last_clean":"2024-04-23T19:10:47.719940+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:10:47.719940+0000","last_undegraded":"2024-04-23T19:10:47.719940+0000","last_fullsized":"2024-04-23T19:10:47.719940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:48.063186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.300676+0000","last_change":"2024-04-23T19:10:13.491370+0000","last_active":"2024-04-23T19:12:14.300676+0000","last_peered":"2024-04-23T19:12:14.300676+0000","last_clean":"2024-04-23T19:12:14.300676+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:12:14.300676+0000","last_undegraded":"2024-04-23T19:12:14.300676+0000","last_fullsized":"2024-04-23T19:12:14.300676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:59.208837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728086+0000","last_change":"2024-04-23T19:10:15.512494+0000","last_active":"2024-04-23T19:10:47.728086+0000","last_peered":"2024-04-23T19:10:47.728086+0000","last_clean":"2024-04-23T19:10:47.728086+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:10:47.728086+0000","last_undegraded":"2024-04-23T19:10:47.728086+0000","last_fullsized":"2024-04-23T19:10:47.728086+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:54.341417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718208+0000","last_change":"2024-04-23T19:10:17.511337+0000","last_active":"2024-04-23T19:12:16.718208+0000","last_peered":"2024-04-23T19:12:16.718208+0000","last_clean":"2024-04-23T19:12:16.718208+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:12:16.718208+0000","last_undegraded":"2024-04-23T19:12:16.718208+0000","last_fullsized":"2024-04-23T19:12:16.718208+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:31.868099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536682+0000","last_change":"2024-04-23T19:10:09.499760+0000","last_active":"2024-04-23T19:13:16.536682+0000","last_peered":"2024-04-23T19:13:16.536682+0000","last_clean":"2024-04-23T19:13:16.536682+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:16.536682+0000","last_undegraded":"2024-04-23T19:13:16.536682+0000","last_fullsized":"2024-04-23T19:13:16.536682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:06:43.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727856+0000","last_change":"2024-04-23T19:10:07.494969+0000","last_active":"2024-04-23T19:10:47.727856+0000","last_peered":"2024-04-23T19:10:47.727856+0000","last_clean":"2024-04-23T19:10:47.727856+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:10:47.727856+0000","last_undegraded":"2024-04-23T19:10:47.727856+0000","last_fullsized":"2024-04-23T19:10:47.727856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:47.276166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728157+0000","last_change":"2024-04-23T19:10:11.491444+0000","last_active":"2024-04-23T19:10:47.728157+0000","last_peered":"2024-04-23T19:10:47.728157+0000","last_clean":"2024-04-23T19:10:47.728157+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:10:47.728157+0000","last_undegraded":"2024-04-23T19:10:47.728157+0000","last_fullsized":"2024-04-23T19:10:47.728157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:00.459979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'39","reported_seq":88,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.164028+0000","last_change":"2024-04-23T19:10:13.493342+0000","last_active":"2024-04-23T19:13:23.164028+0000","last_peered":"2024-04-23T19:13:23.164028+0000","last_clean":"2024-04-23T19:13:23.164028+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:13:23.164028+0000","last_undegraded":"2024-04-23T19:13:23.164028+0000","last_fullsized":"2024-04-23T19:13:23.164028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:44.904145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'285","reported_seq":406,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.024887+0000","last_change":"2024-04-23T19:10:15.507087+0000","last_active":"2024-04-23T19:13:23.024887+0000","last_peered":"2024-04-23T19:13:23.024887+0000","last_clean":"2024-04-23T19:13:23.024887+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:23.024887+0000","last_undegraded":"2024-04-23T19:13:23.024887+0000","last_fullsized":"2024-04-23T19:13:23.024887+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:48.999181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'12","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712278+0000","last_change":"2024-04-23T19:10:17.518152+0000","last_active":"2024-04-23T19:12:25.712278+0000","last_peered":"2024-04-23T19:12:25.712278+0000","last_clean":"2024-04-23T19:12:25.712278+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:12:25.712278+0000","last_undegraded":"2024-04-23T19:12:25.712278+0000","last_fullsized":"2024-04-23T19:12:25.712278+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399883+0000","last_change":"2024-04-23T19:10:09.493242+0000","last_active":"2024-04-23T19:13:16.399883+0000","last_peered":"2024-04-23T19:13:16.399883+0000","last_clean":"2024-04-23T19:13:16.399883+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:13:16.399883+0000","last_undegraded":"2024-04-23T19:13:16.399883+0000","last_fullsized":"2024-04-23T19:13:16.399883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.679903+0000","last_change":"2024-04-23T19:10:07.508434+0000","last_active":"2024-04-23T19:10:49.679903+0000","last_peered":"2024-04-23T19:10:49.679903+0000","last_clean":"2024-04-23T19:10:49.679903+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:10:49.679903+0000","last_undegraded":"2024-04-23T19:10:49.679903+0000","last_fullsized":"2024-04-23T19:10:49.679903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:44.765151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'3","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.385678+0000","last_change":"2024-04-23T19:10:11.491234+0000","last_active":"2024-04-23T19:12:15.385678+0000","last_peered":"2024-04-23T19:12:15.385678+0000","last_clean":"2024-04-23T19:12:15.385678+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:12:15.385678+0000","last_undegraded":"2024-04-23T19:12:15.385678+0000","last_fullsized":"2024-04-23T19:12:15.385678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:03.467829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727806+0000","last_change":"2024-04-23T19:10:13.501643+0000","last_active":"2024-04-23T19:10:47.727806+0000","last_peered":"2024-04-23T19:10:47.727806+0000","last_clean":"2024-04-23T19:10:47.727806+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:10:47.727806+0000","last_undegraded":"2024-04-23T19:10:47.727806+0000","last_fullsized":"2024-04-23T19:10:47.727806+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:28.282546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'282","reported_seq":465,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575725+0000","last_change":"2024-04-23T19:10:15.501932+0000","last_active":"2024-04-23T19:13:16.575725+0000","last_peered":"2024-04-23T19:13:16.575725+0000","last_clean":"2024-04-23T19:13:16.575725+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:16.575725+0000","last_undegraded":"2024-04-23T19:13:16.575725+0000","last_fullsized":"2024-04-23T19:13:16.575725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:08.232109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.546370+0000","last_change":"2024-04-23T19:10:17.515498+0000","last_active":"2024-04-23T19:12:15.546370+0000","last_peered":"2024-04-23T19:12:15.546370+0000","last_clean":"2024-04-23T19:12:15.546370+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:12:15.546370+0000","last_undegraded":"2024-04-23T19:12:15.546370+0000","last_fullsized":"2024-04-23T19:12:15.546370+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:42.717317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.699282+0000","last_change":"2024-04-23T19:10:09.494353+0000","last_active":"2024-04-23T19:10:49.699282+0000","last_peered":"2024-04-23T19:10:49.699282+0000","last_clean":"2024-04-23T19:10:49.699282+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:10:49.699282+0000","last_undegraded":"2024-04-23T19:10:49.699282+0000","last_fullsized":"2024-04-23T19:10:49.699282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:33.419635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.386367+0000","last_change":"2024-04-23T19:10:07.507496+0000","last_active":"2024-04-23T19:13:16.386367+0000","last_peered":"2024-04-23T19:13:16.386367+0000","last_clean":"2024-04-23T19:13:16.386367+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:13:16.386367+0000","last_undegraded":"2024-04-23T19:13:16.386367+0000","last_fullsized":"2024-04-23T19:13:16.386367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:08.216950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728027+0000","last_change":"2024-04-23T19:10:11.490678+0000","last_active":"2024-04-23T19:10:47.728027+0000","last_peered":"2024-04-23T19:10:47.728027+0000","last_clean":"2024-04-23T19:10:47.728027+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:10:47.728027+0000","last_undegraded":"2024-04-23T19:10:47.728027+0000","last_fullsized":"2024-04-23T19:10:47.728027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:00.102054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'38","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.313282+0000","last_change":"2024-04-23T19:10:13.491356+0000","last_active":"2024-04-23T19:12:14.313282+0000","last_peered":"2024-04-23T19:12:14.313282+0000","last_clean":"2024-04-23T19:12:14.313282+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:12:14.313282+0000","last_undegraded":"2024-04-23T19:12:14.313282+0000","last_fullsized":"2024-04-23T19:12:14.313282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:58.150827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720298+0000","last_change":"2024-04-23T19:10:15.506621+0000","last_active":"2024-04-23T19:10:47.720298+0000","last_peered":"2024-04-23T19:10:47.720298+0000","last_clean":"2024-04-23T19:10:47.720298+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:10:47.720298+0000","last_undegraded":"2024-04-23T19:10:47.720298+0000","last_fullsized":"2024-04-23T19:10:47.720298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:01.215939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.516499+0000","last_change":"2024-04-23T19:10:17.506577+0000","last_active":"2024-04-23T19:13:16.516499+0000","last_peered":"2024-04-23T19:13:16.516499+0000","last_clean":"2024-04-23T19:13:16.516499+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:16.516499+0000","last_undegraded":"2024-04-23T19:13:16.516499+0000","last_fullsized":"2024-04-23T19:13:16.516499+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:44.079746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718448+0000","last_change":"2024-04-23T19:10:09.499812+0000","last_active":"2024-04-23T19:12:16.718448+0000","last_peered":"2024-04-23T19:12:16.718448+0000","last_clean":"2024-04-23T19:12:16.718448+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:12:16.718448+0000","last_undegraded":"2024-04-23T19:12:16.718448+0000","last_fullsized":"2024-04-23T19:12:16.718448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:34.586789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":201,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.716709+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:10:47.716709+0000","last_peered":"2024-04-23T19:10:47.716709+0000","last_clean":"2024-04-23T19:10:47.716709+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:10:47.716709+0000","last_undegraded":"2024-04-23T19:10:47.716709+0000","last_fullsized":"2024-04-23T19:10:47.716709+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:35.747545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728488+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:10:47.728488+0000","last_peered":"2024-04-23T19:10:47.728488+0000","last_clean":"2024-04-23T19:10:47.728488+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:10:47.728488+0000","last_undegraded":"2024-04-23T19:10:47.728488+0000","last_fullsized":"2024-04-23T19:10:47.728488+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:09.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.547770+0000","last_change":"2024-04-23T19:10:07.508995+0000","last_active":"2024-04-23T19:10:48.547770+0000","last_peered":"2024-04-23T19:10:48.547770+0000","last_clean":"2024-04-23T19:10:48.547770+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:10:48.547770+0000","last_undegraded":"2024-04-23T19:10:48.547770+0000","last_fullsized":"2024-04-23T19:10:48.547770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:08.545897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718470+0000","last_change":"2024-04-23T19:10:11.486708+0000","last_active":"2024-04-23T19:10:47.718470+0000","last_peered":"2024-04-23T19:10:47.718470+0000","last_clean":"2024-04-23T19:10:47.718470+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:10:47.718470+0000","last_undegraded":"2024-04-23T19:10:47.718470+0000","last_fullsized":"2024-04-23T19:10:47.718470+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:46.474565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'77","reported_seq":146,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.325024+0000","last_change":"2024-04-23T19:10:13.500080+0000","last_active":"2024-04-23T19:12:14.325024+0000","last_peered":"2024-04-23T19:12:14.325024+0000","last_clean":"2024-04-23T19:12:14.325024+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:12:14.325024+0000","last_undegraded":"2024-04-23T19:12:14.325024+0000","last_fullsized":"2024-04-23T19:12:14.325024+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:43.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728972+0000","last_change":"2024-04-23T19:10:15.502149+0000","last_active":"2024-04-23T19:10:47.728972+0000","last_peered":"2024-04-23T19:10:47.728972+0000","last_clean":"2024-04-23T19:10:47.728972+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:10:47.728972+0000","last_undegraded":"2024-04-23T19:10:47.728972+0000","last_fullsized":"2024-04-23T19:10:47.728972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:59.765301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":162,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.491783+0000","last_change":"2024-04-23T19:10:17.516475+0000","last_active":"2024-04-23T19:13:16.491783+0000","last_peered":"2024-04-23T19:13:16.491783+0000","last_clean":"2024-04-23T19:13:16.491783+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:16.491783+0000","last_undegraded":"2024-04-23T19:13:16.491783+0000","last_fullsized":"2024-04-23T19:13:16.491783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:20.418910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.149738+0000","last_change":"2024-04-23T19:10:09.488098+0000","last_active":"2024-04-23T19:11:32.149738+0000","last_peered":"2024-04-23T19:11:32.149738+0000","last_clean":"2024-04-23T19:11:32.149738+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:11:32.149738+0000","last_undegraded":"2024-04-23T19:11:32.149738+0000","last_fullsized":"2024-04-23T19:11:32.149738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:52.645415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725571+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:10:47.725571+0000","last_peered":"2024-04-23T19:10:47.725571+0000","last_clean":"2024-04-23T19:10:47.725571+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:10:47.725571+0000","last_undegraded":"2024-04-23T19:10:47.725571+0000","last_fullsized":"2024-04-23T19:10:47.725571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:17.746806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726921+0000","last_change":"2024-04-23T19:10:07.486483+0000","last_active":"2024-04-23T19:10:47.726921+0000","last_peered":"2024-04-23T19:10:47.726921+0000","last_clean":"2024-04-23T19:10:47.726921+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:10:47.726921+0000","last_undegraded":"2024-04-23T19:10:47.726921+0000","last_fullsized":"2024-04-23T19:10:47.726921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729492+0000","last_change":"2024-04-23T19:10:11.487708+0000","last_active":"2024-04-23T19:10:47.729492+0000","last_peered":"2024-04-23T19:10:47.729492+0000","last_clean":"2024-04-23T19:10:47.729492+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:10:47.729492+0000","last_undegraded":"2024-04-23T19:10:47.729492+0000","last_fullsized":"2024-04-23T19:10:47.729492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:07.101469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'273","reported_seq":476,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575779+0000","last_change":"2024-04-23T19:10:13.500713+0000","last_active":"2024-04-23T19:13:16.575779+0000","last_peered":"2024-04-23T19:13:16.575779+0000","last_clean":"2024-04-23T19:13:16.575779+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:16.575779+0000","last_undegraded":"2024-04-23T19:13:16.575779+0000","last_fullsized":"2024-04-23T19:13:16.575779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:22:46.992080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727751+0000","last_change":"2024-04-23T19:10:15.513032+0000","last_active":"2024-04-23T19:10:47.727751+0000","last_peered":"2024-04-23T19:10:47.727751+0000","last_clean":"2024-04-23T19:10:47.727751+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:10:47.727751+0000","last_undegraded":"2024-04-23T19:10:47.727751+0000","last_fullsized":"2024-04-23T19:10:47.727751+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:13.084568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.968215+0000","last_change":"2024-04-23T19:10:17.518750+0000","last_active":"2024-04-23T19:11:31.968215+0000","last_peered":"2024-04-23T19:11:31.968215+0000","last_clean":"2024-04-23T19:11:31.968215+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:11:31.968215+0000","last_undegraded":"2024-04-23T19:11:31.968215+0000","last_fullsized":"2024-04-23T19:11:31.968215+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:02.438494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482305+0000","last_change":"2024-04-23T19:10:09.486782+0000","last_active":"2024-04-23T19:13:16.482305+0000","last_peered":"2024-04-23T19:13:16.482305+0000","last_clean":"2024-04-23T19:13:16.482305+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:13:16.482305+0000","last_undegraded":"2024-04-23T19:13:16.482305+0000","last_fullsized":"2024-04-23T19:13:16.482305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:49.283977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718645+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:10:47.718645+0000","last_peered":"2024-04-23T19:10:47.718645+0000","last_clean":"2024-04-23T19:10:47.718645+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:10:47.718645+0000","last_undegraded":"2024-04-23T19:10:47.718645+0000","last_fullsized":"2024-04-23T19:10:47.718645+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:52:45.326443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726584+0000","last_change":"2024-04-23T19:10:07.491212+0000","last_active":"2024-04-23T19:10:47.726584+0000","last_peered":"2024-04-23T19:10:47.726584+0000","last_clean":"2024-04-23T19:10:47.726584+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:10:47.726584+0000","last_undegraded":"2024-04-23T19:10:47.726584+0000","last_fullsized":"2024-04-23T19:10:47.726584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"23'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.736899+0000","last_change":"2024-04-23T19:10:47.736369+0000","last_active":"2024-04-23T19:10:47.736899+0000","last_peered":"2024-04-23T19:10:47.736899+0000","last_clean":"2024-04-23T19:10:47.736899+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:10:47.736899+0000","last_undegraded":"2024-04-23T19:10:47.736899+0000","last_fullsized":"2024-04-23T19:10:47.736899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:05.556192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'16","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.576017+0000","last_change":"2024-04-23T19:10:13.495216+0000","last_active":"2024-04-23T19:13:16.576017+0000","last_peered":"2024-04-23T19:13:16.576017+0000","last_clean":"2024-04-23T19:13:16.576017+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:16.576017+0000","last_undegraded":"2024-04-23T19:13:16.576017+0000","last_fullsized":"2024-04-23T19:13:16.576017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:32.747522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728617+0000","last_change":"2024-04-23T19:10:15.502291+0000","last_active":"2024-04-23T19:10:47.728617+0000","last_peered":"2024-04-23T19:10:47.728617+0000","last_clean":"2024-04-23T19:10:47.728617+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:10:47.728617+0000","last_undegraded":"2024-04-23T19:10:47.728617+0000","last_fullsized":"2024-04-23T19:10:47.728617+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:04.115844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725858+0000","last_change":"2024-04-23T19:10:17.517093+0000","last_active":"2024-04-23T19:10:47.725858+0000","last_peered":"2024-04-23T19:10:47.725858+0000","last_clean":"2024-04-23T19:10:47.725858+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:10:47.725858+0000","last_undegraded":"2024-04-23T19:10:47.725858+0000","last_fullsized":"2024-04-23T19:10:47.725858+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:42.364548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399934+0000","last_change":"2024-04-23T19:10:09.477714+0000","last_active":"2024-04-23T19:13:16.399934+0000","last_peered":"2024-04-23T19:13:16.399934+0000","last_clean":"2024-04-23T19:13:16.399934+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:16.399934+0000","last_undegraded":"2024-04-23T19:13:16.399934+0000","last_fullsized":"2024-04-23T19:13:16.399934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726493+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:10:47.726493+0000","last_peered":"2024-04-23T19:10:47.726493+0000","last_clean":"2024-04-23T19:10:47.726493+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:10:47.726493+0000","last_undegraded":"2024-04-23T19:10:47.726493+0000","last_fullsized":"2024-04-23T19:10:47.726493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:51.612339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720012+0000","last_change":"2024-04-23T19:10:07.482292+0000","last_active":"2024-04-23T19:10:47.720012+0000","last_peered":"2024-04-23T19:10:47.720012+0000","last_clean":"2024-04-23T19:10:47.720012+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:10:47.720012+0000","last_undegraded":"2024-04-23T19:10:47.720012+0000","last_fullsized":"2024-04-23T19:10:47.720012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729072+0000","last_change":"2024-04-23T19:10:11.490475+0000","last_active":"2024-04-23T19:10:47.729072+0000","last_peered":"2024-04-23T19:10:47.729072+0000","last_clean":"2024-04-23T19:10:47.729072+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:10:47.729072+0000","last_undegraded":"2024-04-23T19:10:47.729072+0000","last_fullsized":"2024-04-23T19:10:47.729072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:37.989491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'14","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.784607+0000","last_change":"2024-04-23T19:10:13.501263+0000","last_active":"2024-04-23T19:13:15.784607+0000","last_peered":"2024-04-23T19:13:15.784607+0000","last_clean":"2024-04-23T19:13:15.784607+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:15.784607+0000","last_undegraded":"2024-04-23T19:13:15.784607+0000","last_fullsized":"2024-04-23T19:13:15.784607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:45:20.243610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728295+0000","last_change":"2024-04-23T19:10:15.512566+0000","last_active":"2024-04-23T19:10:47.728295+0000","last_peered":"2024-04-23T19:10:47.728295+0000","last_clean":"2024-04-23T19:10:47.728295+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:10:47.728295+0000","last_undegraded":"2024-04-23T19:10:47.728295+0000","last_fullsized":"2024-04-23T19:10:47.728295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:37:48.972551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.182118+0000","last_change":"2024-04-23T19:10:17.515693+0000","last_active":"2024-04-23T19:13:16.182118+0000","last_peered":"2024-04-23T19:13:16.182118+0000","last_clean":"2024-04-23T19:13:16.182118+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:13:16.182118+0000","last_undegraded":"2024-04-23T19:13:16.182118+0000","last_fullsized":"2024-04-23T19:13:16.182118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:12.374714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447462+0000","last_change":"2024-04-23T19:10:09.494884+0000","last_active":"2024-04-23T19:13:16.447462+0000","last_peered":"2024-04-23T19:13:16.447462+0000","last_clean":"2024-04-23T19:13:16.447462+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:16.447462+0000","last_undegraded":"2024-04-23T19:13:16.447462+0000","last_fullsized":"2024-04-23T19:13:16.447462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:55:10.553708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.383442+0000","last_change":"2024-04-23T19:10:07.483919+0000","last_active":"2024-04-23T19:13:16.383442+0000","last_peered":"2024-04-23T19:13:16.383442+0000","last_clean":"2024-04-23T19:13:16.383442+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:13:16.383442+0000","last_undegraded":"2024-04-23T19:13:16.383442+0000","last_fullsized":"2024-04-23T19:13:16.383442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728467+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:10:47.728467+0000","last_peered":"2024-04-23T19:10:47.728467+0000","last_clean":"2024-04-23T19:10:47.728467+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:10:47.728467+0000","last_undegraded":"2024-04-23T19:10:47.728467+0000","last_fullsized":"2024-04-23T19:10:47.728467+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:34:51.167423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.607068+0000","last_change":"2024-04-23T19:10:47.730021+0000","last_active":"2024-04-23T19:13:16.607068+0000","last_peered":"2024-04-23T19:13:16.607068+0000","last_clean":"2024-04-23T19:13:16.607068+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:13:16.607068+0000","last_undegraded":"2024-04-23T19:13:16.607068+0000","last_fullsized":"2024-04-23T19:13:16.607068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:56.417228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'69","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.160725+0000","last_change":"2024-04-23T19:10:13.505680+0000","last_active":"2024-04-23T19:13:23.160725+0000","last_peered":"2024-04-23T19:13:23.160725+0000","last_clean":"2024-04-23T19:13:23.160725+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:13:23.160725+0000","last_undegraded":"2024-04-23T19:13:23.160725+0000","last_fullsized":"2024-04-23T19:13:23.160725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:43:59.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719607+0000","last_change":"2024-04-23T19:10:15.506374+0000","last_active":"2024-04-23T19:10:47.719607+0000","last_peered":"2024-04-23T19:10:47.719607+0000","last_clean":"2024-04-23T19:10:47.719607+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:10:47.719607+0000","last_undegraded":"2024-04-23T19:10:47.719607+0000","last_fullsized":"2024-04-23T19:10:47.719607+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:42.776000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.868259+0000","last_change":"2024-04-23T19:10:17.506903+0000","last_active":"2024-04-23T19:13:15.868259+0000","last_peered":"2024-04-23T19:13:15.868259+0000","last_clean":"2024-04-23T19:13:15.868259+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:15.868259+0000","last_undegraded":"2024-04-23T19:13:15.868259+0000","last_fullsized":"2024-04-23T19:13:15.868259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:24.436159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447101+0000","last_change":"2024-04-23T19:10:09.492787+0000","last_active":"2024-04-23T19:13:16.447101+0000","last_peered":"2024-04-23T19:13:16.447101+0000","last_clean":"2024-04-23T19:13:16.447101+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:16.447101+0000","last_undegraded":"2024-04-23T19:13:16.447101+0000","last_fullsized":"2024-04-23T19:13:16.447101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.545103+0000","last_change":"2024-04-23T19:10:07.486351+0000","last_active":"2024-04-23T19:10:48.545103+0000","last_peered":"2024-04-23T19:10:48.545103+0000","last_clean":"2024-04-23T19:10:48.545103+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:10:48.545103+0000","last_undegraded":"2024-04-23T19:10:48.545103+0000","last_fullsized":"2024-04-23T19:10:48.545103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:48:50.668566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728626+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:10:47.728626+0000","last_peered":"2024-04-23T19:10:47.728626+0000","last_clean":"2024-04-23T19:10:47.728626+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:10:47.728626+0000","last_undegraded":"2024-04-23T19:10:47.728626+0000","last_fullsized":"2024-04-23T19:10:47.728626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:36.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726094+0000","last_change":"2024-04-23T19:10:11.488952+0000","last_active":"2024-04-23T19:10:47.726094+0000","last_peered":"2024-04-23T19:10:47.726094+0000","last_clean":"2024-04-23T19:10:47.726094+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:10:47.726094+0000","last_undegraded":"2024-04-23T19:10:47.726094+0000","last_fullsized":"2024-04-23T19:10:47.726094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:22:09.450816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'15","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.702102+0000","last_change":"2024-04-23T19:10:13.497890+0000","last_active":"2024-04-23T19:10:49.702102+0000","last_peered":"2024-04-23T19:10:49.702102+0000","last_clean":"2024-04-23T19:10:49.702102+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:10:49.702102+0000","last_undegraded":"2024-04-23T19:10:49.702102+0000","last_fullsized":"2024-04-23T19:10:49.702102+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:21.612895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'282","reported_seq":355,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.575687+0000","last_change":"2024-04-23T19:10:15.502267+0000","last_active":"2024-04-23T19:13:16.575687+0000","last_peered":"2024-04-23T19:13:16.575687+0000","last_clean":"2024-04-23T19:13:16.575687+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:16.575687+0000","last_undegraded":"2024-04-23T19:13:16.575687+0000","last_fullsized":"2024-04-23T19:13:16.575687+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.147752+0000","last_change":"2024-04-23T19:10:17.513890+0000","last_active":"2024-04-23T19:11:32.147752+0000","last_peered":"2024-04-23T19:11:32.147752+0000","last_clean":"2024-04-23T19:11:32.147752+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:11:32.147752+0000","last_undegraded":"2024-04-23T19:11:32.147752+0000","last_fullsized":"2024-04-23T19:11:32.147752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:45.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482692+0000","last_change":"2024-04-23T19:10:09.496860+0000","last_active":"2024-04-23T19:13:16.482692+0000","last_peered":"2024-04-23T19:13:16.482692+0000","last_clean":"2024-04-23T19:13:16.482692+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:13:16.482692+0000","last_undegraded":"2024-04-23T19:13:16.482692+0000","last_fullsized":"2024-04-23T19:13:16.482692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:58.945396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729332+0000","last_change":"2024-04-23T19:10:07.493532+0000","last_active":"2024-04-23T19:10:47.729332+0000","last_peered":"2024-04-23T19:10:47.729332+0000","last_clean":"2024-04-23T19:10:47.729332+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:10:47.729332+0000","last_undegraded":"2024-04-23T19:10:47.729332+0000","last_fullsized":"2024-04-23T19:10:47.729332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:03.936999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729461+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:10:47.729461+0000","last_peered":"2024-04-23T19:10:47.729461+0000","last_clean":"2024-04-23T19:10:47.729461+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:10:47.729461+0000","last_undegraded":"2024-04-23T19:10:47.729461+0000","last_fullsized":"2024-04-23T19:10:47.729461+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:13.817485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728459+0000","last_change":"2024-04-23T19:10:11.487496+0000","last_active":"2024-04-23T19:10:47.728459+0000","last_peered":"2024-04-23T19:10:47.728459+0000","last_clean":"2024-04-23T19:10:47.728459+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:10:47.728459+0000","last_undegraded":"2024-04-23T19:10:47.728459+0000","last_fullsized":"2024-04-23T19:10:47.728459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:32:12.234867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"30'11","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728000+0000","last_change":"2024-04-23T19:10:13.503998+0000","last_active":"2024-04-23T19:10:47.728000+0000","last_peered":"2024-04-23T19:10:47.728000+0000","last_clean":"2024-04-23T19:10:47.728000+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:10:47.728000+0000","last_undegraded":"2024-04-23T19:10:47.728000+0000","last_fullsized":"2024-04-23T19:10:47.728000+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:33.974553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'285","reported_seq":432,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.027190+0000","last_change":"2024-04-23T19:10:15.509417+0000","last_active":"2024-04-23T19:13:23.027190+0000","last_peered":"2024-04-23T19:13:23.027190+0000","last_clean":"2024-04-23T19:13:23.027190+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:23.027190+0000","last_undegraded":"2024-04-23T19:13:23.027190+0000","last_fullsized":"2024-04-23T19:13:23.027190+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:29.634429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718167+0000","last_change":"2024-04-23T19:10:17.516808+0000","last_active":"2024-04-23T19:10:47.718167+0000","last_peered":"2024-04-23T19:10:47.718167+0000","last_clean":"2024-04-23T19:10:47.718167+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:10:47.718167+0000","last_undegraded":"2024-04-23T19:10:47.718167+0000","last_fullsized":"2024-04-23T19:10:47.718167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:41.513557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.449088+0000","last_change":"2024-04-23T19:10:09.493717+0000","last_active":"2024-04-23T19:13:16.449088+0000","last_peered":"2024-04-23T19:13:16.449088+0000","last_clean":"2024-04-23T19:13:16.449088+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:16.449088+0000","last_undegraded":"2024-04-23T19:13:16.449088+0000","last_fullsized":"2024-04-23T19:13:16.449088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:44.856835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.749316+0000","last_change":"2024-04-23T19:10:07.500858+0000","last_active":"2024-04-23T19:12:15.749316+0000","last_peered":"2024-04-23T19:12:15.749316+0000","last_clean":"2024-04-23T19:12:15.749316+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:12:15.749316+0000","last_undegraded":"2024-04-23T19:12:15.749316+0000","last_fullsized":"2024-04-23T19:12:15.749316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:16.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728818+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:10:47.728818+0000","last_peered":"2024-04-23T19:10:47.728818+0000","last_clean":"2024-04-23T19:10:47.728818+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:10:47.728818+0000","last_undegraded":"2024-04-23T19:10:47.728818+0000","last_fullsized":"2024-04-23T19:10:47.728818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:06.025384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727223+0000","last_change":"2024-04-23T19:10:11.489138+0000","last_active":"2024-04-23T19:10:47.727223+0000","last_peered":"2024-04-23T19:10:47.727223+0000","last_clean":"2024-04-23T19:10:47.727223+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:10:47.727223+0000","last_undegraded":"2024-04-23T19:10:47.727223+0000","last_fullsized":"2024-04-23T19:10:47.727223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:19.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'93","reported_seq":177,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.180394+0000","last_change":"2024-04-23T19:10:13.503368+0000","last_active":"2024-04-23T19:13:23.180394+0000","last_peered":"2024-04-23T19:13:23.180394+0000","last_clean":"2024-04-23T19:13:23.180394+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:13:23.180394+0000","last_undegraded":"2024-04-23T19:13:23.180394+0000","last_fullsized":"2024-04-23T19:13:23.180394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:31.109448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4316,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":98,"num_read_kb":92,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727890+0000","last_change":"2024-04-23T19:10:15.500592+0000","last_active":"2024-04-23T19:10:47.727890+0000","last_peered":"2024-04-23T19:10:47.727890+0000","last_clean":"2024-04-23T19:10:47.727890+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:10:47.727890+0000","last_undegraded":"2024-04-23T19:10:47.727890+0000","last_fullsized":"2024-04-23T19:10:47.727890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:11.115258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722791+0000","last_change":"2024-04-23T19:10:17.515628+0000","last_active":"2024-04-23T19:10:47.722791+0000","last_peered":"2024-04-23T19:10:47.722791+0000","last_clean":"2024-04-23T19:10:47.722791+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:10:47.722791+0000","last_undegraded":"2024-04-23T19:10:47.722791+0000","last_fullsized":"2024-04-23T19:10:47.722791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.641905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722100+0000","last_change":"2024-04-23T19:10:17.515232+0000","last_active":"2024-04-23T19:10:47.722100+0000","last_peered":"2024-04-23T19:10:47.722100+0000","last_clean":"2024-04-23T19:10:47.722100+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:10:47.722100+0000","last_undegraded":"2024-04-23T19:10:47.722100+0000","last_fullsized":"2024-04-23T19:10:47.722100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:40.692521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720221+0000","last_change":"2024-04-23T19:10:09.492213+0000","last_active":"2024-04-23T19:10:47.720221+0000","last_peered":"2024-04-23T19:10:47.720221+0000","last_clean":"2024-04-23T19:10:47.720221+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:10:47.720221+0000","last_undegraded":"2024-04-23T19:10:47.720221+0000","last_fullsized":"2024-04-23T19:10:47.720221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:54.477619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726467+0000","last_change":"2024-04-23T19:10:07.500107+0000","last_active":"2024-04-23T19:10:47.726467+0000","last_peered":"2024-04-23T19:10:47.726467+0000","last_clean":"2024-04-23T19:10:47.726467+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:10:47.726467+0000","last_undegraded":"2024-04-23T19:10:47.726467+0000","last_fullsized":"2024-04-23T19:10:47.726467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:14.278078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727809+0000","last_change":"2024-04-23T19:10:47.725041+0000","last_active":"2024-04-23T19:10:47.727809+0000","last_peered":"2024-04-23T19:10:47.727809+0000","last_clean":"2024-04-23T19:10:47.727809+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:10:47.727809+0000","last_undegraded":"2024-04-23T19:10:47.727809+0000","last_fullsized":"2024-04-23T19:10:47.727809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:17.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'32","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.317498+0000","last_change":"2024-04-23T19:10:13.497156+0000","last_active":"2024-04-23T19:12:14.317498+0000","last_peered":"2024-04-23T19:12:14.317498+0000","last_clean":"2024-04-23T19:12:14.317498+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:12:14.317498+0000","last_undegraded":"2024-04-23T19:12:14.317498+0000","last_fullsized":"2024-04-23T19:12:14.317498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:49.136472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726980+0000","last_change":"2024-04-23T19:10:15.514289+0000","last_active":"2024-04-23T19:10:47.726980+0000","last_peered":"2024-04-23T19:10:47.726980+0000","last_clean":"2024-04-23T19:10:47.726980+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:10:47.726980+0000","last_undegraded":"2024-04-23T19:10:47.726980+0000","last_fullsized":"2024-04-23T19:10:47.726980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:26:12.517542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.023661+0000","last_change":"2024-04-23T19:10:17.514011+0000","last_active":"2024-04-23T19:12:14.023661+0000","last_peered":"2024-04-23T19:12:14.023661+0000","last_clean":"2024-04-23T19:12:14.023661+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:12:14.023661+0000","last_undegraded":"2024-04-23T19:12:14.023661+0000","last_fullsized":"2024-04-23T19:12:14.023661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:55:22.164549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.733183+0000","last_change":"2024-04-23T19:10:47.732302+0000","last_active":"2024-04-23T19:10:47.733183+0000","last_peered":"2024-04-23T19:10:47.733183+0000","last_clean":"2024-04-23T19:10:47.733183+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:10:47.733183+0000","last_undegraded":"2024-04-23T19:10:47.733183+0000","last_fullsized":"2024-04-23T19:10:47.733183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:05.759643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.139371+0000","last_change":"2024-04-23T19:10:07.493914+0000","last_active":"2024-04-23T19:13:16.139371+0000","last_peered":"2024-04-23T19:13:16.139371+0000","last_clean":"2024-04-23T19:13:16.139371+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:13:16.139371+0000","last_undegraded":"2024-04-23T19:13:16.139371+0000","last_fullsized":"2024-04-23T19:13:16.139371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:01.102865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729231+0000","last_change":"2024-04-23T19:10:11.487861+0000","last_active":"2024-04-23T19:10:47.729231+0000","last_peered":"2024-04-23T19:10:47.729231+0000","last_clean":"2024-04-23T19:10:47.729231+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:10:47.729231+0000","last_undegraded":"2024-04-23T19:10:47.729231+0000","last_fullsized":"2024-04-23T19:10:47.729231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:47.278160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'5","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789104+0000","last_change":"2024-04-23T19:10:13.494591+0000","last_active":"2024-04-23T19:13:15.789104+0000","last_peered":"2024-04-23T19:13:15.789104+0000","last_clean":"2024-04-23T19:13:15.789104+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:15.789104+0000","last_undegraded":"2024-04-23T19:13:15.789104+0000","last_fullsized":"2024-04-23T19:13:15.789104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:58.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718895+0000","last_change":"2024-04-23T19:10:15.513825+0000","last_active":"2024-04-23T19:10:47.718895+0000","last_peered":"2024-04-23T19:10:47.718895+0000","last_clean":"2024-04-23T19:10:47.718895+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:10:47.718895+0000","last_undegraded":"2024-04-23T19:10:47.718895+0000","last_fullsized":"2024-04-23T19:10:47.718895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728340+0000","last_change":"2024-04-23T19:10:17.520788+0000","last_active":"2024-04-23T19:10:47.728340+0000","last_peered":"2024-04-23T19:10:47.728340+0000","last_clean":"2024-04-23T19:10:47.728340+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:10:47.728340+0000","last_undegraded":"2024-04-23T19:10:47.728340+0000","last_fullsized":"2024-04-23T19:10:47.728340+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:36.438306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399875+0000","last_change":"2024-04-23T19:10:09.491571+0000","last_active":"2024-04-23T19:13:16.399875+0000","last_peered":"2024-04-23T19:13:16.399875+0000","last_clean":"2024-04-23T19:13:16.399875+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:16.399875+0000","last_undegraded":"2024-04-23T19:13:16.399875+0000","last_fullsized":"2024-04-23T19:13:16.399875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724424+0000","last_change":"2024-04-23T19:10:07.500769+0000","last_active":"2024-04-23T19:10:47.724424+0000","last_peered":"2024-04-23T19:10:47.724424+0000","last_clean":"2024-04-23T19:10:47.724424+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:10:47.724424+0000","last_undegraded":"2024-04-23T19:10:47.724424+0000","last_fullsized":"2024-04-23T19:10:47.724424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718451+0000","last_change":"2024-04-23T19:10:11.493115+0000","last_active":"2024-04-23T19:10:47.718451+0000","last_peered":"2024-04-23T19:10:47.718451+0000","last_clean":"2024-04-23T19:10:47.718451+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:10:47.718451+0000","last_undegraded":"2024-04-23T19:10:47.718451+0000","last_fullsized":"2024-04-23T19:10:47.718451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:50:51.288167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'43","reported_seq":106,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.085246+0000","last_change":"2024-04-23T19:10:13.491151+0000","last_active":"2024-04-23T19:13:23.085246+0000","last_peered":"2024-04-23T19:13:23.085246+0000","last_clean":"2024-04-23T19:13:23.085246+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:23.085246+0000","last_undegraded":"2024-04-23T19:13:23.085246+0000","last_fullsized":"2024-04-23T19:13:23.085246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:56.656293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717052+0000","last_change":"2024-04-23T19:10:15.497695+0000","last_active":"2024-04-23T19:10:47.717052+0000","last_peered":"2024-04-23T19:10:47.717052+0000","last_clean":"2024-04-23T19:10:47.717052+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:10:47.717052+0000","last_undegraded":"2024-04-23T19:10:47.717052+0000","last_fullsized":"2024-04-23T19:10:47.717052+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:40.709841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.164233+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:10:50.164233+0000","last_peered":"2024-04-23T19:10:50.164233+0000","last_clean":"2024-04-23T19:10:50.164233+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:10:50.164233+0000","last_undegraded":"2024-04-23T19:10:50.164233+0000","last_fullsized":"2024-04-23T19:10:50.164233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:38:04.095526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'9","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165987+0000","last_change":"2024-04-23T19:10:09.498393+0000","last_active":"2024-04-23T19:13:16.165987+0000","last_peered":"2024-04-23T19:13:16.165987+0000","last_clean":"2024-04-23T19:13:16.165987+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:13:16.165987+0000","last_undegraded":"2024-04-23T19:13:16.165987+0000","last_fullsized":"2024-04-23T19:13:16.165987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:24.946389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722073+0000","last_change":"2024-04-23T19:10:07.496451+0000","last_active":"2024-04-23T19:10:47.722073+0000","last_peered":"2024-04-23T19:10:47.722073+0000","last_clean":"2024-04-23T19:10:47.722073+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:10:47.722073+0000","last_undegraded":"2024-04-23T19:10:47.722073+0000","last_fullsized":"2024-04-23T19:10:47.722073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:14.868993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.724117+0000","last_change":"2024-04-23T19:10:11.490728+0000","last_active":"2024-04-23T19:10:47.724117+0000","last_peered":"2024-04-23T19:10:47.724117+0000","last_clean":"2024-04-23T19:10:47.724117+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:10:47.724117+0000","last_undegraded":"2024-04-23T19:10:47.724117+0000","last_fullsized":"2024-04-23T19:10:47.724117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:55.751511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'35","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.075126+0000","last_change":"2024-04-23T19:10:13.493330+0000","last_active":"2024-04-23T19:13:23.075126+0000","last_peered":"2024-04-23T19:13:23.075126+0000","last_clean":"2024-04-23T19:13:23.075126+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:13:23.075126+0000","last_undegraded":"2024-04-23T19:13:23.075126+0000","last_fullsized":"2024-04-23T19:13:23.075126+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:48.481253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":53,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719061+0000","last_change":"2024-04-23T19:10:15.508565+0000","last_active":"2024-04-23T19:10:47.719061+0000","last_peered":"2024-04-23T19:10:47.719061+0000","last_clean":"2024-04-23T19:10:47.719061+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:10:47.719061+0000","last_undegraded":"2024-04-23T19:10:47.719061+0000","last_fullsized":"2024-04-23T19:10:47.719061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:02.695164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703848+0000","last_change":"2024-04-23T19:10:47.728408+0000","last_active":"2024-04-23T19:10:50.703848+0000","last_peered":"2024-04-23T19:10:50.703848+0000","last_clean":"2024-04-23T19:10:50.703848+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:10:50.703848+0000","last_undegraded":"2024-04-23T19:10:50.703848+0000","last_fullsized":"2024-04-23T19:10:50.703848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:36.054291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.720730+0000","last_change":"2024-04-23T19:10:07.503907+0000","last_active":"2024-04-23T19:12:15.720730+0000","last_peered":"2024-04-23T19:12:15.720730+0000","last_clean":"2024-04-23T19:12:15.720730+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:12:15.720730+0000","last_undegraded":"2024-04-23T19:12:15.720730+0000","last_fullsized":"2024-04-23T19:12:15.720730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:04:52.469809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.730560+0000","last_change":"2024-04-23T19:10:11.490872+0000","last_active":"2024-04-23T19:10:47.730560+0000","last_peered":"2024-04-23T19:10:47.730560+0000","last_clean":"2024-04-23T19:10:47.730560+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:10:47.730560+0000","last_undegraded":"2024-04-23T19:10:47.730560+0000","last_fullsized":"2024-04-23T19:10:47.730560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'60","reported_seq":124,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.153793+0000","last_change":"2024-04-23T19:10:13.494268+0000","last_active":"2024-04-23T19:13:23.153793+0000","last_peered":"2024-04-23T19:13:23.153793+0000","last_clean":"2024-04-23T19:13:23.153793+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:13:23.153793+0000","last_undegraded":"2024-04-23T19:13:23.153793+0000","last_fullsized":"2024-04-23T19:13:23.153793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:00.688062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719088+0000","last_change":"2024-04-23T19:10:15.505820+0000","last_active":"2024-04-23T19:10:47.719088+0000","last_peered":"2024-04-23T19:10:47.719088+0000","last_clean":"2024-04-23T19:10:47.719088+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:10:47.719088+0000","last_undegraded":"2024-04-23T19:10:47.719088+0000","last_fullsized":"2024-04-23T19:10:47.719088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:23.619996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.704248+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:15.704248+0000","last_peered":"2024-04-23T19:12:15.704248+0000","last_clean":"2024-04-23T19:12:15.704248+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:12:15.704248+0000","last_undegraded":"2024-04-23T19:12:15.704248+0000","last_fullsized":"2024-04-23T19:12:15.704248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:58.312384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482427+0000","last_change":"2024-04-23T19:10:09.482621+0000","last_active":"2024-04-23T19:13:16.482427+0000","last_peered":"2024-04-23T19:13:16.482427+0000","last_clean":"2024-04-23T19:13:16.482427+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:13:16.482427+0000","last_undegraded":"2024-04-23T19:13:16.482427+0000","last_fullsized":"2024-04-23T19:13:16.482427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720227+0000","last_change":"2024-04-23T19:10:07.493942+0000","last_active":"2024-04-23T19:12:31.720227+0000","last_peered":"2024-04-23T19:12:31.720227+0000","last_clean":"2024-04-23T19:12:31.720227+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:12:31.720227+0000","last_undegraded":"2024-04-23T19:12:31.720227+0000","last_fullsized":"2024-04-23T19:12:31.720227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:21.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.721926+0000","last_change":"2024-04-23T19:10:11.486920+0000","last_active":"2024-04-23T19:10:47.721926+0000","last_peered":"2024-04-23T19:10:47.721926+0000","last_clean":"2024-04-23T19:10:47.721926+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:10:47.721926+0000","last_undegraded":"2024-04-23T19:10:47.721926+0000","last_fullsized":"2024-04-23T19:10:47.721926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:56.355675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'36","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.303836+0000","last_change":"2024-04-23T19:10:13.502855+0000","last_active":"2024-04-23T19:12:14.303836+0000","last_peered":"2024-04-23T19:12:14.303836+0000","last_clean":"2024-04-23T19:12:14.303836+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:12:14.303836+0000","last_undegraded":"2024-04-23T19:12:14.303836+0000","last_fullsized":"2024-04-23T19:12:14.303836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:38:09.770521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717958+0000","last_change":"2024-04-23T19:10:15.510094+0000","last_active":"2024-04-23T19:10:47.717958+0000","last_peered":"2024-04-23T19:10:47.717958+0000","last_clean":"2024-04-23T19:10:47.717958+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:10:47.717958+0000","last_undegraded":"2024-04-23T19:10:47.717958+0000","last_fullsized":"2024-04-23T19:10:47.717958+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:53.627087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.609469+0000","last_change":"2024-04-23T19:10:17.517391+0000","last_active":"2024-04-23T19:12:12.609469+0000","last_peered":"2024-04-23T19:12:12.609469+0000","last_clean":"2024-04-23T19:12:12.609469+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:12:12.609469+0000","last_undegraded":"2024-04-23T19:12:12.609469+0000","last_fullsized":"2024-04-23T19:12:12.609469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:35:39.248022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:50.703799+0000","last_change":"2024-04-23T19:10:47.733611+0000","last_active":"2024-04-23T19:10:50.703799+0000","last_peered":"2024-04-23T19:10:50.703799+0000","last_clean":"2024-04-23T19:10:50.703799+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:10:50.703799+0000","last_undegraded":"2024-04-23T19:10:50.703799+0000","last_fullsized":"2024-04-23T19:10:50.703799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:04.349258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717131+0000","last_change":"2024-04-23T19:10:07.502477+0000","last_active":"2024-04-23T19:10:47.717131+0000","last_peered":"2024-04-23T19:10:47.717131+0000","last_clean":"2024-04-23T19:10:47.717131+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:10:47.717131+0000","last_undegraded":"2024-04-23T19:10:47.717131+0000","last_fullsized":"2024-04-23T19:10:47.717131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:01:05.965128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719460+0000","last_change":"2024-04-23T19:10:11.487445+0000","last_active":"2024-04-23T19:10:47.719460+0000","last_peered":"2024-04-23T19:10:47.719460+0000","last_clean":"2024-04-23T19:10:47.719460+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:10:47.719460+0000","last_undegraded":"2024-04-23T19:10:47.719460+0000","last_fullsized":"2024-04-23T19:10:47.719460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:33:56.771240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'35","reported_seq":82,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.146745+0000","last_change":"2024-04-23T19:10:13.498101+0000","last_active":"2024-04-23T19:13:23.146745+0000","last_peered":"2024-04-23T19:13:23.146745+0000","last_clean":"2024-04-23T19:13:23.146745+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:13:23.146745+0000","last_undegraded":"2024-04-23T19:13:23.146745+0000","last_fullsized":"2024-04-23T19:13:23.146745+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729093+0000","last_change":"2024-04-23T19:10:15.504279+0000","last_active":"2024-04-23T19:10:47.729093+0000","last_peered":"2024-04-23T19:10:47.729093+0000","last_clean":"2024-04-23T19:10:47.729093+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:10:47.729093+0000","last_undegraded":"2024-04-23T19:10:47.729093+0000","last_fullsized":"2024-04-23T19:10:47.729093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:05.168473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.451981+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.451981+0000","last_peered":"2024-04-23T19:13:16.451981+0000","last_clean":"2024-04-23T19:13:16.451981+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.451981+0000","last_undegraded":"2024-04-23T19:13:16.451981+0000","last_fullsized":"2024-04-23T19:13:16.451981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:52.410520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399955+0000","last_change":"2024-04-23T19:10:09.477657+0000","last_active":"2024-04-23T19:13:16.399955+0000","last_peered":"2024-04-23T19:13:16.399955+0000","last_clean":"2024-04-23T19:13:16.399955+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:16.399955+0000","last_undegraded":"2024-04-23T19:13:16.399955+0000","last_fullsized":"2024-04-23T19:13:16.399955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.117950+0000","last_change":"2024-04-23T19:10:07.504027+0000","last_active":"2024-04-23T19:13:16.117950+0000","last_peered":"2024-04-23T19:13:16.117950+0000","last_clean":"2024-04-23T19:13:16.117950+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:13:16.117950+0000","last_undegraded":"2024-04-23T19:13:16.117950+0000","last_fullsized":"2024-04-23T19:13:16.117950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:34.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'4","reported_seq":226,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.374959+0000","last_change":"2024-04-23T19:10:11.485789+0000","last_active":"2024-04-23T19:12:15.374959+0000","last_peered":"2024-04-23T19:12:15.374959+0000","last_clean":"2024-04-23T19:12:15.374959+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:12:15.374959+0000","last_undegraded":"2024-04-23T19:12:15.374959+0000","last_fullsized":"2024-04-23T19:12:15.374959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:54.297420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"30'10","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728532+0000","last_change":"2024-04-23T19:10:13.502626+0000","last_active":"2024-04-23T19:10:47.728532+0000","last_peered":"2024-04-23T19:10:47.728532+0000","last_clean":"2024-04-23T19:10:47.728532+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:10:47.728532+0000","last_undegraded":"2024-04-23T19:10:47.728532+0000","last_fullsized":"2024-04-23T19:10:47.728532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:27.703823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'285","reported_seq":420,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.020758+0000","last_change":"2024-04-23T19:10:15.510611+0000","last_active":"2024-04-23T19:13:23.020758+0000","last_peered":"2024-04-23T19:13:23.020758+0000","last_clean":"2024-04-23T19:13:23.020758+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:23.020758+0000","last_undegraded":"2024-04-23T19:13:23.020758+0000","last_fullsized":"2024-04-23T19:13:23.020758+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:00.157929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725003+0000","last_change":"2024-04-23T19:10:17.513830+0000","last_active":"2024-04-23T19:10:47.725003+0000","last_peered":"2024-04-23T19:10:47.725003+0000","last_clean":"2024-04-23T19:10:47.725003+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:10:47.725003+0000","last_undegraded":"2024-04-23T19:10:47.725003+0000","last_fullsized":"2024-04-23T19:10:47.725003+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:34.594849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:32.150535+0000","last_change":"2024-04-23T19:10:09.476601+0000","last_active":"2024-04-23T19:11:32.150535+0000","last_peered":"2024-04-23T19:11:32.150535+0000","last_clean":"2024-04-23T19:11:32.150535+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:11:32.150535+0000","last_undegraded":"2024-04-23T19:11:32.150535+0000","last_fullsized":"2024-04-23T19:11:32.150535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:10.224761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.718929+0000","last_change":"2024-04-23T19:10:07.497992+0000","last_active":"2024-04-23T19:10:47.718929+0000","last_peered":"2024-04-23T19:10:47.718929+0000","last_clean":"2024-04-23T19:10:47.718929+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:10:47.718929+0000","last_undegraded":"2024-04-23T19:10:47.718929+0000","last_fullsized":"2024-04-23T19:10:47.718929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720669+0000","last_change":"2024-04-23T19:10:11.489334+0000","last_active":"2024-04-23T19:10:47.720669+0000","last_peered":"2024-04-23T19:10:47.720669+0000","last_clean":"2024-04-23T19:10:47.720669+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:10:47.720669+0000","last_undegraded":"2024-04-23T19:10:47.720669+0000","last_fullsized":"2024-04-23T19:10:47.720669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:29.102760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'37","reported_seq":82,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.294646+0000","last_change":"2024-04-23T19:10:13.492238+0000","last_active":"2024-04-23T19:12:14.294646+0000","last_peered":"2024-04-23T19:12:14.294646+0000","last_clean":"2024-04-23T19:12:14.294646+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:12:14.294646+0000","last_undegraded":"2024-04-23T19:12:14.294646+0000","last_fullsized":"2024-04-23T19:12:14.294646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:24:40.768899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726624+0000","last_change":"2024-04-23T19:10:15.502657+0000","last_active":"2024-04-23T19:10:47.726624+0000","last_peered":"2024-04-23T19:10:47.726624+0000","last_clean":"2024-04-23T19:10:47.726624+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:10:47.726624+0000","last_undegraded":"2024-04-23T19:10:47.726624+0000","last_fullsized":"2024-04-23T19:10:47.726624+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:04:18.411434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.722842+0000","last_change":"2024-04-23T19:10:17.518482+0000","last_active":"2024-04-23T19:10:47.722842+0000","last_peered":"2024-04-23T19:10:47.722842+0000","last_clean":"2024-04-23T19:10:47.722842+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:10:47.722842+0000","last_undegraded":"2024-04-23T19:10:47.722842+0000","last_fullsized":"2024-04-23T19:10:47.722842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:16.031706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'3","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.405706+0000","last_change":"2024-04-23T19:10:09.477644+0000","last_active":"2024-04-23T19:13:16.405706+0000","last_peered":"2024-04-23T19:13:16.405706+0000","last_clean":"2024-04-23T19:13:16.405706+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:16.405706+0000","last_undegraded":"2024-04-23T19:13:16.405706+0000","last_fullsized":"2024-04-23T19:13:16.405706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:16.285889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728885+0000","last_change":"2024-04-23T19:10:07.503802+0000","last_active":"2024-04-23T19:10:47.728885+0000","last_peered":"2024-04-23T19:10:47.728885+0000","last_clean":"2024-04-23T19:10:47.728885+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:10:47.728885+0000","last_undegraded":"2024-04-23T19:10:47.728885+0000","last_fullsized":"2024-04-23T19:10:47.728885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:26:03.446904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720608+0000","last_change":"2024-04-23T19:10:11.489940+0000","last_active":"2024-04-23T19:10:47.720608+0000","last_peered":"2024-04-23T19:10:47.720608+0000","last_clean":"2024-04-23T19:10:47.720608+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:10:47.720608+0000","last_undegraded":"2024-04-23T19:10:47.720608+0000","last_fullsized":"2024-04-23T19:10:47.720608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:54:35.423825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'34","reported_seq":80,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.171028+0000","last_change":"2024-04-23T19:10:13.505761+0000","last_active":"2024-04-23T19:13:23.171028+0000","last_peered":"2024-04-23T19:13:23.171028+0000","last_clean":"2024-04-23T19:13:23.171028+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:13:23.171028+0000","last_undegraded":"2024-04-23T19:13:23.171028+0000","last_fullsized":"2024-04-23T19:13:23.171028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:28.957267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.723783+0000","last_change":"2024-04-23T19:10:15.514687+0000","last_active":"2024-04-23T19:10:47.723783+0000","last_peered":"2024-04-23T19:10:47.723783+0000","last_clean":"2024-04-23T19:10:47.723783+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:10:47.723783+0000","last_undegraded":"2024-04-23T19:10:47.723783+0000","last_fullsized":"2024-04-23T19:10:47.723783+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:42:29.022733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.768829+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:15.768829+0000","last_peered":"2024-04-23T19:12:15.768829+0000","last_clean":"2024-04-23T19:12:15.768829+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:15.768829+0000","last_undegraded":"2024-04-23T19:12:15.768829+0000","last_fullsized":"2024-04-23T19:12:15.768829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:42.383976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'11","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165914+0000","last_change":"2024-04-23T19:10:09.490516+0000","last_active":"2024-04-23T19:13:16.165914+0000","last_peered":"2024-04-23T19:13:16.165914+0000","last_clean":"2024-04-23T19:13:16.165914+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:13:16.165914+0000","last_undegraded":"2024-04-23T19:13:16.165914+0000","last_fullsized":"2024-04-23T19:13:16.165914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:13:54.586490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729712+0000","last_change":"2024-04-23T19:10:07.486770+0000","last_active":"2024-04-23T19:10:47.729712+0000","last_peered":"2024-04-23T19:10:47.729712+0000","last_clean":"2024-04-23T19:10:47.729712+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:10:47.729712+0000","last_undegraded":"2024-04-23T19:10:47.729712+0000","last_fullsized":"2024-04-23T19:10:47.729712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:56:06.039657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717835+0000","last_change":"2024-04-23T19:10:11.491433+0000","last_active":"2024-04-23T19:10:47.717835+0000","last_peered":"2024-04-23T19:10:47.717835+0000","last_clean":"2024-04-23T19:10:47.717835+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:10:47.717835+0000","last_undegraded":"2024-04-23T19:10:47.717835+0000","last_fullsized":"2024-04-23T19:10:47.717835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:30.740834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'26","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.319314+0000","last_change":"2024-04-23T19:10:13.503113+0000","last_active":"2024-04-23T19:12:14.319314+0000","last_peered":"2024-04-23T19:12:14.319314+0000","last_clean":"2024-04-23T19:12:14.319314+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:12:14.319314+0000","last_undegraded":"2024-04-23T19:12:14.319314+0000","last_fullsized":"2024-04-23T19:12:14.319314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:25.616871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726203+0000","last_change":"2024-04-23T19:10:15.512937+0000","last_active":"2024-04-23T19:10:47.726203+0000","last_peered":"2024-04-23T19:10:47.726203+0000","last_clean":"2024-04-23T19:10:47.726203+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:10:47.726203+0000","last_undegraded":"2024-04-23T19:10:47.726203+0000","last_fullsized":"2024-04-23T19:10:47.726203+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:07.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:14.329498+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:14.329498+0000","last_peered":"2024-04-23T19:12:14.329498+0000","last_clean":"2024-04-23T19:12:14.329498+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:14.329498+0000","last_undegraded":"2024-04-23T19:12:14.329498+0000","last_fullsized":"2024-04-23T19:12:14.329498+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:30.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.482420+0000","last_change":"2024-04-23T19:10:09.476153+0000","last_active":"2024-04-23T19:13:16.482420+0000","last_peered":"2024-04-23T19:13:16.482420+0000","last_clean":"2024-04-23T19:13:16.482420+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:16.482420+0000","last_undegraded":"2024-04-23T19:13:16.482420+0000","last_fullsized":"2024-04-23T19:13:16.482420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:07.680114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'1","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.716957+0000","last_change":"2024-04-23T19:10:07.501211+0000","last_active":"2024-04-23T19:12:11.716957+0000","last_peered":"2024-04-23T19:12:11.716957+0000","last_clean":"2024-04-23T19:12:11.716957+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:12:11.716957+0000","last_undegraded":"2024-04-23T19:12:11.716957+0000","last_fullsized":"2024-04-23T19:12:11.716957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'2","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:12.979180+0000","last_change":"2024-04-23T19:10:11.491220+0000","last_active":"2024-04-23T19:12:12.979180+0000","last_peered":"2024-04-23T19:12:12.979180+0000","last_clean":"2024-04-23T19:12:12.979180+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:12:12.979180+0000","last_undegraded":"2024-04-23T19:12:12.979180+0000","last_fullsized":"2024-04-23T19:12:12.979180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:49:56.767918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'32","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:15.789943+0000","last_change":"2024-04-23T19:10:13.502743+0000","last_active":"2024-04-23T19:13:15.789943+0000","last_peered":"2024-04-23T19:13:15.789943+0000","last_clean":"2024-04-23T19:13:15.789943+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:15.789943+0000","last_undegraded":"2024-04-23T19:13:15.789943+0000","last_fullsized":"2024-04-23T19:13:15.789943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:13.230238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.727977+0000","last_change":"2024-04-23T19:10:15.502314+0000","last_active":"2024-04-23T19:10:47.727977+0000","last_peered":"2024-04-23T19:10:47.727977+0000","last_clean":"2024-04-23T19:10:47.727977+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:10:47.727977+0000","last_undegraded":"2024-04-23T19:10:47.727977+0000","last_fullsized":"2024-04-23T19:10:47.727977+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:40:26.867666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.434484+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.434484+0000","last_peered":"2024-04-23T19:13:16.434484+0000","last_clean":"2024-04-23T19:13:16.434484+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.434484+0000","last_undegraded":"2024-04-23T19:13:16.434484+0000","last_fullsized":"2024-04-23T19:13:16.434484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:47.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489162+0000","last_change":"2024-04-23T19:10:09.494269+0000","last_active":"2024-04-23T19:13:16.489162+0000","last_peered":"2024-04-23T19:13:16.489162+0000","last_clean":"2024-04-23T19:13:16.489162+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:16.489162+0000","last_undegraded":"2024-04-23T19:13:16.489162+0000","last_fullsized":"2024-04-23T19:13:16.489162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:25.844962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.477833+0000","last_change":"2024-04-23T19:10:07.492825+0000","last_active":"2024-04-23T19:13:16.477833+0000","last_peered":"2024-04-23T19:13:16.477833+0000","last_clean":"2024-04-23T19:13:16.477833+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:13:16.477833+0000","last_undegraded":"2024-04-23T19:13:16.477833+0000","last_fullsized":"2024-04-23T19:13:16.477833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:33:53.651883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.771507+0000","last_change":"2024-04-23T19:10:09.478781+0000","last_active":"2024-04-23T19:12:15.771507+0000","last_peered":"2024-04-23T19:12:15.771507+0000","last_clean":"2024-04-23T19:12:15.771507+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:12:15.771507+0000","last_undegraded":"2024-04-23T19:12:15.771507+0000","last_fullsized":"2024-04-23T19:12:15.771507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:38.645694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.079419+0000","last_change":"2024-04-23T19:10:07.489213+0000","last_active":"2024-04-23T19:13:16.079419+0000","last_peered":"2024-04-23T19:13:16.079419+0000","last_clean":"2024-04-23T19:13:16.079419+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:13:16.079419+0000","last_undegraded":"2024-04-23T19:13:16.079419+0000","last_fullsized":"2024-04-23T19:13:16.079419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:13:34.167425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258191+0000","last_change":"2024-04-23T19:10:09.495393+0000","last_active":"2024-04-23T19:13:16.258191+0000","last_peered":"2024-04-23T19:13:16.258191+0000","last_clean":"2024-04-23T19:13:16.258191+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:13:16.258191+0000","last_undegraded":"2024-04-23T19:13:16.258191+0000","last_fullsized":"2024-04-23T19:13:16.258191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:01:17.602017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726729+0000","last_change":"2024-04-23T19:10:07.506697+0000","last_active":"2024-04-23T19:10:47.726729+0000","last_peered":"2024-04-23T19:10:47.726729+0000","last_clean":"2024-04-23T19:10:47.726729+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:10:47.726729+0000","last_undegraded":"2024-04-23T19:10:47.726729+0000","last_fullsized":"2024-04-23T19:10:47.726729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:36.637883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.158375+0000","last_change":"2024-04-23T19:10:09.478697+0000","last_active":"2024-04-23T19:13:16.158375+0000","last_peered":"2024-04-23T19:13:16.158375+0000","last_clean":"2024-04-23T19:13:16.158375+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:13:16.158375+0000","last_undegraded":"2024-04-23T19:13:16.158375+0000","last_fullsized":"2024-04-23T19:13:16.158375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'4","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.060214+0000","last_change":"2024-04-23T19:10:07.490623+0000","last_active":"2024-04-23T19:13:16.060214+0000","last_peered":"2024-04-23T19:13:16.060214+0000","last_clean":"2024-04-23T19:13:16.060214+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:13:16.060214+0000","last_undegraded":"2024-04-23T19:13:16.060214+0000","last_fullsized":"2024-04-23T19:13:16.060214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:03.777822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698550+0000","last_change":"2024-04-23T19:10:09.491248+0000","last_active":"2024-04-23T19:10:49.698550+0000","last_peered":"2024-04-23T19:10:49.698550+0000","last_clean":"2024-04-23T19:10:49.698550+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:10:49.698550+0000","last_undegraded":"2024-04-23T19:10:49.698550+0000","last_fullsized":"2024-04-23T19:10:49.698550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:46.772962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729029+0000","last_change":"2024-04-23T19:10:07.490715+0000","last_active":"2024-04-23T19:10:47.729029+0000","last_peered":"2024-04-23T19:10:47.729029+0000","last_clean":"2024-04-23T19:10:47.729029+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:10:47.729029+0000","last_undegraded":"2024-04-23T19:10:47.729029+0000","last_fullsized":"2024-04-23T19:10:47.729029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:03.712798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.447089+0000","last_change":"2024-04-23T19:10:09.494954+0000","last_active":"2024-04-23T19:13:16.447089+0000","last_peered":"2024-04-23T19:13:16.447089+0000","last_clean":"2024-04-23T19:13:16.447089+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:16.447089+0000","last_undegraded":"2024-04-23T19:13:16.447089+0000","last_fullsized":"2024-04-23T19:13:16.447089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.351485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:48.548131+0000","last_change":"2024-04-23T19:10:07.490753+0000","last_active":"2024-04-23T19:10:48.548131+0000","last_peered":"2024-04-23T19:10:48.548131+0000","last_clean":"2024-04-23T19:10:48.548131+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:10:48.548131+0000","last_undegraded":"2024-04-23T19:10:48.548131+0000","last_fullsized":"2024-04-23T19:10:48.548131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.157844+0000","last_change":"2024-04-23T19:10:09.482692+0000","last_active":"2024-04-23T19:13:16.157844+0000","last_peered":"2024-04-23T19:13:16.157844+0000","last_clean":"2024-04-23T19:13:16.157844+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:13:16.157844+0000","last_undegraded":"2024-04-23T19:13:16.157844+0000","last_fullsized":"2024-04-23T19:13:16.157844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.380363+0000","last_change":"2024-04-23T19:10:07.500499+0000","last_active":"2024-04-23T19:13:16.380363+0000","last_peered":"2024-04-23T19:13:16.380363+0000","last_clean":"2024-04-23T19:13:16.380363+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:16.380363+0000","last_undegraded":"2024-04-23T19:13:16.380363+0000","last_fullsized":"2024-04-23T19:13:16.380363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258229+0000","last_change":"2024-04-23T19:10:09.484981+0000","last_active":"2024-04-23T19:13:16.258229+0000","last_peered":"2024-04-23T19:13:16.258229+0000","last_clean":"2024-04-23T19:13:16.258229+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:13:16.258229+0000","last_undegraded":"2024-04-23T19:13:16.258229+0000","last_fullsized":"2024-04-23T19:13:16.258229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:13.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.951884+0000","last_change":"2024-04-23T19:10:07.490970+0000","last_active":"2024-04-23T19:11:31.951884+0000","last_peered":"2024-04-23T19:11:31.951884+0000","last_clean":"2024-04-23T19:11:31.951884+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:11:31.951884+0000","last_undegraded":"2024-04-23T19:11:31.951884+0000","last_fullsized":"2024-04-23T19:11:31.951884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:07:19.094583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.698565+0000","last_change":"2024-04-23T19:10:09.484268+0000","last_active":"2024-04-23T19:10:49.698565+0000","last_peered":"2024-04-23T19:10:49.698565+0000","last_clean":"2024-04-23T19:10:49.698565+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:10:49.698565+0000","last_undegraded":"2024-04-23T19:10:49.698565+0000","last_fullsized":"2024-04-23T19:10:49.698565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:46.058996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726273+0000","last_change":"2024-04-23T19:10:07.507068+0000","last_active":"2024-04-23T19:10:47.726273+0000","last_peered":"2024-04-23T19:10:47.726273+0000","last_clean":"2024-04-23T19:10:47.726273+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:10:47.726273+0000","last_undegraded":"2024-04-23T19:10:47.726273+0000","last_fullsized":"2024-04-23T19:10:47.726273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.361176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489138+0000","last_change":"2024-04-23T19:10:09.498436+0000","last_active":"2024-04-23T19:13:16.489138+0000","last_peered":"2024-04-23T19:13:16.489138+0000","last_clean":"2024-04-23T19:13:16.489138+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:13:16.489138+0000","last_undegraded":"2024-04-23T19:13:16.489138+0000","last_fullsized":"2024-04-23T19:13:16.489138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:55.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.685736+0000","last_change":"2024-04-23T19:10:07.489441+0000","last_active":"2024-04-23T19:10:49.685736+0000","last_peered":"2024-04-23T19:10:49.685736+0000","last_clean":"2024-04-23T19:10:49.685736+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:10:49.685736+0000","last_undegraded":"2024-04-23T19:10:49.685736+0000","last_fullsized":"2024-04-23T19:10:49.685736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:36:43.366606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'5","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712408+0000","last_change":"2024-04-23T19:10:09.492013+0000","last_active":"2024-04-23T19:12:25.712408+0000","last_peered":"2024-04-23T19:12:25.712408+0000","last_clean":"2024-04-23T19:12:25.712408+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:12:25.712408+0000","last_undegraded":"2024-04-23T19:12:25.712408+0000","last_fullsized":"2024-04-23T19:12:25.712408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:10.933511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.717325+0000","last_change":"2024-04-23T19:10:07.490880+0000","last_active":"2024-04-23T19:10:47.717325+0000","last_peered":"2024-04-23T19:10:47.717325+0000","last_clean":"2024-04-23T19:10:47.717325+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:10:47.717325+0000","last_undegraded":"2024-04-23T19:10:47.717325+0000","last_fullsized":"2024-04-23T19:10:47.717325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:49.697786+0000","last_change":"2024-04-23T19:10:09.482277+0000","last_active":"2024-04-23T19:10:49.697786+0000","last_peered":"2024-04-23T19:10:49.697786+0000","last_clean":"2024-04-23T19:10:49.697786+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:10:49.697786+0000","last_undegraded":"2024-04-23T19:10:49.697786+0000","last_fullsized":"2024-04-23T19:10:49.697786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:29.033406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.719034+0000","last_change":"2024-04-23T19:10:07.502896+0000","last_active":"2024-04-23T19:10:47.719034+0000","last_peered":"2024-04-23T19:10:47.719034+0000","last_clean":"2024-04-23T19:10:47.719034+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:10:47.719034+0000","last_undegraded":"2024-04-23T19:10:47.719034+0000","last_fullsized":"2024-04-23T19:10:47.719034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:56:03.971569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:31.964394+0000","last_change":"2024-04-23T19:10:09.492153+0000","last_active":"2024-04-23T19:11:31.964394+0000","last_peered":"2024-04-23T19:11:31.964394+0000","last_clean":"2024-04-23T19:11:31.964394+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:11:31.964394+0000","last_undegraded":"2024-04-23T19:11:31.964394+0000","last_fullsized":"2024-04-23T19:11:31.964394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:05.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.490658+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.568491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.720741+0000","last_change":"2024-04-23T19:10:09.483025+0000","last_active":"2024-04-23T19:10:47.720741+0000","last_peered":"2024-04-23T19:10:47.720741+0000","last_clean":"2024-04-23T19:10:47.720741+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:10:47.720741+0000","last_undegraded":"2024-04-23T19:10:47.720741+0000","last_fullsized":"2024-04-23T19:10:47.720741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:47.851863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.729169+0000","last_change":"2024-04-23T19:10:07.502808+0000","last_active":"2024-04-23T19:10:47.729169+0000","last_peered":"2024-04-23T19:10:47.729169+0000","last_clean":"2024-04-23T19:10:47.729169+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:10:47.729169+0000","last_undegraded":"2024-04-23T19:10:47.729169+0000","last_fullsized":"2024-04-23T19:10:47.729169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:11.223317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'5","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719983+0000","last_change":"2024-04-23T19:10:09.495999+0000","last_active":"2024-04-23T19:12:26.719983+0000","last_peered":"2024-04-23T19:12:26.719983+0000","last_clean":"2024-04-23T19:12:26.719983+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:12:26.719983+0000","last_undegraded":"2024-04-23T19:12:26.719983+0000","last_fullsized":"2024-04-23T19:12:26.719983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:24:49.818415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.725442+0000","last_change":"2024-04-23T19:10:07.500476+0000","last_active":"2024-04-23T19:10:47.725442+0000","last_peered":"2024-04-23T19:10:47.725442+0000","last_clean":"2024-04-23T19:10:47.725442+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:10:47.725442+0000","last_undegraded":"2024-04-23T19:10:47.725442+0000","last_fullsized":"2024-04-23T19:10:47.725442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:27:26.813400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.258427+0000","last_change":"2024-04-23T19:10:09.489995+0000","last_active":"2024-04-23T19:13:16.258427+0000","last_peered":"2024-04-23T19:13:16.258427+0000","last_clean":"2024-04-23T19:13:16.258427+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:13:16.258427+0000","last_undegraded":"2024-04-23T19:13:16.258427+0000","last_fullsized":"2024-04-23T19:13:16.258427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:52.872950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.728417+0000","last_change":"2024-04-23T19:10:07.507910+0000","last_active":"2024-04-23T19:10:47.728417+0000","last_peered":"2024-04-23T19:10:47.728417+0000","last_clean":"2024-04-23T19:10:47.728417+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:10:47.728417+0000","last_undegraded":"2024-04-23T19:10:47.728417+0000","last_fullsized":"2024-04-23T19:10:47.728417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:27.025595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:10:47.726700+0000","last_change":"2024-04-23T19:10:07.503476+0000","last_active":"2024-04-23T19:10:47.726700+0000","last_peered":"2024-04-23T19:10:47.726700+0000","last_clean":"2024-04-23T19:10:47.726700+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:10:47.726700+0000","last_undegraded":"2024-04-23T19:10:47.726700+0000","last_fullsized":"2024-04-23T19:10:47.726700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:39.006602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.399931+0000","last_change":"2024-04-23T19:10:09.487033+0000","last_active":"2024-04-23T19:13:16.399931+0000","last_peered":"2024-04-23T19:13:16.399931+0000","last_clean":"2024-04-23T19:13:16.399931+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:13:16.399931+0000","last_undegraded":"2024-04-23T19:13:16.399931+0000","last_fullsized":"2024-04-23T19:13:16.399931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:46:36.162060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.392374+0000","last_change":"2024-04-23T19:10:07.486697+0000","last_active":"2024-04-23T19:13:16.392374+0000","last_peered":"2024-04-23T19:13:16.392374+0000","last_clean":"2024-04-23T19:13:16.392374+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:13:16.392374+0000","last_undegraded":"2024-04-23T19:13:16.392374+0000","last_fullsized":"2024-04-23T19:13:16.392374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:12.555231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.159090+0000","last_change":"2024-04-23T19:10:09.491506+0000","last_active":"2024-04-23T19:13:16.159090+0000","last_peered":"2024-04-23T19:13:16.159090+0000","last_clean":"2024-04-23T19:13:16.159090+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:16.159090+0000","last_undegraded":"2024-04-23T19:13:16.159090+0000","last_fullsized":"2024-04-23T19:13:16.159090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:20.619656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2271,"ondisk_log_size":2271,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1489,"num_read_kb":1286,"num_write":561,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":9114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1318,"ondisk_log_size":1318,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":965,"num_read_kb":965,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":9028480,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":87,"num_write":313,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19521536,"data_stored":19521536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673007,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46348,"kb_used_data":11012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325492,"statfs":{"total":96636764160,"available":96589303808,"internally_reserved":0,"allocated":11276288,"data_stored":10955671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673006,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44844,"kb_used_data":9508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326996,"statfs":{"total":96636764160,"available":96590843904,"internally_reserved":0,"allocated":9736192,"data_stored":9446289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":732924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705711,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":425984,"data_stored":144396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10809344,"data_stored":10809344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:13:27.747 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.747 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.747 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.747 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.748 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.749 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.750 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.751 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.752 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.753 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.754 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.755 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.756 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.757 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T19:09:51.428209+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=51, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.758 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.759 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.760 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.761 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.762 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.763 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.764 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.765 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.766 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.767 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.768 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.769 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.770 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.771 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.772 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.773 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.774 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.775 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:27.776 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T19:13:47.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:13:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e2dac3640 1 Processor -- start 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e2dac3640 1 -- start start 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e2dac3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e2812ec80 0x7f0e28131070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e2dac3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e28128410 con 0x7f0e281288d0 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e2dac3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0e281286b0 con 0x7f0e2812ec80 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e26ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e2812ec80 0x7f0e28131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e277fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e281288d0 0x7f0e28128cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60884/0 (socket says 172.21.15.16:60884) 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.891+0000 7f0e277fe640 1 -- 172.21.15.16:0/4065153421 learned_addr learned my addr 172.21.15.16:0/4065153421 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.892+0000 7f0e26ffd640 1 -- 172.21.15.16:0/4065153421 >> v1:172.21.15.16:6789/0 conn(0x7f0e281288d0 legacy=0x7f0e28128cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.892+0000 7f0e26ffd640 1 -- 172.21.15.16:0/4065153421 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2806de40 con 0x7f0e2812ec80 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.892+0000 7f0e26ffd640 1 --2- 172.21.15.16:0/4065153421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2812ec80 0x7f0e28131070 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009720 tx=0x7f0e1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788b297c8e767761 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.892+0000 7f0e267fc640 1 -- 172.21.15.16:0/4065153421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e1c0316d0 con 0x7f0e2812ec80 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e267fc640 1 -- 172.21.15.16:0/4065153421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1c037070 con 0x7f0e2812ec80 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e267fc640 1 -- 172.21.15.16:0/4065153421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e1c0319c0 con 0x7f0e2812ec80 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e2dac3640 1 -- 172.21.15.16:0/4065153421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2812ec80 msgr2=0x7f0e28131070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e2dac3640 1 --2- 172.21.15.16:0/4065153421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2812ec80 0x7f0e28131070 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009720 tx=0x7f0e1c02b840 comp rx=0 tx=0).stop 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e2dac3640 1 -- 172.21.15.16:0/4065153421 shutdown_connections 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e2dac3640 1 --2- 172.21.15.16:0/4065153421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2812ec80 0x7f0e28131070 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e2dac3640 1 -- 172.21.15.16:0/4065153421 >> 172.21.15.16:0/4065153421 conn(0x7f0e2809a200 msgr2=0x7f0e28097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e2dac3640 1 -- 172.21.15.16:0/4065153421 shutdown_connections 2024-04-23T19:13:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.893+0000 7f0e2dac3640 1 -- 172.21.15.16:0/4065153421 wait complete. 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.894+0000 7f0e2dac3640 1 Processor -- start 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.894+0000 7f0e2dac3640 1 -- start start 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e2dac3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e281288d0 0x7f0e281c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e2dac3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e280945f0 con 0x7f0e281288d0 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e281288d0 0x7f0e281c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e281288d0 0x7f0e281c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56238/0 (socket says 172.21.15.16:56238) 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e277fe640 1 -- 172.21.15.16:0/3059144721 learned_addr learned my addr 172.21.15.16:0/3059144721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e277fe640 1 -- 172.21.15.16:0/3059144721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2812aef0 con 0x7f0e281288d0 2024-04-23T19:13:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e277fe640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e281288d0 0x7f0e281c3890 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c4a0 tx=0x7f0e1800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e0ffff640 1 -- 172.21.15.16:0/3059144721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e18017070 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e0ffff640 1 -- 172.21.15.16:0/3059144721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18005c10 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2812c000 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e0ffff640 1 -- 172.21.15.16:0/3059144721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e18007070 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.895+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e281c3fb0 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.896+0000 7f0e0ffff640 1 -- 172.21.15.16:0/3059144721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f0e1801a070 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.896+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2812cb10 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.896+0000 7f0e0ffff640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0e140385c0 0x7f0e1403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.897+0000 7f0e0ffff640 1 -- 172.21.15.16:0/3059144721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f0e1804b380 con 0x7f0e281288d0 2024-04-23T19:13:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.897+0000 7f0e26ffd640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0e140385c0 0x7f0e1403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:13:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.897+0000 7f0e26ffd640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0e140385c0 0x7f0e1403aa80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c036000 tx=0x7f0e1c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:13:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:47.901+0000 7f0e0ffff640 1 -- 172.21.15.16:0/3059144721 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e180117a0 con 0x7f0e281288d0 2024-04-23T19:13:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.082+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e2812bb60 con 0x7f0e140385c0 2024-04-23T19:13:48.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.098+0000 7f0e0ffff640 1 -- 172.21.15.16:0/3059144721 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+658373 (secure 0 0 0) 0x7f0e2812bb60 con 0x7f0e140385c0 2024-04-23T19:13:48.109 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:13:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:13:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0e140385c0 msgr2=0x7f0e1403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0e140385c0 0x7f0e1403aa80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c036000 tx=0x7f0e1c008030 comp rx=0 tx=0).stop 2024-04-23T19:13:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e281288d0 msgr2=0x7f0e281c3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:13:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e281288d0 0x7f0e281c3890 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c4a0 tx=0x7f0e1800c960 comp rx=0 tx=0).stop 2024-04-23T19:13:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 shutdown_connections 2024-04-23T19:13:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f0e140385c0 0x7f0e1403aa80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 --2- 172.21.15.16:0/3059144721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e281288d0 0x7f0e281c3890 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:13:48.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 >> 172.21.15.16:0/3059144721 conn(0x7f0e2809a200 msgr2=0x7f0e28126310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:13:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 shutdown_connections 2024-04-23T19:13:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:13:48.102+0000 7f0e2dac3640 1 -- 172.21.15.16:0/3059144721 wait complete. 2024-04-23T19:13:48.120 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-23T19:13:46.708407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5092,"num_read_kb":4673,"num_write":2177,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1730,"num_omap_keys":8,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4718,"ondisk_log_size":4718,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":144184,"kb_used_data":2840,"kb_used_omap":6,"kb_used_meta":141305,"kb_avail":377343176,"statfs":{"total":386547056640,"available":386399412224,"internally_reserved":0,"allocated":2908160,"data_stored":1756034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":144697111},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1730,"num_omap_keys":8,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003560"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.715767+0000","last_change":"2024-04-23T19:10:07.492726+0000","last_active":"2024-04-23T19:12:15.715767+0000","last_peered":"2024-04-23T19:12:15.715767+0000","last_clean":"2024-04-23T19:12:15.715767+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:12:15.715767+0000","last_undegraded":"2024-04-23T19:12:15.715767+0000","last_fullsized":"2024-04-23T19:12:15.715767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.721981+0000","last_change":"2024-04-23T19:10:09.485943+0000","last_active":"2024-04-23T19:12:41.721981+0000","last_peered":"2024-04-23T19:12:41.721981+0000","last_clean":"2024-04-23T19:12:41.721981+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:12:41.721981+0000","last_undegraded":"2024-04-23T19:12:41.721981+0000","last_fullsized":"2024-04-23T19:12:41.721981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710986+0000","last_change":"2024-04-23T19:10:07.490929+0000","last_active":"2024-04-23T19:12:15.710986+0000","last_peered":"2024-04-23T19:12:15.710986+0000","last_clean":"2024-04-23T19:12:15.710986+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:12:15.710986+0000","last_undegraded":"2024-04-23T19:12:15.710986+0000","last_fullsized":"2024-04-23T19:12:15.710986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.266815+0000","last_change":"2024-04-23T19:10:09.498665+0000","last_active":"2024-04-23T19:13:16.266815+0000","last_peered":"2024-04-23T19:13:16.266815+0000","last_clean":"2024-04-23T19:13:16.266815+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:16.266815+0000","last_undegraded":"2024-04-23T19:13:16.266815+0000","last_fullsized":"2024-04-23T19:13:16.266815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717659+0000","last_change":"2024-04-23T19:10:07.508341+0000","last_active":"2024-04-23T19:12:11.717659+0000","last_peered":"2024-04-23T19:12:11.717659+0000","last_clean":"2024-04-23T19:12:11.717659+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:12:11.717659+0000","last_undegraded":"2024-04-23T19:12:11.717659+0000","last_fullsized":"2024-04-23T19:12:11.717659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722499+0000","last_change":"2024-04-23T19:10:09.498971+0000","last_active":"2024-04-23T19:12:41.722499+0000","last_peered":"2024-04-23T19:12:41.722499+0000","last_clean":"2024-04-23T19:12:41.722499+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:12:41.722499+0000","last_undegraded":"2024-04-23T19:12:41.722499+0000","last_fullsized":"2024-04-23T19:12:41.722499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.714883+0000","last_change":"2024-04-23T19:10:07.490859+0000","last_active":"2024-04-23T19:12:10.714883+0000","last_peered":"2024-04-23T19:12:10.714883+0000","last_clean":"2024-04-23T19:12:10.714883+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:12:10.714883+0000","last_undegraded":"2024-04-23T19:12:10.714883+0000","last_fullsized":"2024-04-23T19:12:10.714883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.461439+0000","last_change":"2024-04-23T19:10:09.489556+0000","last_active":"2024-04-23T19:13:16.461439+0000","last_peered":"2024-04-23T19:13:16.461439+0000","last_clean":"2024-04-23T19:13:16.461439+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:13:16.461439+0000","last_undegraded":"2024-04-23T19:13:16.461439+0000","last_fullsized":"2024-04-23T19:13:16.461439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721644+0000","last_change":"2024-04-23T19:10:07.504507+0000","last_active":"2024-04-23T19:12:36.721644+0000","last_peered":"2024-04-23T19:12:36.721644+0000","last_clean":"2024-04-23T19:12:36.721644+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:12:36.721644+0000","last_undegraded":"2024-04-23T19:12:36.721644+0000","last_fullsized":"2024-04-23T19:12:36.721644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.770987+0000","last_change":"2024-04-23T19:10:09.482365+0000","last_active":"2024-04-23T19:12:15.770987+0000","last_peered":"2024-04-23T19:12:15.770987+0000","last_clean":"2024-04-23T19:12:15.770987+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:12:15.770987+0000","last_undegraded":"2024-04-23T19:12:15.770987+0000","last_fullsized":"2024-04-23T19:12:15.770987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.105261+0000","last_change":"2024-04-23T19:10:07.490892+0000","last_active":"2024-04-23T19:13:23.105261+0000","last_peered":"2024-04-23T19:13:23.105261+0000","last_clean":"2024-04-23T19:13:23.105261+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:13:23.105261+0000","last_undegraded":"2024-04-23T19:13:23.105261+0000","last_fullsized":"2024-04-23T19:13:23.105261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.266254+0000","last_change":"2024-04-23T19:10:09.499345+0000","last_active":"2024-04-23T19:13:16.266254+0000","last_peered":"2024-04-23T19:13:16.266254+0000","last_clean":"2024-04-23T19:13:16.266254+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:16.266254+0000","last_undegraded":"2024-04-23T19:13:16.266254+0000","last_fullsized":"2024-04-23T19:13:16.266254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721533+0000","last_change":"2024-04-23T19:10:07.489309+0000","last_active":"2024-04-23T19:12:36.721533+0000","last_peered":"2024-04-23T19:12:36.721533+0000","last_clean":"2024-04-23T19:12:36.721533+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:12:36.721533+0000","last_undegraded":"2024-04-23T19:12:36.721533+0000","last_fullsized":"2024-04-23T19:12:36.721533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'23","reported_seq":90,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.537197+0000","last_change":"2024-04-23T19:10:09.492130+0000","last_active":"2024-04-23T19:13:16.537197+0000","last_peered":"2024-04-23T19:13:16.537197+0000","last_clean":"2024-04-23T19:13:16.537197+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:16.537197+0000","last_undegraded":"2024-04-23T19:13:16.537197+0000","last_fullsized":"2024-04-23T19:13:16.537197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.105077+0000","last_change":"2024-04-23T19:10:07.487170+0000","last_active":"2024-04-23T19:13:23.105077+0000","last_peered":"2024-04-23T19:13:23.105077+0000","last_clean":"2024-04-23T19:13:23.105077+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:13:23.105077+0000","last_undegraded":"2024-04-23T19:13:23.105077+0000","last_fullsized":"2024-04-23T19:13:23.105077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'17","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.537243+0000","last_change":"2024-04-23T19:10:09.493592+0000","last_active":"2024-04-23T19:13:16.537243+0000","last_peered":"2024-04-23T19:13:16.537243+0000","last_clean":"2024-04-23T19:13:16.537243+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:13:16.537243+0000","last_undegraded":"2024-04-23T19:13:16.537243+0000","last_fullsized":"2024-04-23T19:13:16.537243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720728+0000","last_change":"2024-04-23T19:10:09.499521+0000","last_active":"2024-04-23T19:12:31.720728+0000","last_peered":"2024-04-23T19:12:31.720728+0000","last_clean":"2024-04-23T19:12:31.720728+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:12:31.720728+0000","last_undegraded":"2024-04-23T19:12:31.720728+0000","last_fullsized":"2024-04-23T19:12:31.720728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720541+0000","last_change":"2024-04-23T19:10:07.491914+0000","last_active":"2024-04-23T19:12:31.720541+0000","last_peered":"2024-04-23T19:12:31.720541+0000","last_clean":"2024-04-23T19:12:31.720541+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:12:31.720541+0000","last_undegraded":"2024-04-23T19:12:31.720541+0000","last_fullsized":"2024-04-23T19:12:31.720541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.456078+0000","last_change":"2024-04-23T19:10:09.482719+0000","last_active":"2024-04-23T19:13:16.456078+0000","last_peered":"2024-04-23T19:13:16.456078+0000","last_clean":"2024-04-23T19:13:16.456078+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:13:16.456078+0000","last_undegraded":"2024-04-23T19:13:16.456078+0000","last_fullsized":"2024-04-23T19:13:16.456078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718957+0000","last_change":"2024-04-23T19:10:07.489933+0000","last_active":"2024-04-23T19:12:35.718957+0000","last_peered":"2024-04-23T19:12:35.718957+0000","last_clean":"2024-04-23T19:12:35.718957+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:12:35.718957+0000","last_undegraded":"2024-04-23T19:12:35.718957+0000","last_fullsized":"2024-04-23T19:12:35.718957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720311+0000","last_change":"2024-04-23T19:10:09.484959+0000","last_active":"2024-04-23T19:12:31.720311+0000","last_peered":"2024-04-23T19:12:31.720311+0000","last_clean":"2024-04-23T19:12:31.720311+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:12:31.720311+0000","last_undegraded":"2024-04-23T19:12:31.720311+0000","last_fullsized":"2024-04-23T19:12:31.720311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.107188+0000","last_change":"2024-04-23T19:10:07.493585+0000","last_active":"2024-04-23T19:13:23.107188+0000","last_peered":"2024-04-23T19:13:23.107188+0000","last_clean":"2024-04-23T19:13:23.107188+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:13:23.107188+0000","last_undegraded":"2024-04-23T19:13:23.107188+0000","last_fullsized":"2024-04-23T19:13:23.107188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408358+0000","last_change":"2024-04-23T19:10:09.491221+0000","last_active":"2024-04-23T19:13:16.408358+0000","last_peered":"2024-04-23T19:13:16.408358+0000","last_clean":"2024-04-23T19:13:16.408358+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:16.408358+0000","last_undegraded":"2024-04-23T19:13:16.408358+0000","last_fullsized":"2024-04-23T19:13:16.408358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720406+0000","last_change":"2024-04-23T19:10:07.501878+0000","last_active":"2024-04-23T19:12:31.720406+0000","last_peered":"2024-04-23T19:12:31.720406+0000","last_clean":"2024-04-23T19:12:31.720406+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:12:31.720406+0000","last_undegraded":"2024-04-23T19:12:31.720406+0000","last_fullsized":"2024-04-23T19:12:31.720406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718886+0000","last_change":"2024-04-23T19:10:09.492916+0000","last_active":"2024-04-23T19:12:35.718886+0000","last_peered":"2024-04-23T19:12:35.718886+0000","last_clean":"2024-04-23T19:12:35.718886+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:12:35.718886+0000","last_undegraded":"2024-04-23T19:12:35.718886+0000","last_fullsized":"2024-04-23T19:12:35.718886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.713939+0000","last_change":"2024-04-23T19:10:07.490910+0000","last_active":"2024-04-23T19:12:35.713939+0000","last_peered":"2024-04-23T19:12:35.713939+0000","last_clean":"2024-04-23T19:12:35.713939+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:12:35.713939+0000","last_undegraded":"2024-04-23T19:12:35.713939+0000","last_fullsized":"2024-04-23T19:12:35.713939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489138+0000","last_change":"2024-04-23T19:10:09.482527+0000","last_active":"2024-04-23T19:13:16.489138+0000","last_peered":"2024-04-23T19:13:16.489138+0000","last_clean":"2024-04-23T19:13:16.489138+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:16.489138+0000","last_undegraded":"2024-04-23T19:13:16.489138+0000","last_fullsized":"2024-04-23T19:13:16.489138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720008+0000","last_change":"2024-04-23T19:10:07.493956+0000","last_active":"2024-04-23T19:12:26.720008+0000","last_peered":"2024-04-23T19:12:26.720008+0000","last_clean":"2024-04-23T19:12:26.720008+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:12:26.720008+0000","last_undegraded":"2024-04-23T19:12:26.720008+0000","last_fullsized":"2024-04-23T19:12:26.720008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'35","reported_seq":100,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.497434+0000","last_change":"2024-04-23T19:10:17.514587+0000","last_active":"2024-04-23T19:13:16.497434+0000","last_peered":"2024-04-23T19:13:16.497434+0000","last_clean":"2024-04-23T19:13:16.497434+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:16.497434+0000","last_undegraded":"2024-04-23T19:13:16.497434+0000","last_fullsized":"2024-04-23T19:13:16.497434+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720840+0000","last_change":"2024-04-23T19:10:09.487433+0000","last_active":"2024-04-23T19:12:31.720840+0000","last_peered":"2024-04-23T19:12:31.720840+0000","last_clean":"2024-04-23T19:12:31.720840+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:12:31.720840+0000","last_undegraded":"2024-04-23T19:12:31.720840+0000","last_fullsized":"2024-04-23T19:12:31.720840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720039+0000","last_change":"2024-04-23T19:10:07.507346+0000","last_active":"2024-04-23T19:12:26.720039+0000","last_peered":"2024-04-23T19:12:26.720039+0000","last_clean":"2024-04-23T19:12:26.720039+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:12:26.720039+0000","last_undegraded":"2024-04-23T19:12:26.720039+0000","last_fullsized":"2024-04-23T19:12:26.720039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.986548+0000","last_change":"2024-04-23T19:13:32.986571+0000","last_active":"2024-04-23T19:13:32.986548+0000","last_peered":"2024-04-23T19:13:32.986548+0000","last_clean":"2024-04-23T19:13:32.986548+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:13:32.986548+0000","last_undegraded":"2024-04-23T19:13:32.986548+0000","last_fullsized":"2024-04-23T19:13:32.986548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.986453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:15.050911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'34","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.118750+0000","last_change":"2024-04-23T19:10:13.501588+0000","last_active":"2024-04-23T19:13:23.118750+0000","last_peered":"2024-04-23T19:13:23.118750+0000","last_clean":"2024-04-23T19:13:23.118750+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:13:23.118750+0000","last_undegraded":"2024-04-23T19:13:23.118750+0000","last_fullsized":"2024-04-23T19:13:23.118750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.929910+0000","last_change":"2024-04-23T19:13:38.929939+0000","last_active":"2024-04-23T19:13:38.929910+0000","last_peered":"2024-04-23T19:13:38.929910+0000","last_clean":"2024-04-23T19:13:38.929910+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:13:38.929910+0000","last_undegraded":"2024-04-23T19:13:38.929910+0000","last_fullsized":"2024-04-23T19:13:38.929910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.929807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:00.218571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'9","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.597740+0000","last_change":"2024-04-23T19:10:17.514347+0000","last_active":"2024-04-23T19:13:16.597740+0000","last_peered":"2024-04-23T19:13:16.597740+0000","last_clean":"2024-04-23T19:13:16.597740+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:16.597740+0000","last_undegraded":"2024-04-23T19:13:16.597740+0000","last_fullsized":"2024-04-23T19:13:16.597740+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.405978+0000","last_change":"2024-04-23T19:10:09.496014+0000","last_active":"2024-04-23T19:13:16.405978+0000","last_peered":"2024-04-23T19:13:16.405978+0000","last_clean":"2024-04-23T19:13:16.405978+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:13:16.405978+0000","last_undegraded":"2024-04-23T19:13:16.405978+0000","last_fullsized":"2024-04-23T19:13:16.405978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:30.718135+0000","last_change":"2024-04-23T19:10:07.492730+0000","last_active":"2024-04-23T19:12:30.718135+0000","last_peered":"2024-04-23T19:12:30.718135+0000","last_clean":"2024-04-23T19:12:30.718135+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:12:30.718135+0000","last_undegraded":"2024-04-23T19:12:30.718135+0000","last_fullsized":"2024-04-23T19:12:30.718135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"23'1","reported_seq":221,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.017270+0000","last_change":"2024-04-23T19:10:11.489225+0000","last_active":"2024-04-23T19:13:23.017270+0000","last_peered":"2024-04-23T19:13:23.017270+0000","last_clean":"2024-04-23T19:13:23.017270+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:13:23.017270+0000","last_undegraded":"2024-04-23T19:13:23.017270+0000","last_fullsized":"2024-04-23T19:13:23.017270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"32'9","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.851896+0000","last_change":"2024-04-23T19:13:32.851932+0000","last_active":"2024-04-23T19:13:32.851896+0000","last_peered":"2024-04-23T19:13:32.851896+0000","last_clean":"2024-04-23T19:13:32.851896+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:13:32.851896+0000","last_undegraded":"2024-04-23T19:13:32.851896+0000","last_fullsized":"2024-04-23T19:13:32.851896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'9","last_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_deep_scrub":"32'9","last_deep_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.851806+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:35.714512+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.911789+0000","last_change":"2024-04-23T19:13:34.911815+0000","last_active":"2024-04-23T19:13:34.911789+0000","last_peered":"2024-04-23T19:13:34.911789+0000","last_clean":"2024-04-23T19:13:34.911789+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:13:34.911789+0000","last_undegraded":"2024-04-23T19:13:34.911789+0000","last_fullsized":"2024-04-23T19:13:34.911789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.911697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:51.912852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:50.715845+0000","last_change":"2024-04-23T19:10:17.518249+0000","last_active":"2024-04-23T19:12:50.715845+0000","last_peered":"2024-04-23T19:12:50.715845+0000","last_clean":"2024-04-23T19:12:50.715845+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:12:50.715845+0000","last_undegraded":"2024-04-23T19:12:50.715845+0000","last_fullsized":"2024-04-23T19:12:50.715845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.456147+0000","last_change":"2024-04-23T19:10:09.490234+0000","last_active":"2024-04-23T19:13:16.456147+0000","last_peered":"2024-04-23T19:13:16.456147+0000","last_clean":"2024-04-23T19:13:16.456147+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:13:16.456147+0000","last_undegraded":"2024-04-23T19:13:16.456147+0000","last_fullsized":"2024-04-23T19:13:16.456147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.391967+0000","last_change":"2024-04-23T19:10:07.504260+0000","last_active":"2024-04-23T19:13:16.391967+0000","last_peered":"2024-04-23T19:13:16.391967+0000","last_clean":"2024-04-23T19:13:16.391967+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:13:16.391967+0000","last_undegraded":"2024-04-23T19:13:16.391967+0000","last_fullsized":"2024-04-23T19:13:16.391967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.612267+0000","last_change":"2024-04-23T19:13:44.612314+0000","last_active":"2024-04-23T19:13:44.612267+0000","last_peered":"2024-04-23T19:13:44.612267+0000","last_clean":"2024-04-23T19:13:44.612267+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:13:44.612267+0000","last_undegraded":"2024-04-23T19:13:44.612267+0000","last_fullsized":"2024-04-23T19:13:44.612267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.612159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:59.958714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'16","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.181820+0000","last_change":"2024-04-23T19:13:36.181858+0000","last_active":"2024-04-23T19:13:36.181820+0000","last_peered":"2024-04-23T19:13:36.181820+0000","last_clean":"2024-04-23T19:13:36.181820+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:13:36.181820+0000","last_undegraded":"2024-04-23T19:13:36.181820+0000","last_fullsized":"2024-04-23T19:13:36.181820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.181735+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:21.509377+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'285","reported_seq":389,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.193029+0000","last_change":"2024-04-23T19:10:15.515398+0000","last_active":"2024-04-23T19:13:23.193029+0000","last_peered":"2024-04-23T19:13:23.193029+0000","last_clean":"2024-04-23T19:13:23.193029+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:13:23.193029+0000","last_undegraded":"2024-04-23T19:13:23.193029+0000","last_fullsized":"2024-04-23T19:13:23.193029+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:50.721007+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:50.721007+0000","last_peered":"2024-04-23T19:12:50.721007+0000","last_clean":"2024-04-23T19:12:50.721007+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:12:50.721007+0000","last_undegraded":"2024-04-23T19:12:50.721007+0000","last_fullsized":"2024-04-23T19:12:50.721007+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489503+0000","last_change":"2024-04-23T19:10:09.487095+0000","last_active":"2024-04-23T19:13:16.489503+0000","last_peered":"2024-04-23T19:13:16.489503+0000","last_clean":"2024-04-23T19:13:16.489503+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:13:16.489503+0000","last_undegraded":"2024-04-23T19:13:16.489503+0000","last_fullsized":"2024-04-23T19:13:16.489503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719376+0000","last_change":"2024-04-23T19:10:07.504154+0000","last_active":"2024-04-23T19:12:26.719376+0000","last_peered":"2024-04-23T19:12:26.719376+0000","last_clean":"2024-04-23T19:12:26.719376+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:12:26.719376+0000","last_undegraded":"2024-04-23T19:12:26.719376+0000","last_fullsized":"2024-04-23T19:12:26.719376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.940488+0000","last_change":"2024-04-23T19:13:34.940545+0000","last_active":"2024-04-23T19:13:34.940488+0000","last_peered":"2024-04-23T19:13:34.940488+0000","last_clean":"2024-04-23T19:13:34.940488+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:13:34.940488+0000","last_undegraded":"2024-04-23T19:13:34.940488+0000","last_fullsized":"2024-04-23T19:13:34.940488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.940371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:19.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.236705+0000","last_change":"2024-04-23T19:13:34.236743+0000","last_active":"2024-04-23T19:13:34.236705+0000","last_peered":"2024-04-23T19:13:34.236705+0000","last_clean":"2024-04-23T19:13:34.236705+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:13:34.236705+0000","last_undegraded":"2024-04-23T19:13:34.236705+0000","last_fullsized":"2024-04-23T19:13:34.236705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.236613+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:39.299958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'285","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.684726+0000","last_change":"2024-04-23T19:13:32.684754+0000","last_active":"2024-04-23T19:13:32.684726+0000","last_peered":"2024-04-23T19:13:32.684726+0000","last_clean":"2024-04-23T19:13:32.684726+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:32.684726+0000","last_undegraded":"2024-04-23T19:13:32.684726+0000","last_fullsized":"2024-04-23T19:13:32.684726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.684622+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:16.152805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458795+0000","last_change":"2024-04-23T19:10:47.730482+0000","last_active":"2024-04-23T19:13:16.458795+0000","last_peered":"2024-04-23T19:13:16.458795+0000","last_clean":"2024-04-23T19:13:16.458795+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:13:16.458795+0000","last_undegraded":"2024-04-23T19:13:16.458795+0000","last_fullsized":"2024-04-23T19:13:16.458795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.717521+0000","last_change":"2024-04-23T19:10:07.492480+0000","last_active":"2024-04-23T19:12:25.717521+0000","last_peered":"2024-04-23T19:12:25.717521+0000","last_clean":"2024-04-23T19:12:25.717521+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:12:25.717521+0000","last_undegraded":"2024-04-23T19:12:25.717521+0000","last_fullsized":"2024-04-23T19:12:25.717521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.919126+0000","last_change":"2024-04-23T19:13:40.919152+0000","last_active":"2024-04-23T19:13:40.919126+0000","last_peered":"2024-04-23T19:13:40.919126+0000","last_clean":"2024-04-23T19:13:40.919126+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:13:40.919126+0000","last_undegraded":"2024-04-23T19:13:40.919126+0000","last_fullsized":"2024-04-23T19:13:40.919126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.919038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:35.160636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'51","reported_seq":111,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.205995+0000","last_change":"2024-04-23T19:13:39.206079+0000","last_active":"2024-04-23T19:13:39.205995+0000","last_peered":"2024-04-23T19:13:39.205995+0000","last_clean":"2024-04-23T19:13:39.205995+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:13:39.205995+0000","last_undegraded":"2024-04-23T19:13:39.205995+0000","last_fullsized":"2024-04-23T19:13:39.205995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'51","last_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_deep_scrub":"32'51","last_deep_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.205895+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:03.299394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.646105+0000","last_change":"2024-04-23T19:13:43.646133+0000","last_active":"2024-04-23T19:13:43.646105+0000","last_peered":"2024-04-23T19:13:43.646105+0000","last_clean":"2024-04-23T19:13:43.646105+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:13:43.646105+0000","last_undegraded":"2024-04-23T19:13:43.646105+0000","last_fullsized":"2024-04-23T19:13:43.646105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.646006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:44.344504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.717405+0000","last_change":"2024-04-23T19:10:17.519710+0000","last_active":"2024-04-23T19:12:25.717405+0000","last_peered":"2024-04-23T19:12:25.717405+0000","last_clean":"2024-04-23T19:12:25.717405+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:12:25.717405+0000","last_undegraded":"2024-04-23T19:12:25.717405+0000","last_fullsized":"2024-04-23T19:12:25.717405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'7","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.535976+0000","last_change":"2024-04-23T19:10:09.482665+0000","last_active":"2024-04-23T19:13:16.535976+0000","last_peered":"2024-04-23T19:13:16.535976+0000","last_clean":"2024-04-23T19:13:16.535976+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:16.535976+0000","last_undegraded":"2024-04-23T19:13:16.535976+0000","last_fullsized":"2024-04-23T19:13:16.535976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.717476+0000","last_change":"2024-04-23T19:10:07.500938+0000","last_active":"2024-04-23T19:12:25.717476+0000","last_peered":"2024-04-23T19:12:25.717476+0000","last_clean":"2024-04-23T19:12:25.717476+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:12:25.717476+0000","last_undegraded":"2024-04-23T19:12:25.717476+0000","last_fullsized":"2024-04-23T19:12:25.717476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.680623+0000","last_change":"2024-04-23T19:13:42.680645+0000","last_active":"2024-04-23T19:13:42.680623+0000","last_peered":"2024-04-23T19:13:42.680623+0000","last_clean":"2024-04-23T19:13:42.680623+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:13:42.680623+0000","last_undegraded":"2024-04-23T19:13:42.680623+0000","last_fullsized":"2024-04-23T19:13:42.680623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.680534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:03.703466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'34","reported_seq":81,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.175252+0000","last_change":"2024-04-23T19:10:13.501851+0000","last_active":"2024-04-23T19:13:23.175252+0000","last_peered":"2024-04-23T19:13:23.175252+0000","last_clean":"2024-04-23T19:13:23.175252+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:13:23.175252+0000","last_undegraded":"2024-04-23T19:13:23.175252+0000","last_fullsized":"2024-04-23T19:13:23.175252+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.585103+0000","last_change":"2024-04-23T19:13:28.585132+0000","last_active":"2024-04-23T19:13:28.585103+0000","last_peered":"2024-04-23T19:13:28.585103+0000","last_clean":"2024-04-23T19:13:28.585103+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:13:28.585103+0000","last_undegraded":"2024-04-23T19:13:28.585103+0000","last_fullsized":"2024-04-23T19:13:28.585103+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.585004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:05.071466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'5","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.267889+0000","last_change":"2024-04-23T19:10:17.518314+0000","last_active":"2024-04-23T19:13:16.267889+0000","last_peered":"2024-04-23T19:13:16.267889+0000","last_clean":"2024-04-23T19:13:16.267889+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:13:16.267889+0000","last_undegraded":"2024-04-23T19:13:16.267889+0000","last_fullsized":"2024-04-23T19:13:16.267889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":80,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.168713+0000","last_change":"2024-04-23T19:10:09.492229+0000","last_active":"2024-04-23T19:13:16.168713+0000","last_peered":"2024-04-23T19:13:16.168713+0000","last_clean":"2024-04-23T19:13:16.168713+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:13:16.168713+0000","last_undegraded":"2024-04-23T19:13:16.168713+0000","last_fullsized":"2024-04-23T19:13:16.168713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719245+0000","last_change":"2024-04-23T19:10:07.492637+0000","last_active":"2024-04-23T19:12:21.719245+0000","last_peered":"2024-04-23T19:12:21.719245+0000","last_clean":"2024-04-23T19:12:21.719245+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:12:21.719245+0000","last_undegraded":"2024-04-23T19:12:21.719245+0000","last_fullsized":"2024-04-23T19:12:21.719245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.573918+0000","last_change":"2024-04-23T19:13:45.573946+0000","last_active":"2024-04-23T19:13:45.573918+0000","last_peered":"2024-04-23T19:13:45.573918+0000","last_clean":"2024-04-23T19:13:45.573918+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:13:45.573918+0000","last_undegraded":"2024-04-23T19:13:45.573918+0000","last_fullsized":"2024-04-23T19:13:45.573918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.573803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:16.532433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'39","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.138728+0000","last_change":"2024-04-23T19:10:13.508355+0000","last_active":"2024-04-23T19:13:23.138728+0000","last_peered":"2024-04-23T19:13:23.138728+0000","last_clean":"2024-04-23T19:13:23.138728+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:13:23.138728+0000","last_undegraded":"2024-04-23T19:13:23.138728+0000","last_fullsized":"2024-04-23T19:13:23.138728+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'285","reported_seq":352,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.198661+0000","last_change":"2024-04-23T19:10:15.509684+0000","last_active":"2024-04-23T19:13:23.198661+0000","last_peered":"2024-04-23T19:13:23.198661+0000","last_clean":"2024-04-23T19:13:23.198661+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:23.198661+0000","last_undegraded":"2024-04-23T19:13:23.198661+0000","last_fullsized":"2024-04-23T19:13:23.198661+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:30.713313+0000","last_change":"2024-04-23T19:10:17.520791+0000","last_active":"2024-04-23T19:12:30.713313+0000","last_peered":"2024-04-23T19:12:30.713313+0000","last_clean":"2024-04-23T19:12:30.713313+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:12:30.713313+0000","last_undegraded":"2024-04-23T19:12:30.713313+0000","last_fullsized":"2024-04-23T19:12:30.713313+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.405951+0000","last_change":"2024-04-23T19:10:09.476300+0000","last_active":"2024-04-23T19:13:16.405951+0000","last_peered":"2024-04-23T19:13:16.405951+0000","last_clean":"2024-04-23T19:13:16.405951+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:16.405951+0000","last_undegraded":"2024-04-23T19:13:16.405951+0000","last_fullsized":"2024-04-23T19:13:16.405951+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719160+0000","last_change":"2024-04-23T19:10:07.501083+0000","last_active":"2024-04-23T19:12:21.719160+0000","last_peered":"2024-04-23T19:12:21.719160+0000","last_clean":"2024-04-23T19:12:21.719160+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:12:21.719160+0000","last_undegraded":"2024-04-23T19:12:21.719160+0000","last_fullsized":"2024-04-23T19:12:21.719160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.667015+0000","last_change":"2024-04-23T19:13:41.667080+0000","last_active":"2024-04-23T19:13:41.667015+0000","last_peered":"2024-04-23T19:13:41.667015+0000","last_clean":"2024-04-23T19:13:41.667015+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:13:41.667015+0000","last_undegraded":"2024-04-23T19:13:41.667015+0000","last_fullsized":"2024-04-23T19:13:41.667015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.666920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:52.673059+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.975145+0000","last_change":"2024-04-23T19:13:33.975191+0000","last_active":"2024-04-23T19:13:33.975145+0000","last_peered":"2024-04-23T19:13:33.975145+0000","last_clean":"2024-04-23T19:13:33.975145+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:13:33.975145+0000","last_undegraded":"2024-04-23T19:13:33.975145+0000","last_fullsized":"2024-04-23T19:13:33.975145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.975049+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:07.187015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.695421+0000","last_change":"2024-04-23T19:13:35.695444+0000","last_active":"2024-04-23T19:13:35.695421+0000","last_peered":"2024-04-23T19:13:35.695421+0000","last_clean":"2024-04-23T19:13:35.695421+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:13:35.695421+0000","last_undegraded":"2024-04-23T19:13:35.695421+0000","last_fullsized":"2024-04-23T19:13:35.695421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.695326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:43.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'18","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.587428+0000","last_change":"2024-04-23T19:10:17.513990+0000","last_active":"2024-04-23T19:13:16.587428+0000","last_peered":"2024-04-23T19:13:16.587428+0000","last_clean":"2024-04-23T19:13:16.587428+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:13:16.587428+0000","last_undegraded":"2024-04-23T19:13:16.587428+0000","last_fullsized":"2024-04-23T19:13:16.587428+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.265877+0000","last_change":"2024-04-23T19:10:09.490325+0000","last_active":"2024-04-23T19:13:16.265877+0000","last_peered":"2024-04-23T19:13:16.265877+0000","last_clean":"2024-04-23T19:13:16.265877+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:16.265877+0000","last_undegraded":"2024-04-23T19:13:16.265877+0000","last_fullsized":"2024-04-23T19:13:16.265877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712349+0000","last_change":"2024-04-23T19:10:07.490182+0000","last_active":"2024-04-23T19:12:25.712349+0000","last_peered":"2024-04-23T19:12:25.712349+0000","last_clean":"2024-04-23T19:12:25.712349+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:12:25.712349+0000","last_undegraded":"2024-04-23T19:12:25.712349+0000","last_fullsized":"2024-04-23T19:12:25.712349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.190333+0000","last_change":"2024-04-23T19:13:44.190374+0000","last_active":"2024-04-23T19:13:44.190333+0000","last_peered":"2024-04-23T19:13:44.190333+0000","last_clean":"2024-04-23T19:13:44.190333+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:13:44.190333+0000","last_undegraded":"2024-04-23T19:13:44.190333+0000","last_fullsized":"2024-04-23T19:13:44.190333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.190242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:32.140502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.877432+0000","last_change":"2024-04-23T19:13:33.877469+0000","last_active":"2024-04-23T19:13:33.877432+0000","last_peered":"2024-04-23T19:13:33.877432+0000","last_clean":"2024-04-23T19:13:33.877432+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:13:33.877432+0000","last_undegraded":"2024-04-23T19:13:33.877432+0000","last_fullsized":"2024-04-23T19:13:33.877432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.877330+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:17.345513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720919+0000","last_change":"2024-04-23T19:10:15.512494+0000","last_active":"2024-04-23T19:12:31.720919+0000","last_peered":"2024-04-23T19:12:31.720919+0000","last_clean":"2024-04-23T19:12:31.720919+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:12:31.720919+0000","last_undegraded":"2024-04-23T19:12:31.720919+0000","last_fullsized":"2024-04-23T19:12:31.720919+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.540893+0000","last_change":"2024-04-23T19:10:17.511337+0000","last_active":"2024-04-23T19:13:16.540893+0000","last_peered":"2024-04-23T19:13:16.540893+0000","last_clean":"2024-04-23T19:13:16.540893+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:13:16.540893+0000","last_undegraded":"2024-04-23T19:13:16.540893+0000","last_fullsized":"2024-04-23T19:13:16.540893+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536783+0000","last_change":"2024-04-23T19:10:09.499760+0000","last_active":"2024-04-23T19:13:16.536783+0000","last_peered":"2024-04-23T19:13:16.536783+0000","last_clean":"2024-04-23T19:13:16.536783+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:16.536783+0000","last_undegraded":"2024-04-23T19:13:16.536783+0000","last_fullsized":"2024-04-23T19:13:16.536783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718347+0000","last_change":"2024-04-23T19:10:07.494969+0000","last_active":"2024-04-23T19:12:16.718347+0000","last_peered":"2024-04-23T19:12:16.718347+0000","last_clean":"2024-04-23T19:12:16.718347+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:12:16.718347+0000","last_undegraded":"2024-04-23T19:12:16.718347+0000","last_fullsized":"2024-04-23T19:12:16.718347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712379+0000","last_change":"2024-04-23T19:10:11.491444+0000","last_active":"2024-04-23T19:12:25.712379+0000","last_peered":"2024-04-23T19:12:25.712379+0000","last_clean":"2024-04-23T19:12:25.712379+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:12:25.712379+0000","last_undegraded":"2024-04-23T19:12:25.712379+0000","last_fullsized":"2024-04-23T19:12:25.712379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'45","reported_seq":103,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.188689+0000","last_change":"2024-04-23T19:13:40.188736+0000","last_active":"2024-04-23T19:13:40.188689+0000","last_peered":"2024-04-23T19:13:40.188689+0000","last_clean":"2024-04-23T19:13:40.188689+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:13:40.188689+0000","last_undegraded":"2024-04-23T19:13:40.188689+0000","last_fullsized":"2024-04-23T19:13:40.188689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.188598+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:54.836675+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'285","reported_seq":415,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.128944+0000","last_change":"2024-04-23T19:13:28.128972+0000","last_active":"2024-04-23T19:13:28.128944+0000","last_peered":"2024-04-23T19:13:28.128944+0000","last_clean":"2024-04-23T19:13:28.128944+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:28.128944+0000","last_undegraded":"2024-04-23T19:13:28.128944+0000","last_fullsized":"2024-04-23T19:13:28.128944+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.128856+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:09.022827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'12","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.542668+0000","last_change":"2024-04-23T19:10:17.518152+0000","last_active":"2024-04-23T19:13:16.542668+0000","last_peered":"2024-04-23T19:13:16.542668+0000","last_clean":"2024-04-23T19:13:16.542668+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:13:16.542668+0000","last_undegraded":"2024-04-23T19:13:16.542668+0000","last_fullsized":"2024-04-23T19:13:16.542668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408231+0000","last_change":"2024-04-23T19:10:09.493242+0000","last_active":"2024-04-23T19:13:16.408231+0000","last_peered":"2024-04-23T19:13:16.408231+0000","last_clean":"2024-04-23T19:13:16.408231+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:13:16.408231+0000","last_undegraded":"2024-04-23T19:13:16.408231+0000","last_fullsized":"2024-04-23T19:13:16.408231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718301+0000","last_change":"2024-04-23T19:10:07.508434+0000","last_active":"2024-04-23T19:12:16.718301+0000","last_peered":"2024-04-23T19:12:16.718301+0000","last_clean":"2024-04-23T19:12:16.718301+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:12:16.718301+0000","last_undegraded":"2024-04-23T19:12:16.718301+0000","last_fullsized":"2024-04-23T19:12:16.718301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'5","reported_seq":428,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.911991+0000","last_change":"2024-04-23T19:13:37.912027+0000","last_active":"2024-04-23T19:13:37.911991+0000","last_peered":"2024-04-23T19:13:37.911991+0000","last_clean":"2024-04-23T19:13:37.911991+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:13:37.911991+0000","last_undegraded":"2024-04-23T19:13:37.911991+0000","last_fullsized":"2024-04-23T19:13:37.911991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.911892+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:23.295978+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"30'10","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720850+0000","last_change":"2024-04-23T19:10:13.501643+0000","last_active":"2024-04-23T19:12:31.720850+0000","last_peered":"2024-04-23T19:12:31.720850+0000","last_clean":"2024-04-23T19:12:31.720850+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:12:31.720850+0000","last_undegraded":"2024-04-23T19:12:31.720850+0000","last_fullsized":"2024-04-23T19:12:31.720850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'285","reported_seq":477,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.784157+0000","last_change":"2024-04-23T19:13:26.784184+0000","last_active":"2024-04-23T19:13:26.784157+0000","last_peered":"2024-04-23T19:13:26.784157+0000","last_clean":"2024-04-23T19:13:26.784157+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:26.784157+0000","last_undegraded":"2024-04-23T19:13:26.784157+0000","last_fullsized":"2024-04-23T19:13:26.784157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.784064+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:31.847408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.211829+0000","last_change":"2024-04-23T19:13:41.211854+0000","last_active":"2024-04-23T19:13:41.211829+0000","last_peered":"2024-04-23T19:13:41.211829+0000","last_clean":"2024-04-23T19:13:41.211829+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:13:41.211829+0000","last_undegraded":"2024-04-23T19:13:41.211829+0000","last_fullsized":"2024-04-23T19:13:41.211829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.211740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.450923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719568+0000","last_change":"2024-04-23T19:10:09.494353+0000","last_active":"2024-04-23T19:12:26.719568+0000","last_peered":"2024-04-23T19:12:26.719568+0000","last_clean":"2024-04-23T19:12:26.719568+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:12:26.719568+0000","last_undegraded":"2024-04-23T19:12:26.719568+0000","last_fullsized":"2024-04-23T19:12:26.719568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.388835+0000","last_change":"2024-04-23T19:10:07.507496+0000","last_active":"2024-04-23T19:13:16.388835+0000","last_peered":"2024-04-23T19:13:16.388835+0000","last_clean":"2024-04-23T19:13:16.388835+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:13:16.388835+0000","last_undegraded":"2024-04-23T19:13:16.388835+0000","last_fullsized":"2024-04-23T19:13:16.388835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.111154+0000","last_change":"2024-04-23T19:13:28.111184+0000","last_active":"2024-04-23T19:13:28.111154+0000","last_peered":"2024-04-23T19:13:28.111154+0000","last_clean":"2024-04-23T19:13:28.111154+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:13:28.111154+0000","last_undegraded":"2024-04-23T19:13:28.111154+0000","last_fullsized":"2024-04-23T19:13:28.111154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.111056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:18.387489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'48","reported_seq":107,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.948761+0000","last_change":"2024-04-23T19:13:35.948796+0000","last_active":"2024-04-23T19:13:35.948761+0000","last_peered":"2024-04-23T19:13:35.948761+0000","last_clean":"2024-04-23T19:13:35.948761+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:13:35.948761+0000","last_undegraded":"2024-04-23T19:13:35.948761+0000","last_fullsized":"2024-04-23T19:13:35.948761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'48","last_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_deep_scrub":"32'48","last_deep_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.948676+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:52.110221+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.116201+0000","last_change":"2024-04-23T19:13:30.116258+0000","last_active":"2024-04-23T19:13:30.116201+0000","last_peered":"2024-04-23T19:13:30.116201+0000","last_clean":"2024-04-23T19:13:30.116201+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:13:30.116201+0000","last_undegraded":"2024-04-23T19:13:30.116201+0000","last_fullsized":"2024-04-23T19:13:30.116201+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.116108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:58.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'6","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.585694+0000","last_change":"2024-04-23T19:10:17.506577+0000","last_active":"2024-04-23T19:13:16.585694+0000","last_peered":"2024-04-23T19:13:16.585694+0000","last_clean":"2024-04-23T19:13:16.585694+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:16.585694+0000","last_undegraded":"2024-04-23T19:13:16.585694+0000","last_fullsized":"2024-04-23T19:13:16.585694+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.894478+0000","last_change":"2024-04-23T19:13:40.894509+0000","last_active":"2024-04-23T19:13:40.894478+0000","last_peered":"2024-04-23T19:13:40.894478+0000","last_clean":"2024-04-23T19:13:40.894478+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:13:40.894478+0000","last_undegraded":"2024-04-23T19:13:40.894478+0000","last_fullsized":"2024-04-23T19:13:40.894478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.894372+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:11.403485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":202,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:11:51.714109+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:11:51.714109+0000","last_peered":"2024-04-23T19:11:51.714109+0000","last_clean":"2024-04-23T19:11:51.714109+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:11:51.714109+0000","last_undegraded":"2024-04-23T19:11:51.714109+0000","last_fullsized":"2024-04-23T19:11:51.714109+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:06.716831+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:12:06.716831+0000","last_peered":"2024-04-23T19:12:06.716831+0000","last_clean":"2024-04-23T19:12:06.716831+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:12:06.716831+0000","last_undegraded":"2024-04-23T19:12:06.716831+0000","last_fullsized":"2024-04-23T19:12:06.716831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717583+0000","last_change":"2024-04-23T19:10:07.508995+0000","last_active":"2024-04-23T19:12:11.717583+0000","last_peered":"2024-04-23T19:12:11.717583+0000","last_clean":"2024-04-23T19:12:11.717583+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:12:11.717583+0000","last_undegraded":"2024-04-23T19:12:11.717583+0000","last_fullsized":"2024-04-23T19:12:11.717583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.717831+0000","last_change":"2024-04-23T19:10:11.486708+0000","last_active":"2024-04-23T19:12:16.717831+0000","last_peered":"2024-04-23T19:12:16.717831+0000","last_clean":"2024-04-23T19:12:16.717831+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:12:16.717831+0000","last_undegraded":"2024-04-23T19:12:16.717831+0000","last_fullsized":"2024-04-23T19:12:16.717831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'88","reported_seq":178,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.876527+0000","last_change":"2024-04-23T19:13:37.876563+0000","last_active":"2024-04-23T19:13:37.876527+0000","last_peered":"2024-04-23T19:13:37.876527+0000","last_clean":"2024-04-23T19:13:37.876527+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:13:37.876527+0000","last_undegraded":"2024-04-23T19:13:37.876527+0000","last_fullsized":"2024-04-23T19:13:37.876527+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'88","last_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_deep_scrub":"32'88","last_deep_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.876439+0000","objects_scrubbed":5,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:33:54.212735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.621369+0000","last_change":"2024-04-23T19:13:29.621413+0000","last_active":"2024-04-23T19:13:29.621369+0000","last_peered":"2024-04-23T19:13:29.621369+0000","last_clean":"2024-04-23T19:13:29.621369+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:13:29.621369+0000","last_undegraded":"2024-04-23T19:13:29.621369+0000","last_fullsized":"2024-04-23T19:13:29.621369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.621264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:03.242868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":163,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.495876+0000","last_change":"2024-04-23T19:10:17.516475+0000","last_active":"2024-04-23T19:13:16.495876+0000","last_peered":"2024-04-23T19:13:16.495876+0000","last_clean":"2024-04-23T19:13:16.495876+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:16.495876+0000","last_undegraded":"2024-04-23T19:13:16.495876+0000","last_fullsized":"2024-04-23T19:13:16.495876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719274+0000","last_change":"2024-04-23T19:10:09.488098+0000","last_active":"2024-04-23T19:12:21.719274+0000","last_peered":"2024-04-23T19:12:21.719274+0000","last_clean":"2024-04-23T19:12:21.719274+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:12:21.719274+0000","last_undegraded":"2024-04-23T19:12:21.719274+0000","last_fullsized":"2024-04-23T19:12:21.719274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:01.715953+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:12:01.715953+0000","last_peered":"2024-04-23T19:12:01.715953+0000","last_clean":"2024-04-23T19:12:01.715953+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:12:01.715953+0000","last_undegraded":"2024-04-23T19:12:01.715953+0000","last_fullsized":"2024-04-23T19:12:01.715953+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710873+0000","last_change":"2024-04-23T19:10:07.486483+0000","last_active":"2024-04-23T19:12:15.710873+0000","last_peered":"2024-04-23T19:12:15.710873+0000","last_clean":"2024-04-23T19:12:15.710873+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:12:15.710873+0000","last_undegraded":"2024-04-23T19:12:15.710873+0000","last_fullsized":"2024-04-23T19:12:15.710873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.891584+0000","last_change":"2024-04-23T19:13:36.891613+0000","last_active":"2024-04-23T19:13:36.891584+0000","last_peered":"2024-04-23T19:13:36.891584+0000","last_clean":"2024-04-23T19:13:36.891584+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:13:36.891584+0000","last_undegraded":"2024-04-23T19:13:36.891584+0000","last_fullsized":"2024-04-23T19:13:36.891584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.891497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:43.191904+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'290","reported_seq":515,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.753074+0000","last_change":"2024-04-23T19:13:28.753118+0000","last_active":"2024-04-23T19:13:28.753074+0000","last_peered":"2024-04-23T19:13:28.753074+0000","last_clean":"2024-04-23T19:13:28.753074+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:28.753074+0000","last_undegraded":"2024-04-23T19:13:28.753074+0000","last_fullsized":"2024-04-23T19:13:28.753074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'290","last_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_deep_scrub":"32'290","last_deep_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.752962+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:14.080605+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719191+0000","last_change":"2024-04-23T19:10:15.513032+0000","last_active":"2024-04-23T19:12:21.719191+0000","last_peered":"2024-04-23T19:12:21.719191+0000","last_clean":"2024-04-23T19:12:21.719191+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:12:21.719191+0000","last_undegraded":"2024-04-23T19:12:21.719191+0000","last_fullsized":"2024-04-23T19:12:21.719191+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719457+0000","last_change":"2024-04-23T19:10:17.518750+0000","last_active":"2024-04-23T19:12:26.719457+0000","last_peered":"2024-04-23T19:12:26.719457+0000","last_clean":"2024-04-23T19:12:26.719457+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:12:26.719457+0000","last_undegraded":"2024-04-23T19:12:26.719457+0000","last_fullsized":"2024-04-23T19:12:26.719457+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489441+0000","last_change":"2024-04-23T19:10:09.486782+0000","last_active":"2024-04-23T19:13:16.489441+0000","last_peered":"2024-04-23T19:13:16.489441+0000","last_clean":"2024-04-23T19:13:16.489441+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:13:16.489441+0000","last_undegraded":"2024-04-23T19:13:16.489441+0000","last_fullsized":"2024-04-23T19:13:16.489441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:05.714047+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:12:05.714047+0000","last_peered":"2024-04-23T19:12:05.714047+0000","last_clean":"2024-04-23T19:12:05.714047+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:12:05.714047+0000","last_undegraded":"2024-04-23T19:12:05.714047+0000","last_fullsized":"2024-04-23T19:12:05.714047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.710265+0000","last_change":"2024-04-23T19:10:07.491212+0000","last_active":"2024-04-23T19:12:10.710265+0000","last_peered":"2024-04-23T19:12:10.710265+0000","last_clean":"2024-04-23T19:12:10.710265+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:12:10.710265+0000","last_undegraded":"2024-04-23T19:12:10.710265+0000","last_fullsized":"2024-04-23T19:12:10.710265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"23'1","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.716581+0000","last_change":"2024-04-23T19:10:47.736369+0000","last_active":"2024-04-23T19:12:20.716581+0000","last_peered":"2024-04-23T19:12:20.716581+0000","last_clean":"2024-04-23T19:12:20.716581+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:12:20.716581+0000","last_undegraded":"2024-04-23T19:12:20.716581+0000","last_fullsized":"2024-04-23T19:12:20.716581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'23","reported_seq":100,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.228683+0000","last_change":"2024-04-23T19:13:38.228773+0000","last_active":"2024-04-23T19:13:38.228683+0000","last_peered":"2024-04-23T19:13:38.228683+0000","last_clean":"2024-04-23T19:13:38.228683+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:38.228683+0000","last_undegraded":"2024-04-23T19:13:38.228683+0000","last_fullsized":"2024-04-23T19:13:38.228683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'23","last_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_deep_scrub":"32'23","last_deep_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.228582+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:14.715044+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.555803+0000","last_change":"2024-04-23T19:13:26.555830+0000","last_active":"2024-04-23T19:13:26.555803+0000","last_peered":"2024-04-23T19:13:26.555803+0000","last_clean":"2024-04-23T19:13:26.555803+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:13:26.555803+0000","last_undegraded":"2024-04-23T19:13:26.555803+0000","last_fullsized":"2024-04-23T19:13:26.555803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.555714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:11.883356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720790+0000","last_change":"2024-04-23T19:10:17.517093+0000","last_active":"2024-04-23T19:12:31.720790+0000","last_peered":"2024-04-23T19:12:31.720790+0000","last_clean":"2024-04-23T19:12:31.720790+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:12:31.720790+0000","last_undegraded":"2024-04-23T19:12:31.720790+0000","last_fullsized":"2024-04-23T19:12:31.720790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408081+0000","last_change":"2024-04-23T19:10:09.477714+0000","last_active":"2024-04-23T19:13:16.408081+0000","last_peered":"2024-04-23T19:13:16.408081+0000","last_clean":"2024-04-23T19:13:16.408081+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:16.408081+0000","last_undegraded":"2024-04-23T19:13:16.408081+0000","last_fullsized":"2024-04-23T19:13:16.408081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:06.716784+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:12:06.716784+0000","last_peered":"2024-04-23T19:12:06.716784+0000","last_clean":"2024-04-23T19:12:06.716784+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:12:06.716784+0000","last_undegraded":"2024-04-23T19:12:06.716784+0000","last_fullsized":"2024-04-23T19:12:06.716784+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.715645+0000","last_change":"2024-04-23T19:10:07.482292+0000","last_active":"2024-04-23T19:12:15.715645+0000","last_peered":"2024-04-23T19:12:15.715645+0000","last_clean":"2024-04-23T19:12:15.715645+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:12:15.715645+0000","last_undegraded":"2024-04-23T19:12:15.715645+0000","last_fullsized":"2024-04-23T19:12:15.715645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710961+0000","last_change":"2024-04-23T19:10:11.490475+0000","last_active":"2024-04-23T19:12:15.710961+0000","last_peered":"2024-04-23T19:12:15.710961+0000","last_clean":"2024-04-23T19:12:15.710961+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:12:15.710961+0000","last_undegraded":"2024-04-23T19:12:15.710961+0000","last_fullsized":"2024-04-23T19:12:15.710961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'21","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.866530+0000","last_change":"2024-04-23T19:13:39.866575+0000","last_active":"2024-04-23T19:13:39.866530+0000","last_peered":"2024-04-23T19:13:39.866530+0000","last_clean":"2024-04-23T19:13:39.866530+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:39.866530+0000","last_undegraded":"2024-04-23T19:13:39.866530+0000","last_fullsized":"2024-04-23T19:13:39.866530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.866421+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:20.099121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720167+0000","last_change":"2024-04-23T19:10:15.512566+0000","last_active":"2024-04-23T19:12:26.720167+0000","last_peered":"2024-04-23T19:12:26.720167+0000","last_clean":"2024-04-23T19:12:26.720167+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:12:26.720167+0000","last_undegraded":"2024-04-23T19:12:26.720167+0000","last_fullsized":"2024-04-23T19:12:26.720167+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.403673+0000","last_change":"2024-04-23T19:10:17.515693+0000","last_active":"2024-04-23T19:13:16.403673+0000","last_peered":"2024-04-23T19:13:16.403673+0000","last_clean":"2024-04-23T19:13:16.403673+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:13:16.403673+0000","last_undegraded":"2024-04-23T19:13:16.403673+0000","last_fullsized":"2024-04-23T19:13:16.403673+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.456331+0000","last_change":"2024-04-23T19:10:09.494884+0000","last_active":"2024-04-23T19:13:16.456331+0000","last_peered":"2024-04-23T19:13:16.456331+0000","last_clean":"2024-04-23T19:13:16.456331+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:16.456331+0000","last_undegraded":"2024-04-23T19:13:16.456331+0000","last_fullsized":"2024-04-23T19:13:16.456331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.385922+0000","last_change":"2024-04-23T19:10:07.483919+0000","last_active":"2024-04-23T19:13:16.385922+0000","last_peered":"2024-04-23T19:13:16.385922+0000","last_clean":"2024-04-23T19:13:16.385922+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:13:16.385922+0000","last_undegraded":"2024-04-23T19:13:16.385922+0000","last_fullsized":"2024-04-23T19:13:16.385922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:05.709506+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:12:05.709506+0000","last_peered":"2024-04-23T19:12:05.709506+0000","last_clean":"2024-04-23T19:12:05.709506+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:12:05.709506+0000","last_undegraded":"2024-04-23T19:12:05.709506+0000","last_fullsized":"2024-04-23T19:12:05.709506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.017789+0000","last_change":"2024-04-23T19:10:47.730021+0000","last_active":"2024-04-23T19:13:23.017789+0000","last_peered":"2024-04-23T19:13:23.017789+0000","last_clean":"2024-04-23T19:13:23.017789+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:13:23.017789+0000","last_undegraded":"2024-04-23T19:13:23.017789+0000","last_fullsized":"2024-04-23T19:13:23.017789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'75","reported_seq":151,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.702503+0000","last_change":"2024-04-23T19:13:34.702548+0000","last_active":"2024-04-23T19:13:34.702503+0000","last_peered":"2024-04-23T19:13:34.702503+0000","last_clean":"2024-04-23T19:13:34.702503+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:13:34.702503+0000","last_undegraded":"2024-04-23T19:13:34.702503+0000","last_fullsized":"2024-04-23T19:13:34.702503+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'75","last_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_deep_scrub":"32'75","last_deep_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.702393+0000","objects_scrubbed":9,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:50.863938+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.162541+0000","last_change":"2024-04-23T19:13:31.162563+0000","last_active":"2024-04-23T19:13:31.162541+0000","last_peered":"2024-04-23T19:13:31.162541+0000","last_clean":"2024-04-23T19:13:31.162541+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:13:31.162541+0000","last_undegraded":"2024-04-23T19:13:31.162541+0000","last_fullsized":"2024-04-23T19:13:31.162541+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.162449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:47.422034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.163104+0000","last_change":"2024-04-23T19:10:17.506903+0000","last_active":"2024-04-23T19:13:16.163104+0000","last_peered":"2024-04-23T19:13:16.163104+0000","last_clean":"2024-04-23T19:13:16.163104+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:16.163104+0000","last_undegraded":"2024-04-23T19:13:16.163104+0000","last_fullsized":"2024-04-23T19:13:16.163104+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.456339+0000","last_change":"2024-04-23T19:10:09.492787+0000","last_active":"2024-04-23T19:13:16.456339+0000","last_peered":"2024-04-23T19:13:16.456339+0000","last_clean":"2024-04-23T19:13:16.456339+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:16.456339+0000","last_undegraded":"2024-04-23T19:13:16.456339+0000","last_fullsized":"2024-04-23T19:13:16.456339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710939+0000","last_change":"2024-04-23T19:10:07.486351+0000","last_active":"2024-04-23T19:12:15.710939+0000","last_peered":"2024-04-23T19:12:15.710939+0000","last_clean":"2024-04-23T19:12:15.710939+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:12:15.710939+0000","last_undegraded":"2024-04-23T19:12:15.710939+0000","last_fullsized":"2024-04-23T19:12:15.710939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:06.716874+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:12:06.716874+0000","last_peered":"2024-04-23T19:12:06.716874+0000","last_clean":"2024-04-23T19:12:06.716874+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:12:06.716874+0000","last_undegraded":"2024-04-23T19:12:06.716874+0000","last_fullsized":"2024-04-23T19:12:06.716874+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.046332+0000","last_change":"2024-04-23T19:13:31.046354+0000","last_active":"2024-04-23T19:13:31.046332+0000","last_peered":"2024-04-23T19:13:31.046332+0000","last_clean":"2024-04-23T19:13:31.046332+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:13:31.046332+0000","last_undegraded":"2024-04-23T19:13:31.046332+0000","last_fullsized":"2024-04-23T19:13:31.046332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.046239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:01.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'22","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.762294+0000","last_change":"2024-04-23T19:13:29.762358+0000","last_active":"2024-04-23T19:13:29.762294+0000","last_peered":"2024-04-23T19:13:29.762294+0000","last_clean":"2024-04-23T19:13:29.762294+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:13:29.762294+0000","last_undegraded":"2024-04-23T19:13:29.762294+0000","last_fullsized":"2024-04-23T19:13:29.762294+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'22","last_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_deep_scrub":"32'22","last_deep_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.762203+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:37.849358+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'285","reported_seq":367,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.782353+0000","last_change":"2024-04-23T19:13:30.782396+0000","last_active":"2024-04-23T19:13:30.782353+0000","last_peered":"2024-04-23T19:13:30.782353+0000","last_clean":"2024-04-23T19:13:30.782353+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:30.782353+0000","last_undegraded":"2024-04-23T19:13:30.782353+0000","last_fullsized":"2024-04-23T19:13:30.782353+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.782266+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:07.268727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720909+0000","last_change":"2024-04-23T19:10:17.513890+0000","last_active":"2024-04-23T19:12:31.720909+0000","last_peered":"2024-04-23T19:12:31.720909+0000","last_clean":"2024-04-23T19:12:31.720909+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:12:31.720909+0000","last_undegraded":"2024-04-23T19:12:31.720909+0000","last_fullsized":"2024-04-23T19:12:31.720909+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489441+0000","last_change":"2024-04-23T19:10:09.496860+0000","last_active":"2024-04-23T19:13:16.489441+0000","last_peered":"2024-04-23T19:13:16.489441+0000","last_clean":"2024-04-23T19:13:16.489441+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:13:16.489441+0000","last_undegraded":"2024-04-23T19:13:16.489441+0000","last_fullsized":"2024-04-23T19:13:16.489441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718497+0000","last_change":"2024-04-23T19:10:07.493532+0000","last_active":"2024-04-23T19:12:16.718497+0000","last_peered":"2024-04-23T19:12:16.718497+0000","last_clean":"2024-04-23T19:12:16.718497+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:12:16.718497+0000","last_undegraded":"2024-04-23T19:12:16.718497+0000","last_fullsized":"2024-04-23T19:12:16.718497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.710374+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:12:10.710374+0000","last_peered":"2024-04-23T19:12:10.710374+0000","last_clean":"2024-04-23T19:12:10.710374+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:12:10.710374+0000","last_undegraded":"2024-04-23T19:12:10.710374+0000","last_fullsized":"2024-04-23T19:12:10.710374+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.862675+0000","last_change":"2024-04-23T19:13:38.862700+0000","last_active":"2024-04-23T19:13:38.862675+0000","last_peered":"2024-04-23T19:13:38.862675+0000","last_clean":"2024-04-23T19:13:38.862675+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:13:38.862675+0000","last_undegraded":"2024-04-23T19:13:38.862675+0000","last_fullsized":"2024-04-23T19:13:38.862675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.862584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:18.385955+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"30'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719115+0000","last_change":"2024-04-23T19:10:13.503998+0000","last_active":"2024-04-23T19:12:21.719115+0000","last_peered":"2024-04-23T19:12:21.719115+0000","last_clean":"2024-04-23T19:12:21.719115+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:12:21.719115+0000","last_undegraded":"2024-04-23T19:12:21.719115+0000","last_fullsized":"2024-04-23T19:12:21.719115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'285","reported_seq":433,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.196390+0000","last_change":"2024-04-23T19:10:15.509417+0000","last_active":"2024-04-23T19:13:23.196390+0000","last_peered":"2024-04-23T19:13:23.196390+0000","last_clean":"2024-04-23T19:13:23.196390+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:23.196390+0000","last_undegraded":"2024-04-23T19:13:23.196390+0000","last_fullsized":"2024-04-23T19:13:23.196390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720369+0000","last_change":"2024-04-23T19:10:17.516808+0000","last_active":"2024-04-23T19:12:31.720369+0000","last_peered":"2024-04-23T19:12:31.720369+0000","last_clean":"2024-04-23T19:12:31.720369+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:12:31.720369+0000","last_undegraded":"2024-04-23T19:12:31.720369+0000","last_fullsized":"2024-04-23T19:12:31.720369+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458957+0000","last_change":"2024-04-23T19:10:09.493717+0000","last_active":"2024-04-23T19:13:16.458957+0000","last_peered":"2024-04-23T19:13:16.458957+0000","last_clean":"2024-04-23T19:13:16.458957+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:16.458957+0000","last_undegraded":"2024-04-23T19:13:16.458957+0000","last_fullsized":"2024-04-23T19:13:16.458957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.717876+0000","last_change":"2024-04-23T19:10:07.500858+0000","last_active":"2024-04-23T19:12:16.717876+0000","last_peered":"2024-04-23T19:12:16.717876+0000","last_clean":"2024-04-23T19:12:16.717876+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:12:16.717876+0000","last_undegraded":"2024-04-23T19:12:16.717876+0000","last_fullsized":"2024-04-23T19:12:16.717876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717695+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:12:11.717695+0000","last_peered":"2024-04-23T19:12:11.717695+0000","last_clean":"2024-04-23T19:12:11.717695+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:12:11.717695+0000","last_undegraded":"2024-04-23T19:12:11.717695+0000","last_fullsized":"2024-04-23T19:12:11.717695+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.119744+0000","last_change":"2024-04-23T19:13:29.119771+0000","last_active":"2024-04-23T19:13:29.119744+0000","last_peered":"2024-04-23T19:13:29.119744+0000","last_clean":"2024-04-23T19:13:29.119744+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:13:29.119744+0000","last_undegraded":"2024-04-23T19:13:29.119744+0000","last_fullsized":"2024-04-23T19:13:29.119744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.119655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:06.217082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'98","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.195210+0000","last_change":"2024-04-23T19:13:35.195253+0000","last_active":"2024-04-23T19:13:35.195210+0000","last_peered":"2024-04-23T19:13:35.195210+0000","last_clean":"2024-04-23T19:13:35.195210+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:13:35.195210+0000","last_undegraded":"2024-04-23T19:13:35.195210+0000","last_fullsized":"2024-04-23T19:13:35.195210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'98","last_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_deep_scrub":"32'98","last_deep_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.195120+0000","objects_scrubbed":8,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:11.071311+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4316,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":98,"num_read_kb":92,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.664505+0000","last_change":"2024-04-23T19:13:30.664532+0000","last_active":"2024-04-23T19:13:30.664505+0000","last_peered":"2024-04-23T19:13:30.664505+0000","last_clean":"2024-04-23T19:13:30.664505+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:13:30.664505+0000","last_undegraded":"2024-04-23T19:13:30.664505+0000","last_fullsized":"2024-04-23T19:13:30.664505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.664414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:40.569067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.161812+0000","last_change":"2024-04-23T19:13:45.161859+0000","last_active":"2024-04-23T19:13:45.161812+0000","last_peered":"2024-04-23T19:13:45.161812+0000","last_clean":"2024-04-23T19:13:45.161812+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:13:45.161812+0000","last_undegraded":"2024-04-23T19:13:45.161812+0000","last_fullsized":"2024-04-23T19:13:45.161812+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.161710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:30.545797+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.187419+0000","last_change":"2024-04-23T19:13:37.187444+0000","last_active":"2024-04-23T19:13:37.187419+0000","last_peered":"2024-04-23T19:13:37.187419+0000","last_clean":"2024-04-23T19:13:37.187419+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:13:37.187419+0000","last_undegraded":"2024-04-23T19:13:37.187419+0000","last_fullsized":"2024-04-23T19:13:37.187419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.187313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.274468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:30.718176+0000","last_change":"2024-04-23T19:10:09.492213+0000","last_active":"2024-04-23T19:12:30.718176+0000","last_peered":"2024-04-23T19:12:30.718176+0000","last_clean":"2024-04-23T19:12:30.718176+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:12:30.718176+0000","last_undegraded":"2024-04-23T19:12:30.718176+0000","last_fullsized":"2024-04-23T19:12:30.718176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720068+0000","last_change":"2024-04-23T19:10:07.500107+0000","last_active":"2024-04-23T19:12:26.720068+0000","last_peered":"2024-04-23T19:12:26.720068+0000","last_clean":"2024-04-23T19:12:26.720068+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:12:26.720068+0000","last_undegraded":"2024-04-23T19:12:26.720068+0000","last_fullsized":"2024-04-23T19:12:26.720068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720337+0000","last_change":"2024-04-23T19:10:47.725041+0000","last_active":"2024-04-23T19:12:31.720337+0000","last_peered":"2024-04-23T19:12:31.720337+0000","last_clean":"2024-04-23T19:12:31.720337+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:12:31.720337+0000","last_undegraded":"2024-04-23T19:12:31.720337+0000","last_fullsized":"2024-04-23T19:12:31.720337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.033330+0000","last_change":"2024-04-23T19:13:32.033364+0000","last_active":"2024-04-23T19:13:32.033330+0000","last_peered":"2024-04-23T19:13:32.033330+0000","last_clean":"2024-04-23T19:13:32.033330+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:13:32.033330+0000","last_undegraded":"2024-04-23T19:13:32.033330+0000","last_fullsized":"2024-04-23T19:13:32.033330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.033246+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:16.932749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721466+0000","last_change":"2024-04-23T19:10:15.514289+0000","last_active":"2024-04-23T19:12:36.721466+0000","last_peered":"2024-04-23T19:12:36.721466+0000","last_clean":"2024-04-23T19:12:36.721466+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:12:36.721466+0000","last_undegraded":"2024-04-23T19:12:36.721466+0000","last_fullsized":"2024-04-23T19:12:36.721466+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.711683+0000","last_change":"2024-04-23T19:10:17.514011+0000","last_active":"2024-04-23T19:12:20.711683+0000","last_peered":"2024-04-23T19:12:20.711683+0000","last_clean":"2024-04-23T19:12:20.711683+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:12:20.711683+0000","last_undegraded":"2024-04-23T19:12:20.711683+0000","last_fullsized":"2024-04-23T19:12:20.711683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720290+0000","last_change":"2024-04-23T19:10:47.732302+0000","last_active":"2024-04-23T19:12:31.720290+0000","last_peered":"2024-04-23T19:12:31.720290+0000","last_clean":"2024-04-23T19:12:31.720290+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:12:31.720290+0000","last_undegraded":"2024-04-23T19:12:31.720290+0000","last_fullsized":"2024-04-23T19:12:31.720290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.142280+0000","last_change":"2024-04-23T19:10:07.493914+0000","last_active":"2024-04-23T19:13:16.142280+0000","last_peered":"2024-04-23T19:13:16.142280+0000","last_clean":"2024-04-23T19:13:16.142280+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:13:16.142280+0000","last_undegraded":"2024-04-23T19:13:16.142280+0000","last_fullsized":"2024-04-23T19:13:16.142280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.891432+0000","last_change":"2024-04-23T19:13:35.891471+0000","last_active":"2024-04-23T19:13:35.891432+0000","last_peered":"2024-04-23T19:13:35.891432+0000","last_clean":"2024-04-23T19:13:35.891432+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:13:35.891432+0000","last_undegraded":"2024-04-23T19:13:35.891432+0000","last_fullsized":"2024-04-23T19:13:35.891432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.891324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:22.760900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.738794+0000","last_change":"2024-04-23T19:13:37.738834+0000","last_active":"2024-04-23T19:13:37.738794+0000","last_peered":"2024-04-23T19:13:37.738794+0000","last_clean":"2024-04-23T19:13:37.738794+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:37.738794+0000","last_undegraded":"2024-04-23T19:13:37.738794+0000","last_fullsized":"2024-04-23T19:13:37.738794+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.738703+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:59.027468+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.218628+0000","last_change":"2024-04-23T19:13:33.218654+0000","last_active":"2024-04-23T19:13:33.218628+0000","last_peered":"2024-04-23T19:13:33.218628+0000","last_clean":"2024-04-23T19:13:33.218628+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:13:33.218628+0000","last_undegraded":"2024-04-23T19:13:33.218628+0000","last_fullsized":"2024-04-23T19:13:33.218628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.218534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:31.120188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:50.715910+0000","last_change":"2024-04-23T19:10:17.520788+0000","last_active":"2024-04-23T19:12:50.715910+0000","last_peered":"2024-04-23T19:12:50.715910+0000","last_clean":"2024-04-23T19:12:50.715910+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:12:50.715910+0000","last_undegraded":"2024-04-23T19:12:50.715910+0000","last_fullsized":"2024-04-23T19:12:50.715910+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408335+0000","last_change":"2024-04-23T19:10:09.491571+0000","last_active":"2024-04-23T19:13:16.408335+0000","last_peered":"2024-04-23T19:13:16.408335+0000","last_clean":"2024-04-23T19:13:16.408335+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:16.408335+0000","last_undegraded":"2024-04-23T19:13:16.408335+0000","last_fullsized":"2024-04-23T19:13:16.408335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718941+0000","last_change":"2024-04-23T19:10:07.500769+0000","last_active":"2024-04-23T19:12:35.718941+0000","last_peered":"2024-04-23T19:12:35.718941+0000","last_clean":"2024-04-23T19:12:35.718941+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:12:35.718941+0000","last_undegraded":"2024-04-23T19:12:35.718941+0000","last_fullsized":"2024-04-23T19:12:35.718941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.223615+0000","last_change":"2024-04-23T19:13:43.223641+0000","last_active":"2024-04-23T19:13:43.223615+0000","last_peered":"2024-04-23T19:13:43.223615+0000","last_clean":"2024-04-23T19:13:43.223615+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:13:43.223615+0000","last_undegraded":"2024-04-23T19:13:43.223615+0000","last_fullsized":"2024-04-23T19:13:43.223615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.223515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:59.385060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'47","reported_seq":119,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.709428+0000","last_change":"2024-04-23T19:13:40.709462+0000","last_active":"2024-04-23T19:13:40.709428+0000","last_peered":"2024-04-23T19:13:40.709428+0000","last_clean":"2024-04-23T19:13:40.709428+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:40.709428+0000","last_undegraded":"2024-04-23T19:13:40.709428+0000","last_fullsized":"2024-04-23T19:13:40.709428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'47","last_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_deep_scrub":"32'47","last_deep_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.709338+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:40.827875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.737111+0000","last_change":"2024-04-23T19:13:27.737136+0000","last_active":"2024-04-23T19:13:27.737111+0000","last_peered":"2024-04-23T19:13:27.737111+0000","last_clean":"2024-04-23T19:13:27.737111+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:13:27.737111+0000","last_undegraded":"2024-04-23T19:13:27.737111+0000","last_fullsized":"2024-04-23T19:13:27.737111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.737016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:03.613207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:46.723450+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:12:46.723450+0000","last_peered":"2024-04-23T19:12:46.723450+0000","last_clean":"2024-04-23T19:12:46.723450+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:12:46.723450+0000","last_undegraded":"2024-04-23T19:12:46.723450+0000","last_fullsized":"2024-04-23T19:12:46.723450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'9","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536654+0000","last_change":"2024-04-23T19:10:09.498393+0000","last_active":"2024-04-23T19:13:16.536654+0000","last_peered":"2024-04-23T19:13:16.536654+0000","last_clean":"2024-04-23T19:13:16.536654+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:13:16.536654+0000","last_undegraded":"2024-04-23T19:13:16.536654+0000","last_fullsized":"2024-04-23T19:13:16.536654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720518+0000","last_change":"2024-04-23T19:10:07.496451+0000","last_active":"2024-04-23T19:12:31.720518+0000","last_peered":"2024-04-23T19:12:31.720518+0000","last_clean":"2024-04-23T19:12:31.720518+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:12:31.720518+0000","last_undegraded":"2024-04-23T19:12:31.720518+0000","last_fullsized":"2024-04-23T19:12:31.720518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.092812+0000","last_change":"2024-04-23T19:13:30.092835+0000","last_active":"2024-04-23T19:13:30.092812+0000","last_peered":"2024-04-23T19:13:30.092812+0000","last_clean":"2024-04-23T19:13:30.092812+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:30.092812+0000","last_undegraded":"2024-04-23T19:13:30.092812+0000","last_fullsized":"2024-04-23T19:13:30.092812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.092707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:02.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'38","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.254489+0000","last_change":"2024-04-23T19:13:42.254523+0000","last_active":"2024-04-23T19:13:42.254489+0000","last_peered":"2024-04-23T19:13:42.254489+0000","last_clean":"2024-04-23T19:13:42.254489+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:13:42.254489+0000","last_undegraded":"2024-04-23T19:13:42.254489+0000","last_fullsized":"2024-04-23T19:13:42.254489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.254400+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:59.255554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":53,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.124596+0000","last_change":"2024-04-23T19:13:27.124622+0000","last_active":"2024-04-23T19:13:27.124596+0000","last_peered":"2024-04-23T19:13:27.124596+0000","last_clean":"2024-04-23T19:13:27.124596+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:13:27.124596+0000","last_undegraded":"2024-04-23T19:13:27.124596+0000","last_fullsized":"2024-04-23T19:13:27.124596+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.124488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:56.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:55.721838+0000","last_change":"2024-04-23T19:10:47.728408+0000","last_active":"2024-04-23T19:12:55.721838+0000","last_peered":"2024-04-23T19:12:55.721838+0000","last_clean":"2024-04-23T19:12:55.721838+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:12:55.721838+0000","last_undegraded":"2024-04-23T19:12:55.721838+0000","last_fullsized":"2024-04-23T19:12:55.721838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720691+0000","last_change":"2024-04-23T19:10:07.503907+0000","last_active":"2024-04-23T19:12:31.720691+0000","last_peered":"2024-04-23T19:12:31.720691+0000","last_clean":"2024-04-23T19:12:31.720691+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:12:31.720691+0000","last_undegraded":"2024-04-23T19:12:31.720691+0000","last_fullsized":"2024-04-23T19:12:31.720691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.710411+0000","last_change":"2024-04-23T19:10:11.490872+0000","last_active":"2024-04-23T19:12:10.710411+0000","last_peered":"2024-04-23T19:12:10.710411+0000","last_clean":"2024-04-23T19:12:10.710411+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:12:10.710411+0000","last_undegraded":"2024-04-23T19:12:10.710411+0000","last_fullsized":"2024-04-23T19:12:10.710411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.697721+0000","last_change":"2024-04-23T19:13:38.697755+0000","last_active":"2024-04-23T19:13:38.697721+0000","last_peered":"2024-04-23T19:13:38.697721+0000","last_clean":"2024-04-23T19:13:38.697721+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:13:38.697721+0000","last_undegraded":"2024-04-23T19:13:38.697721+0000","last_fullsized":"2024-04-23T19:13:38.697721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.697637+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:22.296607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.201793+0000","last_change":"2024-04-23T19:13:32.201821+0000","last_active":"2024-04-23T19:13:32.201793+0000","last_peered":"2024-04-23T19:13:32.201793+0000","last_clean":"2024-04-23T19:13:32.201793+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:13:32.201793+0000","last_undegraded":"2024-04-23T19:13:32.201793+0000","last_fullsized":"2024-04-23T19:13:32.201793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.201694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:43.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.714595+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:40.714595+0000","last_peered":"2024-04-23T19:12:40.714595+0000","last_clean":"2024-04-23T19:12:40.714595+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:12:40.714595+0000","last_undegraded":"2024-04-23T19:12:40.714595+0000","last_fullsized":"2024-04-23T19:12:40.714595+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489507+0000","last_change":"2024-04-23T19:10:09.482621+0000","last_active":"2024-04-23T19:13:16.489507+0000","last_peered":"2024-04-23T19:13:16.489507+0000","last_clean":"2024-04-23T19:13:16.489507+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:13:16.489507+0000","last_undegraded":"2024-04-23T19:13:16.489507+0000","last_fullsized":"2024-04-23T19:13:16.489507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.105129+0000","last_change":"2024-04-23T19:10:07.493942+0000","last_active":"2024-04-23T19:13:23.105129+0000","last_peered":"2024-04-23T19:13:23.105129+0000","last_clean":"2024-04-23T19:13:23.105129+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:13:23.105129+0000","last_undegraded":"2024-04-23T19:13:23.105129+0000","last_fullsized":"2024-04-23T19:13:23.105129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.715753+0000","last_change":"2024-04-23T19:10:11.486920+0000","last_active":"2024-04-23T19:12:15.715753+0000","last_peered":"2024-04-23T19:12:15.715753+0000","last_clean":"2024-04-23T19:12:15.715753+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:12:15.715753+0000","last_undegraded":"2024-04-23T19:12:15.715753+0000","last_fullsized":"2024-04-23T19:12:15.715753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'39","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.079598+0000","last_change":"2024-04-23T19:10:13.502855+0000","last_active":"2024-04-23T19:13:23.079598+0000","last_peered":"2024-04-23T19:13:23.079598+0000","last_clean":"2024-04-23T19:13:23.079598+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:13:23.079598+0000","last_undegraded":"2024-04-23T19:13:23.079598+0000","last_fullsized":"2024-04-23T19:13:23.079598+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:46.722859+0000","last_change":"2024-04-23T19:10:15.510094+0000","last_active":"2024-04-23T19:12:46.722859+0000","last_peered":"2024-04-23T19:12:46.722859+0000","last_clean":"2024-04-23T19:12:46.722859+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:12:46.722859+0000","last_undegraded":"2024-04-23T19:12:46.722859+0000","last_fullsized":"2024-04-23T19:12:46.722859+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721579+0000","last_change":"2024-04-23T19:10:17.517391+0000","last_active":"2024-04-23T19:12:36.721579+0000","last_peered":"2024-04-23T19:12:36.721579+0000","last_clean":"2024-04-23T19:12:36.721579+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:12:36.721579+0000","last_undegraded":"2024-04-23T19:12:36.721579+0000","last_fullsized":"2024-04-23T19:12:36.721579+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:20.724924+0000","last_change":"2024-04-23T19:10:47.733611+0000","last_active":"2024-04-23T19:13:20.724924+0000","last_peered":"2024-04-23T19:13:20.724924+0000","last_clean":"2024-04-23T19:13:20.724924+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:13:20.724924+0000","last_undegraded":"2024-04-23T19:13:20.724924+0000","last_fullsized":"2024-04-23T19:13:20.724924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721300+0000","last_change":"2024-04-23T19:10:07.502477+0000","last_active":"2024-04-23T19:12:36.721300+0000","last_peered":"2024-04-23T19:12:36.721300+0000","last_clean":"2024-04-23T19:12:36.721300+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:12:36.721300+0000","last_undegraded":"2024-04-23T19:12:36.721300+0000","last_fullsized":"2024-04-23T19:12:36.721300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719797+0000","last_change":"2024-04-23T19:10:11.487445+0000","last_active":"2024-04-23T19:12:40.719797+0000","last_peered":"2024-04-23T19:12:40.719797+0000","last_clean":"2024-04-23T19:12:40.719797+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:12:40.719797+0000","last_undegraded":"2024-04-23T19:12:40.719797+0000","last_fullsized":"2024-04-23T19:12:40.719797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.723348+0000","last_change":"2024-04-23T19:13:36.723412+0000","last_active":"2024-04-23T19:13:36.723348+0000","last_peered":"2024-04-23T19:13:36.723348+0000","last_clean":"2024-04-23T19:13:36.723348+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:13:36.723348+0000","last_undegraded":"2024-04-23T19:13:36.723348+0000","last_fullsized":"2024-04-23T19:13:36.723348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.723239+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:22.107325+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.719921+0000","last_change":"2024-04-23T19:13:39.719943+0000","last_active":"2024-04-23T19:13:39.719921+0000","last_peered":"2024-04-23T19:13:39.719921+0000","last_clean":"2024-04-23T19:13:39.719921+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:13:39.719921+0000","last_undegraded":"2024-04-23T19:13:39.719921+0000","last_fullsized":"2024-04-23T19:13:39.719921+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:33.961424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.462496+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.462496+0000","last_peered":"2024-04-23T19:13:16.462496+0000","last_clean":"2024-04-23T19:13:16.462496+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.462496+0000","last_undegraded":"2024-04-23T19:13:16.462496+0000","last_fullsized":"2024-04-23T19:13:16.462496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.408350+0000","last_change":"2024-04-23T19:10:09.477657+0000","last_active":"2024-04-23T19:13:16.408350+0000","last_peered":"2024-04-23T19:13:16.408350+0000","last_clean":"2024-04-23T19:13:16.408350+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:16.408350+0000","last_undegraded":"2024-04-23T19:13:16.408350+0000","last_fullsized":"2024-04-23T19:13:16.408350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.121043+0000","last_change":"2024-04-23T19:10:07.504027+0000","last_active":"2024-04-23T19:13:16.121043+0000","last_peered":"2024-04-23T19:13:16.121043+0000","last_clean":"2024-04-23T19:13:16.121043+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:13:16.121043+0000","last_undegraded":"2024-04-23T19:13:16.121043+0000","last_fullsized":"2024-04-23T19:13:16.121043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'4","reported_seq":229,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.005492+0000","last_change":"2024-04-23T19:10:11.485789+0000","last_active":"2024-04-23T19:13:23.005492+0000","last_peered":"2024-04-23T19:13:23.005492+0000","last_clean":"2024-04-23T19:13:23.005492+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:13:23.005492+0000","last_undegraded":"2024-04-23T19:13:23.005492+0000","last_fullsized":"2024-04-23T19:13:23.005492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"30'10","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718473+0000","last_change":"2024-04-23T19:10:13.502626+0000","last_active":"2024-04-23T19:12:16.718473+0000","last_peered":"2024-04-23T19:12:16.718473+0000","last_clean":"2024-04-23T19:12:16.718473+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:12:16.718473+0000","last_undegraded":"2024-04-23T19:12:16.718473+0000","last_fullsized":"2024-04-23T19:12:16.718473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'285","reported_seq":429,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.137923+0000","last_change":"2024-04-23T19:13:29.137953+0000","last_active":"2024-04-23T19:13:29.137923+0000","last_peered":"2024-04-23T19:13:29.137923+0000","last_clean":"2024-04-23T19:13:29.137923+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:29.137923+0000","last_undegraded":"2024-04-23T19:13:29.137923+0000","last_fullsized":"2024-04-23T19:13:29.137923+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.137819+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:17.728927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721419+0000","last_change":"2024-04-23T19:10:17.513830+0000","last_active":"2024-04-23T19:12:36.721419+0000","last_peered":"2024-04-23T19:12:36.721419+0000","last_clean":"2024-04-23T19:12:36.721419+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:12:36.721419+0000","last_undegraded":"2024-04-23T19:12:36.721419+0000","last_fullsized":"2024-04-23T19:12:36.721419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719755+0000","last_change":"2024-04-23T19:10:09.476601+0000","last_active":"2024-04-23T19:12:40.719755+0000","last_peered":"2024-04-23T19:12:40.719755+0000","last_clean":"2024-04-23T19:12:40.719755+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:12:40.719755+0000","last_undegraded":"2024-04-23T19:12:40.719755+0000","last_fullsized":"2024-04-23T19:12:40.719755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721362+0000","last_change":"2024-04-23T19:10:07.497992+0000","last_active":"2024-04-23T19:12:36.721362+0000","last_peered":"2024-04-23T19:12:36.721362+0000","last_clean":"2024-04-23T19:12:36.721362+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:12:36.721362+0000","last_undegraded":"2024-04-23T19:12:36.721362+0000","last_fullsized":"2024-04-23T19:12:36.721362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722048+0000","last_change":"2024-04-23T19:10:11.489334+0000","last_active":"2024-04-23T19:12:41.722048+0000","last_peered":"2024-04-23T19:12:41.722048+0000","last_clean":"2024-04-23T19:12:41.722048+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:12:41.722048+0000","last_undegraded":"2024-04-23T19:12:41.722048+0000","last_fullsized":"2024-04-23T19:12:41.722048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.806873+0000","last_change":"2024-04-23T19:13:31.806949+0000","last_active":"2024-04-23T19:13:31.806873+0000","last_peered":"2024-04-23T19:13:31.806873+0000","last_clean":"2024-04-23T19:13:31.806873+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:13:31.806873+0000","last_undegraded":"2024-04-23T19:13:31.806873+0000","last_fullsized":"2024-04-23T19:13:31.806873+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.806771+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:55.900270+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.655391+0000","last_change":"2024-04-23T19:13:31.655412+0000","last_active":"2024-04-23T19:13:31.655391+0000","last_peered":"2024-04-23T19:13:31.655391+0000","last_clean":"2024-04-23T19:13:31.655391+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:13:31.655391+0000","last_undegraded":"2024-04-23T19:13:31.655391+0000","last_fullsized":"2024-04-23T19:13:31.655391+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.655302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:46.303379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722147+0000","last_change":"2024-04-23T19:10:17.518482+0000","last_active":"2024-04-23T19:12:41.722147+0000","last_peered":"2024-04-23T19:12:41.722147+0000","last_clean":"2024-04-23T19:12:41.722147+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:12:41.722147+0000","last_undegraded":"2024-04-23T19:12:41.722147+0000","last_fullsized":"2024-04-23T19:12:41.722147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'3","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536232+0000","last_change":"2024-04-23T19:10:09.477644+0000","last_active":"2024-04-23T19:13:16.536232+0000","last_peered":"2024-04-23T19:13:16.536232+0000","last_clean":"2024-04-23T19:13:16.536232+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:16.536232+0000","last_undegraded":"2024-04-23T19:13:16.536232+0000","last_fullsized":"2024-04-23T19:13:16.536232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721726+0000","last_change":"2024-04-23T19:10:07.503802+0000","last_active":"2024-04-23T19:12:36.721726+0000","last_peered":"2024-04-23T19:12:36.721726+0000","last_clean":"2024-04-23T19:12:36.721726+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:12:36.721726+0000","last_undegraded":"2024-04-23T19:12:36.721726+0000","last_fullsized":"2024-04-23T19:12:36.721726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.719773+0000","last_change":"2024-04-23T19:10:11.489940+0000","last_active":"2024-04-23T19:12:40.719773+0000","last_peered":"2024-04-23T19:12:40.719773+0000","last_clean":"2024-04-23T19:12:40.719773+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:12:40.719773+0000","last_undegraded":"2024-04-23T19:12:40.719773+0000","last_fullsized":"2024-04-23T19:12:40.719773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.718061+0000","last_change":"2024-04-23T19:13:33.718094+0000","last_active":"2024-04-23T19:13:33.718061+0000","last_peered":"2024-04-23T19:13:33.718061+0000","last_clean":"2024-04-23T19:13:33.718061+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:13:33.718061+0000","last_undegraded":"2024-04-23T19:13:33.718061+0000","last_fullsized":"2024-04-23T19:13:33.718061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.717974+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:50.719128+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722435+0000","last_change":"2024-04-23T19:10:15.514687+0000","last_active":"2024-04-23T19:12:41.722435+0000","last_peered":"2024-04-23T19:12:41.722435+0000","last_clean":"2024-04-23T19:12:41.722435+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:12:41.722435+0000","last_undegraded":"2024-04-23T19:12:41.722435+0000","last_fullsized":"2024-04-23T19:12:41.722435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722640+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:41.722640+0000","last_peered":"2024-04-23T19:12:41.722640+0000","last_clean":"2024-04-23T19:12:41.722640+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:41.722640+0000","last_undegraded":"2024-04-23T19:12:41.722640+0000","last_fullsized":"2024-04-23T19:12:41.722640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'11","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.537050+0000","last_change":"2024-04-23T19:10:09.490516+0000","last_active":"2024-04-23T19:13:16.537050+0000","last_peered":"2024-04-23T19:13:16.537050+0000","last_clean":"2024-04-23T19:13:16.537050+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:13:16.537050+0000","last_undegraded":"2024-04-23T19:13:16.537050+0000","last_fullsized":"2024-04-23T19:13:16.537050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.710387+0000","last_change":"2024-04-23T19:10:07.486770+0000","last_active":"2024-04-23T19:12:10.710387+0000","last_peered":"2024-04-23T19:12:10.710387+0000","last_clean":"2024-04-23T19:12:10.710387+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:12:10.710387+0000","last_undegraded":"2024-04-23T19:12:10.710387+0000","last_fullsized":"2024-04-23T19:12:10.710387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.929784+0000","last_change":"2024-04-23T19:13:36.929808+0000","last_active":"2024-04-23T19:13:36.929784+0000","last_peered":"2024-04-23T19:13:36.929784+0000","last_clean":"2024-04-23T19:13:36.929784+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:13:36.929784+0000","last_undegraded":"2024-04-23T19:13:36.929784+0000","last_fullsized":"2024-04-23T19:13:36.929784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.929690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:45.019264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'29","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.141607+0000","last_change":"2024-04-23T19:10:13.503113+0000","last_active":"2024-04-23T19:13:23.141607+0000","last_peered":"2024-04-23T19:13:23.141607+0000","last_clean":"2024-04-23T19:13:23.141607+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:13:23.141607+0000","last_undegraded":"2024-04-23T19:13:23.141607+0000","last_fullsized":"2024-04-23T19:13:23.141607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:46.723307+0000","last_change":"2024-04-23T19:10:15.512937+0000","last_active":"2024-04-23T19:12:46.723307+0000","last_peered":"2024-04-23T19:12:46.723307+0000","last_clean":"2024-04-23T19:12:46.723307+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:12:46.723307+0000","last_undegraded":"2024-04-23T19:12:46.723307+0000","last_fullsized":"2024-04-23T19:12:46.723307+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722581+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:41.722581+0000","last_peered":"2024-04-23T19:12:41.722581+0000","last_clean":"2024-04-23T19:12:41.722581+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:41.722581+0000","last_undegraded":"2024-04-23T19:12:41.722581+0000","last_fullsized":"2024-04-23T19:12:41.722581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.488929+0000","last_change":"2024-04-23T19:10:09.476153+0000","last_active":"2024-04-23T19:13:16.488929+0000","last_peered":"2024-04-23T19:13:16.488929+0000","last_clean":"2024-04-23T19:13:16.488929+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:16.488929+0000","last_undegraded":"2024-04-23T19:13:16.488929+0000","last_fullsized":"2024-04-23T19:13:16.488929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.104929+0000","last_change":"2024-04-23T19:10:07.501211+0000","last_active":"2024-04-23T19:13:23.104929+0000","last_peered":"2024-04-23T19:13:23.104929+0000","last_clean":"2024-04-23T19:13:23.104929+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:13:23.104929+0000","last_undegraded":"2024-04-23T19:13:23.104929+0000","last_fullsized":"2024-04-23T19:13:23.104929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'3","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.954199+0000","last_change":"2024-04-23T19:13:39.954265+0000","last_active":"2024-04-23T19:13:39.954199+0000","last_peered":"2024-04-23T19:13:39.954199+0000","last_clean":"2024-04-23T19:13:39.954199+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:39.954199+0000","last_undegraded":"2024-04-23T19:13:39.954199+0000","last_fullsized":"2024-04-23T19:13:39.954199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.954105+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:23.553074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'35","reported_seq":84,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.150538+0000","last_change":"2024-04-23T19:10:13.502743+0000","last_active":"2024-04-23T19:13:23.150538+0000","last_peered":"2024-04-23T19:13:23.150538+0000","last_clean":"2024-04-23T19:13:23.150538+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:23.150538+0000","last_undegraded":"2024-04-23T19:13:23.150538+0000","last_fullsized":"2024-04-23T19:13:23.150538+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:12.473911+0000","last_clean_scrub_stamp":"2024-04-23T19:10:12.473911+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.549277+0000","last_change":"2024-04-23T19:13:27.549295+0000","last_active":"2024-04-23T19:13:27.549277+0000","last_peered":"2024-04-23T19:13:27.549277+0000","last_clean":"2024-04-23T19:13:27.549277+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:13:27.549277+0000","last_undegraded":"2024-04-23T19:13:27.549277+0000","last_fullsized":"2024-04-23T19:13:27.549277+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.549205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:33.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.486600+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.486600+0000","last_peered":"2024-04-23T19:13:16.486600+0000","last_clean":"2024-04-23T19:13:16.486600+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.486600+0000","last_undegraded":"2024-04-23T19:13:16.486600+0000","last_fullsized":"2024-04-23T19:13:16.486600+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489318+0000","last_change":"2024-04-23T19:10:09.494269+0000","last_active":"2024-04-23T19:13:16.489318+0000","last_peered":"2024-04-23T19:13:16.489318+0000","last_clean":"2024-04-23T19:13:16.489318+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:16.489318+0000","last_undegraded":"2024-04-23T19:13:16.489318+0000","last_fullsized":"2024-04-23T19:13:16.489318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.480071+0000","last_change":"2024-04-23T19:10:07.492825+0000","last_active":"2024-04-23T19:13:16.480071+0000","last_peered":"2024-04-23T19:13:16.480071+0000","last_clean":"2024-04-23T19:13:16.480071+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:13:16.480071+0000","last_undegraded":"2024-04-23T19:13:16.480071+0000","last_fullsized":"2024-04-23T19:13:16.480071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.711704+0000","last_change":"2024-04-23T19:10:09.478781+0000","last_active":"2024-04-23T19:12:20.711704+0000","last_peered":"2024-04-23T19:12:20.711704+0000","last_clean":"2024-04-23T19:12:20.711704+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:12:20.711704+0000","last_undegraded":"2024-04-23T19:12:20.711704+0000","last_fullsized":"2024-04-23T19:12:20.711704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.104929+0000","last_change":"2024-04-23T19:10:07.489213+0000","last_active":"2024-04-23T19:13:23.104929+0000","last_peered":"2024-04-23T19:13:23.104929+0000","last_clean":"2024-04-23T19:13:23.104929+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:13:23.104929+0000","last_undegraded":"2024-04-23T19:13:23.104929+0000","last_fullsized":"2024-04-23T19:13:23.104929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.266042+0000","last_change":"2024-04-23T19:10:09.495393+0000","last_active":"2024-04-23T19:13:16.266042+0000","last_peered":"2024-04-23T19:13:16.266042+0000","last_clean":"2024-04-23T19:13:16.266042+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:13:16.266042+0000","last_undegraded":"2024-04-23T19:13:16.266042+0000","last_fullsized":"2024-04-23T19:13:16.266042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717563+0000","last_change":"2024-04-23T19:10:07.506697+0000","last_active":"2024-04-23T19:12:11.717563+0000","last_peered":"2024-04-23T19:12:11.717563+0000","last_clean":"2024-04-23T19:12:11.717563+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:12:11.717563+0000","last_undegraded":"2024-04-23T19:12:11.717563+0000","last_fullsized":"2024-04-23T19:12:11.717563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165501+0000","last_change":"2024-04-23T19:10:09.478697+0000","last_active":"2024-04-23T19:13:16.165501+0000","last_peered":"2024-04-23T19:13:16.165501+0000","last_clean":"2024-04-23T19:13:16.165501+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:13:16.165501+0000","last_undegraded":"2024-04-23T19:13:16.165501+0000","last_fullsized":"2024-04-23T19:13:16.165501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.107204+0000","last_change":"2024-04-23T19:10:07.490623+0000","last_active":"2024-04-23T19:13:23.107204+0000","last_peered":"2024-04-23T19:13:23.107204+0000","last_clean":"2024-04-23T19:13:23.107204+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:13:23.107204+0000","last_undegraded":"2024-04-23T19:13:23.107204+0000","last_fullsized":"2024-04-23T19:13:23.107204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710910+0000","last_change":"2024-04-23T19:10:09.491248+0000","last_active":"2024-04-23T19:12:15.710910+0000","last_peered":"2024-04-23T19:12:15.710910+0000","last_clean":"2024-04-23T19:12:15.710910+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:12:15.710910+0000","last_undegraded":"2024-04-23T19:12:15.710910+0000","last_fullsized":"2024-04-23T19:12:15.710910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.711661+0000","last_change":"2024-04-23T19:10:07.490715+0000","last_active":"2024-04-23T19:12:20.711661+0000","last_peered":"2024-04-23T19:12:20.711661+0000","last_clean":"2024-04-23T19:12:20.711661+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:12:20.711661+0000","last_undegraded":"2024-04-23T19:12:20.711661+0000","last_fullsized":"2024-04-23T19:12:20.711661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.456281+0000","last_change":"2024-04-23T19:10:09.494954+0000","last_active":"2024-04-23T19:13:16.456281+0000","last_peered":"2024-04-23T19:13:16.456281+0000","last_clean":"2024-04-23T19:13:16.456281+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:16.456281+0000","last_undegraded":"2024-04-23T19:13:16.456281+0000","last_fullsized":"2024-04-23T19:13:16.456281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.717850+0000","last_change":"2024-04-23T19:10:07.490753+0000","last_active":"2024-04-23T19:12:16.717850+0000","last_peered":"2024-04-23T19:12:16.717850+0000","last_clean":"2024-04-23T19:12:16.717850+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:12:16.717850+0000","last_undegraded":"2024-04-23T19:12:16.717850+0000","last_fullsized":"2024-04-23T19:12:16.717850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165756+0000","last_change":"2024-04-23T19:10:09.482692+0000","last_active":"2024-04-23T19:13:16.165756+0000","last_peered":"2024-04-23T19:13:16.165756+0000","last_clean":"2024-04-23T19:13:16.165756+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:13:16.165756+0000","last_undegraded":"2024-04-23T19:13:16.165756+0000","last_fullsized":"2024-04-23T19:13:16.165756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.383019+0000","last_change":"2024-04-23T19:10:07.500499+0000","last_active":"2024-04-23T19:13:16.383019+0000","last_peered":"2024-04-23T19:13:16.383019+0000","last_clean":"2024-04-23T19:13:16.383019+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:16.383019+0000","last_undegraded":"2024-04-23T19:13:16.383019+0000","last_fullsized":"2024-04-23T19:13:16.383019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.265314+0000","last_change":"2024-04-23T19:10:09.484981+0000","last_active":"2024-04-23T19:13:16.265314+0000","last_peered":"2024-04-23T19:13:16.265314+0000","last_clean":"2024-04-23T19:13:16.265314+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:13:16.265314+0000","last_undegraded":"2024-04-23T19:13:16.265314+0000","last_fullsized":"2024-04-23T19:13:16.265314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.711729+0000","last_change":"2024-04-23T19:10:07.490970+0000","last_active":"2024-04-23T19:12:20.711729+0000","last_peered":"2024-04-23T19:12:20.711729+0000","last_clean":"2024-04-23T19:12:20.711729+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:12:20.711729+0000","last_undegraded":"2024-04-23T19:12:20.711729+0000","last_fullsized":"2024-04-23T19:12:20.711729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720563+0000","last_change":"2024-04-23T19:10:09.484268+0000","last_active":"2024-04-23T19:12:31.720563+0000","last_peered":"2024-04-23T19:12:31.720563+0000","last_clean":"2024-04-23T19:12:31.720563+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:12:31.720563+0000","last_undegraded":"2024-04-23T19:12:31.720563+0000","last_fullsized":"2024-04-23T19:12:31.720563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718248+0000","last_change":"2024-04-23T19:10:07.507068+0000","last_active":"2024-04-23T19:12:16.718248+0000","last_peered":"2024-04-23T19:12:16.718248+0000","last_clean":"2024-04-23T19:12:16.718248+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:12:16.718248+0000","last_undegraded":"2024-04-23T19:12:16.718248+0000","last_fullsized":"2024-04-23T19:12:16.718248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489346+0000","last_change":"2024-04-23T19:10:09.498436+0000","last_active":"2024-04-23T19:13:16.489346+0000","last_peered":"2024-04-23T19:13:16.489346+0000","last_clean":"2024-04-23T19:13:16.489346+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:13:16.489346+0000","last_undegraded":"2024-04-23T19:13:16.489346+0000","last_fullsized":"2024-04-23T19:13:16.489346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712299+0000","last_change":"2024-04-23T19:10:07.489441+0000","last_active":"2024-04-23T19:12:25.712299+0000","last_peered":"2024-04-23T19:12:25.712299+0000","last_clean":"2024-04-23T19:12:25.712299+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:12:25.712299+0000","last_undegraded":"2024-04-23T19:12:25.712299+0000","last_fullsized":"2024-04-23T19:12:25.712299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'5","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536475+0000","last_change":"2024-04-23T19:10:09.492013+0000","last_active":"2024-04-23T19:13:16.536475+0000","last_peered":"2024-04-23T19:13:16.536475+0000","last_clean":"2024-04-23T19:13:16.536475+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:13:16.536475+0000","last_undegraded":"2024-04-23T19:13:16.536475+0000","last_fullsized":"2024-04-23T19:13:16.536475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.718693+0000","last_change":"2024-04-23T19:10:07.490880+0000","last_active":"2024-04-23T19:12:21.718693+0000","last_peered":"2024-04-23T19:12:21.718693+0000","last_clean":"2024-04-23T19:12:21.718693+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:12:21.718693+0000","last_undegraded":"2024-04-23T19:12:21.718693+0000","last_fullsized":"2024-04-23T19:12:21.718693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719515+0000","last_change":"2024-04-23T19:10:09.482277+0000","last_active":"2024-04-23T19:12:26.719515+0000","last_peered":"2024-04-23T19:12:26.719515+0000","last_clean":"2024-04-23T19:12:26.719515+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:12:26.719515+0000","last_undegraded":"2024-04-23T19:12:26.719515+0000","last_fullsized":"2024-04-23T19:12:26.719515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.718719+0000","last_change":"2024-04-23T19:10:07.502896+0000","last_active":"2024-04-23T19:12:21.718719+0000","last_peered":"2024-04-23T19:12:21.718719+0000","last_clean":"2024-04-23T19:12:21.718719+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:12:21.718719+0000","last_undegraded":"2024-04-23T19:12:21.718719+0000","last_fullsized":"2024-04-23T19:12:21.718719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719488+0000","last_change":"2024-04-23T19:10:09.492153+0000","last_active":"2024-04-23T19:12:26.719488+0000","last_peered":"2024-04-23T19:12:26.719488+0000","last_clean":"2024-04-23T19:12:26.719488+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:12:26.719488+0000","last_undegraded":"2024-04-23T19:12:26.719488+0000","last_fullsized":"2024-04-23T19:12:26.719488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719175+0000","last_change":"2024-04-23T19:10:07.490658+0000","last_active":"2024-04-23T19:12:21.719175+0000","last_peered":"2024-04-23T19:12:21.719175+0000","last_clean":"2024-04-23T19:12:21.719175+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:12:21.719175+0000","last_undegraded":"2024-04-23T19:12:21.719175+0000","last_fullsized":"2024-04-23T19:12:21.719175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719471+0000","last_change":"2024-04-23T19:10:09.483025+0000","last_active":"2024-04-23T19:12:26.719471+0000","last_peered":"2024-04-23T19:12:26.719471+0000","last_clean":"2024-04-23T19:12:26.719471+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:12:26.719471+0000","last_undegraded":"2024-04-23T19:12:26.719471+0000","last_fullsized":"2024-04-23T19:12:26.719471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719234+0000","last_change":"2024-04-23T19:10:07.502808+0000","last_active":"2024-04-23T19:12:21.719234+0000","last_peered":"2024-04-23T19:12:21.719234+0000","last_clean":"2024-04-23T19:12:21.719234+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:12:21.719234+0000","last_undegraded":"2024-04-23T19:12:21.719234+0000","last_fullsized":"2024-04-23T19:12:21.719234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'5","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536612+0000","last_change":"2024-04-23T19:10:09.495999+0000","last_active":"2024-04-23T19:13:16.536612+0000","last_peered":"2024-04-23T19:13:16.536612+0000","last_clean":"2024-04-23T19:13:16.536612+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:13:16.536612+0000","last_undegraded":"2024-04-23T19:13:16.536612+0000","last_fullsized":"2024-04-23T19:13:16.536612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:30.718279+0000","last_change":"2024-04-23T19:10:07.500476+0000","last_active":"2024-04-23T19:12:30.718279+0000","last_peered":"2024-04-23T19:12:30.718279+0000","last_clean":"2024-04-23T19:12:30.718279+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:12:30.718279+0000","last_undegraded":"2024-04-23T19:12:30.718279+0000","last_fullsized":"2024-04-23T19:12:30.718279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.265739+0000","last_change":"2024-04-23T19:10:09.489995+0000","last_active":"2024-04-23T19:13:16.265739+0000","last_peered":"2024-04-23T19:13:16.265739+0000","last_clean":"2024-04-23T19:13:16.265739+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:13:16.265739+0000","last_undegraded":"2024-04-23T19:13:16.265739+0000","last_fullsized":"2024-04-23T19:13:16.265739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720175+0000","last_change":"2024-04-23T19:10:07.507910+0000","last_active":"2024-04-23T19:12:26.720175+0000","last_peered":"2024-04-23T19:12:26.720175+0000","last_clean":"2024-04-23T19:12:26.720175+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:12:26.720175+0000","last_undegraded":"2024-04-23T19:12:26.720175+0000","last_fullsized":"2024-04-23T19:12:26.720175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719909+0000","last_change":"2024-04-23T19:10:07.503476+0000","last_active":"2024-04-23T19:12:26.719909+0000","last_peered":"2024-04-23T19:12:26.719909+0000","last_clean":"2024-04-23T19:12:26.719909+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:12:26.719909+0000","last_undegraded":"2024-04-23T19:12:26.719909+0000","last_fullsized":"2024-04-23T19:12:26.719909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.407993+0000","last_change":"2024-04-23T19:10:09.487033+0000","last_active":"2024-04-23T19:13:16.407993+0000","last_peered":"2024-04-23T19:13:16.407993+0000","last_clean":"2024-04-23T19:13:16.407993+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:13:16.407993+0000","last_undegraded":"2024-04-23T19:13:16.407993+0000","last_fullsized":"2024-04-23T19:13:16.407993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.395378+0000","last_change":"2024-04-23T19:10:07.486697+0000","last_active":"2024-04-23T19:13:16.395378+0000","last_peered":"2024-04-23T19:13:16.395378+0000","last_clean":"2024-04-23T19:13:16.395378+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:13:16.395378+0000","last_undegraded":"2024-04-23T19:13:16.395378+0000","last_fullsized":"2024-04-23T19:13:16.395378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.166264+0000","last_change":"2024-04-23T19:10:09.491506+0000","last_active":"2024-04-23T19:13:16.166264+0000","last_peered":"2024-04-23T19:13:16.166264+0000","last_clean":"2024-04-23T19:13:16.166264+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:16.166264+0000","last_undegraded":"2024-04-23T19:13:16.166264+0000","last_fullsized":"2024-04-23T19:13:16.166264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1730,"num_omap_keys":8,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1483,"ondisk_log_size":1483,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673011,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39948,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331892,"statfs":{"total":96636764160,"available":96595857408,"internally_reserved":0,"allocated":462848,"data_stored":144617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673010,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335504,"statfs":{"total":96636764160,"available":96599556096,"internally_reserved":0,"allocated":1024000,"data_stored":734097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":995328,"data_stored":732924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705715,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":425984,"data_stored":144396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:13:48.174 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.175 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.175 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.175 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.175 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.175 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.175 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.175 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.176 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.177 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.178 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.179 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.180 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.180 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.180 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.180 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.180 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.180 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.180 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.181 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.182 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.182 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.182 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.182 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.182 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.182 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.182 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.183 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.184 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.184 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.184 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T19:09:51.428209+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=51, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.184 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.184 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.184 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.184 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.185 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.186 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.186 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.186 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.186 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.186 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.186 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.186 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.187 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.188 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.188 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.188 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.188 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.188 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.188 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.189 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.190 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.190 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.190 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.190 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.190 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.190 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.191 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.191 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.191 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.191 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.191 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.191 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.191 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.192 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.192 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.192 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.192 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.192 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.192 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.192 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.193 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.194 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.194 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.194 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.194 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.194 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.194 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.195 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.195 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.195 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.195 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.195 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.195 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.195 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-23T19:10:12.473911+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=12, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.196 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.197 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.198 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.198 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.198 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.198 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.198 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.198 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.198 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.199 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.200 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.200 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.200 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.200 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.200 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.200 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.200 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.201 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.201 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.201 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.201 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:13:48.201 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T19:14:08.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:14:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.317+0000 7f9b407e6640 1 Processor -- start 2024-04-23T19:14:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.317+0000 7f9b407e6640 1 -- start start 2024-04-23T19:14:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b407e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b3812ec80 0x7f9b38131070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b407e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b38128410 con 0x7f9b381288d0 2024-04-23T19:14:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b407e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9b381286b0 con 0x7f9b3812ec80 2024-04-23T19:14:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b3dd81640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b3812ec80 0x7f9b38131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b3e582640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b381288d0 0x7f9b38128cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34132/0 (socket says 172.21.15.16:34132) 2024-04-23T19:14:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b3e582640 1 -- 172.21.15.16:0/677309953 learned_addr learned my addr 172.21.15.16:0/677309953 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:14:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b3dd81640 1 -- 172.21.15.16:0/677309953 >> v1:172.21.15.16:6789/0 conn(0x7f9b381288d0 legacy=0x7f9b38128cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.318+0000 7f9b3dd81640 1 -- 172.21.15.16:0/677309953 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3806de40 con 0x7f9b3812ec80 2024-04-23T19:14:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.319+0000 7f9b3dd81640 1 --2- 172.21.15.16:0/677309953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3812ec80 0x7f9b38131070 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009720 tx=0x7f9b2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fbb97300aa1d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.319+0000 7f9b3d580640 1 -- 172.21.15.16:0/677309953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b280316d0 con 0x7f9b3812ec80 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.319+0000 7f9b3d580640 1 -- 172.21.15.16:0/677309953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b28037070 con 0x7f9b3812ec80 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.319+0000 7f9b3d580640 1 -- 172.21.15.16:0/677309953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b280319c0 con 0x7f9b3812ec80 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.319+0000 7f9b407e6640 1 -- 172.21.15.16:0/677309953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3812ec80 msgr2=0x7f9b38131070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.319+0000 7f9b407e6640 1 --2- 172.21.15.16:0/677309953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3812ec80 0x7f9b38131070 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009720 tx=0x7f9b2802b840 comp rx=0 tx=0).stop 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.320+0000 7f9b407e6640 1 -- 172.21.15.16:0/677309953 shutdown_connections 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.320+0000 7f9b407e6640 1 --2- 172.21.15.16:0/677309953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3812ec80 0x7f9b38131070 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.320+0000 7f9b407e6640 1 -- 172.21.15.16:0/677309953 >> 172.21.15.16:0/677309953 conn(0x7f9b3809a200 msgr2=0x7f9b38097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.320+0000 7f9b407e6640 1 -- 172.21.15.16:0/677309953 shutdown_connections 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.320+0000 7f9b407e6640 1 -- 172.21.15.16:0/677309953 wait complete. 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.320+0000 7f9b407e6640 1 Processor -- start 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b407e6640 1 -- start start 2024-04-23T19:14:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b407e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b381288d0 0x7f9b3812c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b407e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b380945f0 con 0x7f9b381288d0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b3e582640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b381288d0 0x7f9b3812c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b3e582640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b381288d0 0x7f9b3812c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37282/0 (socket says 172.21.15.16:37282) 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b3e582640 1 -- 172.21.15.16:0/4292409517 learned_addr learned my addr 172.21.15.16:0/4292409517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b3e582640 1 -- 172.21.15.16:0/4292409517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b38072ff0 con 0x7f9b381288d0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.321+0000 7f9b3e582640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b381288d0 0x7f9b3812c800 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c4a0 tx=0x7f9b3400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.322+0000 7f9b2effd640 1 -- 172.21.15.16:0/4292409517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b34017070 con 0x7f9b381288d0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.322+0000 7f9b2effd640 1 -- 172.21.15.16:0/4292409517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b34005c10 con 0x7f9b381288d0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.322+0000 7f9b2effd640 1 -- 172.21.15.16:0/4292409517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b34007070 con 0x7f9b381288d0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.322+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b380727d0 con 0x7f9b381288d0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.322+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3812de90 con 0x7f9b381288d0 2024-04-23T19:14:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.322+0000 7f9b2effd640 1 -- 172.21.15.16:0/4292409517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f9b3401a070 con 0x7f9b381288d0 2024-04-23T19:14:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.322+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b38096b00 con 0x7f9b381288d0 2024-04-23T19:14:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.323+0000 7f9b2effd640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9b1c0385c0 0x7f9b1c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.323+0000 7f9b2effd640 1 -- 172.21.15.16:0/4292409517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f9b3404b360 con 0x7f9b381288d0 2024-04-23T19:14:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.323+0000 7f9b3dd81640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9b1c0385c0 0x7f9b1c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.324+0000 7f9b3dd81640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9b1c0385c0 0x7f9b1c03aa80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9b28036000 tx=0x7f9b28008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.327+0000 7f9b2effd640 1 -- 172.21.15.16:0/4292409517 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b38096b00 con 0x7f9b381288d0 2024-04-23T19:14:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.512+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b38063660 con 0x7f9b1c0385c0 2024-04-23T19:14:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.528+0000 7f9b2effd640 1 -- 172.21.15.16:0/4292409517 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+661515 (secure 0 0 0) 0x7f9b38063660 con 0x7f9b1c0385c0 2024-04-23T19:14:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:14:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9b1c0385c0 msgr2=0x7f9b1c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9b1c0385c0 0x7f9b1c03aa80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9b28036000 tx=0x7f9b28008030 comp rx=0 tx=0).stop 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b381288d0 msgr2=0x7f9b3812c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b381288d0 0x7f9b3812c800 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c4a0 tx=0x7f9b3400c960 comp rx=0 tx=0).stop 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 shutdown_connections 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f9b1c0385c0 0x7f9b1c03aa80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 --2- 172.21.15.16:0/4292409517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b381288d0 0x7f9b3812c800 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 >> 172.21.15.16:0/4292409517 conn(0x7f9b3809a200 msgr2=0x7f9b38094e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 shutdown_connections 2024-04-23T19:14:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:08.533+0000 7f9b407e6640 1 -- 172.21.15.16:0/4292409517 wait complete. 2024-04-23T19:14:08.547 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-23T19:14:06.715496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5092,"num_read_kb":4673,"num_write":2177,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4768,"ondisk_log_size":4768,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":148344,"kb_used_data":2840,"kb_used_omap":6,"kb_used_meta":145465,"kb_avail":377339016,"statfs":{"total":386547056640,"available":386395152384,"internally_reserved":0,"allocated":2908160,"data_stored":1756034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":148956951},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003874"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.715767+0000","last_change":"2024-04-23T19:10:07.492726+0000","last_active":"2024-04-23T19:12:15.715767+0000","last_peered":"2024-04-23T19:12:15.715767+0000","last_clean":"2024-04-23T19:12:15.715767+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:12:15.715767+0000","last_undegraded":"2024-04-23T19:12:15.715767+0000","last_fullsized":"2024-04-23T19:12:15.715767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.721981+0000","last_change":"2024-04-23T19:10:09.485943+0000","last_active":"2024-04-23T19:12:41.721981+0000","last_peered":"2024-04-23T19:12:41.721981+0000","last_clean":"2024-04-23T19:12:41.721981+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:12:41.721981+0000","last_undegraded":"2024-04-23T19:12:41.721981+0000","last_fullsized":"2024-04-23T19:12:41.721981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.564766+0000","last_change":"2024-04-23T19:14:04.564794+0000","last_active":"2024-04-23T19:14:04.564766+0000","last_peered":"2024-04-23T19:14:04.564766+0000","last_clean":"2024-04-23T19:14:04.564766+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:14:04.564766+0000","last_undegraded":"2024-04-23T19:14:04.564766+0000","last_fullsized":"2024-04-23T19:14:04.564766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.564668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:53.911600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.937584+0000","last_change":"2024-04-23T19:13:59.937609+0000","last_active":"2024-04-23T19:13:59.937584+0000","last_peered":"2024-04-23T19:13:59.937584+0000","last_clean":"2024-04-23T19:13:59.937584+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:59.937584+0000","last_undegraded":"2024-04-23T19:13:59.937584+0000","last_fullsized":"2024-04-23T19:13:59.937584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.937477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:01.611995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717659+0000","last_change":"2024-04-23T19:10:07.508341+0000","last_active":"2024-04-23T19:12:11.717659+0000","last_peered":"2024-04-23T19:12:11.717659+0000","last_clean":"2024-04-23T19:12:11.717659+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:12:11.717659+0000","last_undegraded":"2024-04-23T19:12:11.717659+0000","last_fullsized":"2024-04-23T19:12:11.717659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.946559+0000","last_change":"2024-04-23T19:13:46.946583+0000","last_active":"2024-04-23T19:13:46.946559+0000","last_peered":"2024-04-23T19:13:46.946559+0000","last_clean":"2024-04-23T19:13:46.946559+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:13:46.946559+0000","last_undegraded":"2024-04-23T19:13:46.946559+0000","last_fullsized":"2024-04-23T19:13:46.946559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.946470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:19.468921+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.714883+0000","last_change":"2024-04-23T19:10:07.490859+0000","last_active":"2024-04-23T19:12:10.714883+0000","last_peered":"2024-04-23T19:12:10.714883+0000","last_clean":"2024-04-23T19:12:10.714883+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:12:10.714883+0000","last_undegraded":"2024-04-23T19:12:10.714883+0000","last_fullsized":"2024-04-23T19:12:10.714883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.461439+0000","last_change":"2024-04-23T19:10:09.489556+0000","last_active":"2024-04-23T19:13:16.461439+0000","last_peered":"2024-04-23T19:13:16.461439+0000","last_clean":"2024-04-23T19:13:16.461439+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:13:16.461439+0000","last_undegraded":"2024-04-23T19:13:16.461439+0000","last_fullsized":"2024-04-23T19:13:16.461439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721644+0000","last_change":"2024-04-23T19:10:07.504507+0000","last_active":"2024-04-23T19:12:36.721644+0000","last_peered":"2024-04-23T19:12:36.721644+0000","last_clean":"2024-04-23T19:12:36.721644+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:12:36.721644+0000","last_undegraded":"2024-04-23T19:12:36.721644+0000","last_fullsized":"2024-04-23T19:12:36.721644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.789928+0000","last_change":"2024-04-23T19:13:57.789955+0000","last_active":"2024-04-23T19:13:57.789928+0000","last_peered":"2024-04-23T19:13:57.789928+0000","last_clean":"2024-04-23T19:13:57.789928+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:13:57.789928+0000","last_undegraded":"2024-04-23T19:13:57.789928+0000","last_fullsized":"2024-04-23T19:13:57.789928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.789839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:44.659415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.105261+0000","last_change":"2024-04-23T19:10:07.490892+0000","last_active":"2024-04-23T19:13:23.105261+0000","last_peered":"2024-04-23T19:13:23.105261+0000","last_clean":"2024-04-23T19:13:23.105261+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:13:23.105261+0000","last_undegraded":"2024-04-23T19:13:23.105261+0000","last_fullsized":"2024-04-23T19:13:23.105261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":85,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.905840+0000","last_change":"2024-04-23T19:13:44.905886+0000","last_active":"2024-04-23T19:13:44.905840+0000","last_peered":"2024-04-23T19:13:44.905840+0000","last_clean":"2024-04-23T19:13:44.905840+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:44.905840+0000","last_undegraded":"2024-04-23T19:13:44.905840+0000","last_fullsized":"2024-04-23T19:13:44.905840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.905747+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.252680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721533+0000","last_change":"2024-04-23T19:10:07.489309+0000","last_active":"2024-04-23T19:12:36.721533+0000","last_peered":"2024-04-23T19:12:36.721533+0000","last_clean":"2024-04-23T19:12:36.721533+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:12:36.721533+0000","last_undegraded":"2024-04-23T19:12:36.721533+0000","last_fullsized":"2024-04-23T19:12:36.721533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'24","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.035801+0000","last_change":"2024-04-23T19:13:57.035838+0000","last_active":"2024-04-23T19:13:57.035801+0000","last_peered":"2024-04-23T19:13:57.035801+0000","last_clean":"2024-04-23T19:13:57.035801+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:57.035801+0000","last_undegraded":"2024-04-23T19:13:57.035801+0000","last_fullsized":"2024-04-23T19:13:57.035801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'24","last_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_deep_scrub":"32'24","last_deep_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.035696+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:34.133123+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.105077+0000","last_change":"2024-04-23T19:10:07.487170+0000","last_active":"2024-04-23T19:13:23.105077+0000","last_peered":"2024-04-23T19:13:23.105077+0000","last_clean":"2024-04-23T19:13:23.105077+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:13:23.105077+0000","last_undegraded":"2024-04-23T19:13:23.105077+0000","last_fullsized":"2024-04-23T19:13:23.105077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'17","reported_seq":73,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.537243+0000","last_change":"2024-04-23T19:10:09.493592+0000","last_active":"2024-04-23T19:13:16.537243+0000","last_peered":"2024-04-23T19:13:16.537243+0000","last_clean":"2024-04-23T19:13:16.537243+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:13:16.537243+0000","last_undegraded":"2024-04-23T19:13:16.537243+0000","last_fullsized":"2024-04-23T19:13:16.537243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.943596+0000","last_change":"2024-04-23T19:13:43.943623+0000","last_active":"2024-04-23T19:13:43.943596+0000","last_peered":"2024-04-23T19:13:43.943596+0000","last_clean":"2024-04-23T19:13:43.943596+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:13:43.943596+0000","last_undegraded":"2024-04-23T19:13:43.943596+0000","last_fullsized":"2024-04-23T19:13:43.943596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.943496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:10.354547+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720541+0000","last_change":"2024-04-23T19:10:07.491914+0000","last_active":"2024-04-23T19:12:31.720541+0000","last_peered":"2024-04-23T19:12:31.720541+0000","last_clean":"2024-04-23T19:12:31.720541+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:12:31.720541+0000","last_undegraded":"2024-04-23T19:12:31.720541+0000","last_fullsized":"2024-04-23T19:12:31.720541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.911745+0000","last_change":"2024-04-23T19:14:00.911780+0000","last_active":"2024-04-23T19:14:00.911745+0000","last_peered":"2024-04-23T19:14:00.911745+0000","last_clean":"2024-04-23T19:14:00.911745+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:14:00.911745+0000","last_undegraded":"2024-04-23T19:14:00.911745+0000","last_fullsized":"2024-04-23T19:14:00.911745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.911657+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.206563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.718957+0000","last_change":"2024-04-23T19:10:07.489933+0000","last_active":"2024-04-23T19:12:35.718957+0000","last_peered":"2024-04-23T19:12:35.718957+0000","last_clean":"2024-04-23T19:12:35.718957+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:12:35.718957+0000","last_undegraded":"2024-04-23T19:12:35.718957+0000","last_fullsized":"2024-04-23T19:12:35.718957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720311+0000","last_change":"2024-04-23T19:10:09.484959+0000","last_active":"2024-04-23T19:12:31.720311+0000","last_peered":"2024-04-23T19:12:31.720311+0000","last_clean":"2024-04-23T19:12:31.720311+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:12:31.720311+0000","last_undegraded":"2024-04-23T19:12:31.720311+0000","last_fullsized":"2024-04-23T19:12:31.720311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.107188+0000","last_change":"2024-04-23T19:10:07.493585+0000","last_active":"2024-04-23T19:13:23.107188+0000","last_peered":"2024-04-23T19:13:23.107188+0000","last_clean":"2024-04-23T19:13:23.107188+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:13:23.107188+0000","last_undegraded":"2024-04-23T19:13:23.107188+0000","last_fullsized":"2024-04-23T19:13:23.107188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.526561+0000","last_change":"2024-04-23T19:13:56.526589+0000","last_active":"2024-04-23T19:13:56.526561+0000","last_peered":"2024-04-23T19:13:56.526561+0000","last_clean":"2024-04-23T19:13:56.526561+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:56.526561+0000","last_undegraded":"2024-04-23T19:13:56.526561+0000","last_fullsized":"2024-04-23T19:13:56.526561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.526450+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:02.826857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720406+0000","last_change":"2024-04-23T19:10:07.501878+0000","last_active":"2024-04-23T19:12:31.720406+0000","last_peered":"2024-04-23T19:12:31.720406+0000","last_clean":"2024-04-23T19:12:31.720406+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:12:31.720406+0000","last_undegraded":"2024-04-23T19:12:31.720406+0000","last_fullsized":"2024-04-23T19:12:31.720406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.044859+0000","last_change":"2024-04-23T19:14:03.044887+0000","last_active":"2024-04-23T19:14:03.044859+0000","last_peered":"2024-04-23T19:14:03.044859+0000","last_clean":"2024-04-23T19:14:03.044859+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:14:03.044859+0000","last_undegraded":"2024-04-23T19:14:03.044859+0000","last_fullsized":"2024-04-23T19:14:03.044859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.044766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:47.262891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:35.713939+0000","last_change":"2024-04-23T19:10:07.490910+0000","last_active":"2024-04-23T19:12:35.713939+0000","last_peered":"2024-04-23T19:12:35.713939+0000","last_clean":"2024-04-23T19:12:35.713939+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:12:35.713939+0000","last_undegraded":"2024-04-23T19:12:35.713939+0000","last_fullsized":"2024-04-23T19:12:35.713939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.816135+0000","last_change":"2024-04-23T19:13:56.816160+0000","last_active":"2024-04-23T19:13:56.816135+0000","last_peered":"2024-04-23T19:13:56.816135+0000","last_clean":"2024-04-23T19:13:56.816135+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:56.816135+0000","last_undegraded":"2024-04-23T19:13:56.816135+0000","last_fullsized":"2024-04-23T19:13:56.816135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.816040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:41.034165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720008+0000","last_change":"2024-04-23T19:10:07.493956+0000","last_active":"2024-04-23T19:12:26.720008+0000","last_peered":"2024-04-23T19:12:26.720008+0000","last_clean":"2024-04-23T19:12:26.720008+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:12:26.720008+0000","last_undegraded":"2024-04-23T19:12:26.720008+0000","last_fullsized":"2024-04-23T19:12:26.720008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'36","reported_seq":109,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.799937+0000","last_change":"2024-04-23T19:13:54.799967+0000","last_active":"2024-04-23T19:13:54.799937+0000","last_peered":"2024-04-23T19:13:54.799937+0000","last_clean":"2024-04-23T19:13:54.799937+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:54.799937+0000","last_undegraded":"2024-04-23T19:13:54.799937+0000","last_fullsized":"2024-04-23T19:13:54.799937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'36","last_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_deep_scrub":"32'36","last_deep_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.799848+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:43.632765+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720840+0000","last_change":"2024-04-23T19:10:09.487433+0000","last_active":"2024-04-23T19:12:31.720840+0000","last_peered":"2024-04-23T19:12:31.720840+0000","last_clean":"2024-04-23T19:12:31.720840+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:12:31.720840+0000","last_undegraded":"2024-04-23T19:12:31.720840+0000","last_fullsized":"2024-04-23T19:12:31.720840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720039+0000","last_change":"2024-04-23T19:10:07.507346+0000","last_active":"2024-04-23T19:12:26.720039+0000","last_peered":"2024-04-23T19:12:26.720039+0000","last_clean":"2024-04-23T19:12:26.720039+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:12:26.720039+0000","last_undegraded":"2024-04-23T19:12:26.720039+0000","last_fullsized":"2024-04-23T19:12:26.720039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.986548+0000","last_change":"2024-04-23T19:13:32.986571+0000","last_active":"2024-04-23T19:13:32.986548+0000","last_peered":"2024-04-23T19:13:32.986548+0000","last_clean":"2024-04-23T19:13:32.986548+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:13:32.986548+0000","last_undegraded":"2024-04-23T19:13:32.986548+0000","last_fullsized":"2024-04-23T19:13:32.986548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.986453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:15.050911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.889110+0000","last_change":"2024-04-23T19:13:41.889150+0000","last_active":"2024-04-23T19:13:41.889110+0000","last_peered":"2024-04-23T19:13:41.889110+0000","last_clean":"2024-04-23T19:13:41.889110+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:13:41.889110+0000","last_undegraded":"2024-04-23T19:13:41.889110+0000","last_fullsized":"2024-04-23T19:13:41.889110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.889007+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:50.624792+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.929910+0000","last_change":"2024-04-23T19:13:38.929939+0000","last_active":"2024-04-23T19:13:38.929910+0000","last_peered":"2024-04-23T19:13:38.929910+0000","last_clean":"2024-04-23T19:13:38.929910+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:13:38.929910+0000","last_undegraded":"2024-04-23T19:13:38.929910+0000","last_fullsized":"2024-04-23T19:13:38.929910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.929807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:00.218571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'10","reported_seq":37,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.758575+0000","last_change":"2024-04-23T19:13:50.758606+0000","last_active":"2024-04-23T19:13:50.758575+0000","last_peered":"2024-04-23T19:13:50.758575+0000","last_clean":"2024-04-23T19:13:50.758575+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:50.758575+0000","last_undegraded":"2024-04-23T19:13:50.758575+0000","last_fullsized":"2024-04-23T19:13:50.758575+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.758473+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:28.510048+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.405978+0000","last_change":"2024-04-23T19:10:09.496014+0000","last_active":"2024-04-23T19:13:16.405978+0000","last_peered":"2024-04-23T19:13:16.405978+0000","last_clean":"2024-04-23T19:13:16.405978+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:13:16.405978+0000","last_undegraded":"2024-04-23T19:13:16.405978+0000","last_fullsized":"2024-04-23T19:13:16.405978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.010646+0000","last_change":"2024-04-23T19:14:04.010672+0000","last_active":"2024-04-23T19:14:04.010646+0000","last_peered":"2024-04-23T19:14:04.010646+0000","last_clean":"2024-04-23T19:14:04.010646+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:14:04.010646+0000","last_undegraded":"2024-04-23T19:14:04.010646+0000","last_fullsized":"2024-04-23T19:14:04.010646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.010558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:50.880134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":230,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.491387+0000","last_change":"2024-04-23T19:13:50.491419+0000","last_active":"2024-04-23T19:13:50.491387+0000","last_peered":"2024-04-23T19:13:50.491387+0000","last_clean":"2024-04-23T19:13:50.491387+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:13:50.491387+0000","last_undegraded":"2024-04-23T19:13:50.491387+0000","last_fullsized":"2024-04-23T19:13:50.491387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.491297+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:21.383865+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"32'9","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.851896+0000","last_change":"2024-04-23T19:13:32.851932+0000","last_active":"2024-04-23T19:13:32.851896+0000","last_peered":"2024-04-23T19:13:32.851896+0000","last_clean":"2024-04-23T19:13:32.851896+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:13:32.851896+0000","last_undegraded":"2024-04-23T19:13:32.851896+0000","last_fullsized":"2024-04-23T19:13:32.851896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'9","last_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_deep_scrub":"32'9","last_deep_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.851806+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:35.714512+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.911789+0000","last_change":"2024-04-23T19:13:34.911815+0000","last_active":"2024-04-23T19:13:34.911789+0000","last_peered":"2024-04-23T19:13:34.911789+0000","last_clean":"2024-04-23T19:13:34.911789+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:13:34.911789+0000","last_undegraded":"2024-04-23T19:13:34.911789+0000","last_fullsized":"2024-04-23T19:13:34.911789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.911697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:51.912852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:50.715845+0000","last_change":"2024-04-23T19:10:17.518249+0000","last_active":"2024-04-23T19:12:50.715845+0000","last_peered":"2024-04-23T19:12:50.715845+0000","last_clean":"2024-04-23T19:12:50.715845+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:12:50.715845+0000","last_undegraded":"2024-04-23T19:12:50.715845+0000","last_fullsized":"2024-04-23T19:12:50.715845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.456147+0000","last_change":"2024-04-23T19:10:09.490234+0000","last_active":"2024-04-23T19:13:16.456147+0000","last_peered":"2024-04-23T19:13:16.456147+0000","last_clean":"2024-04-23T19:13:16.456147+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:13:16.456147+0000","last_undegraded":"2024-04-23T19:13:16.456147+0000","last_fullsized":"2024-04-23T19:13:16.456147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.391967+0000","last_change":"2024-04-23T19:10:07.504260+0000","last_active":"2024-04-23T19:13:16.391967+0000","last_peered":"2024-04-23T19:13:16.391967+0000","last_clean":"2024-04-23T19:13:16.391967+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:13:16.391967+0000","last_undegraded":"2024-04-23T19:13:16.391967+0000","last_fullsized":"2024-04-23T19:13:16.391967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.612267+0000","last_change":"2024-04-23T19:13:44.612314+0000","last_active":"2024-04-23T19:13:44.612267+0000","last_peered":"2024-04-23T19:13:44.612267+0000","last_clean":"2024-04-23T19:13:44.612267+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:13:44.612267+0000","last_undegraded":"2024-04-23T19:13:44.612267+0000","last_fullsized":"2024-04-23T19:13:44.612267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.612159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:59.958714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'16","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.181820+0000","last_change":"2024-04-23T19:13:36.181858+0000","last_active":"2024-04-23T19:13:36.181820+0000","last_peered":"2024-04-23T19:13:36.181820+0000","last_clean":"2024-04-23T19:13:36.181820+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:13:36.181820+0000","last_undegraded":"2024-04-23T19:13:36.181820+0000","last_fullsized":"2024-04-23T19:13:36.181820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.181735+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:21.509377+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'285","reported_seq":389,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.193029+0000","last_change":"2024-04-23T19:10:15.515398+0000","last_active":"2024-04-23T19:13:23.193029+0000","last_peered":"2024-04-23T19:13:23.193029+0000","last_clean":"2024-04-23T19:13:23.193029+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:13:23.193029+0000","last_undegraded":"2024-04-23T19:13:23.193029+0000","last_fullsized":"2024-04-23T19:13:23.193029+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:50.721007+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:50.721007+0000","last_peered":"2024-04-23T19:12:50.721007+0000","last_clean":"2024-04-23T19:12:50.721007+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:12:50.721007+0000","last_undegraded":"2024-04-23T19:12:50.721007+0000","last_fullsized":"2024-04-23T19:12:50.721007+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489503+0000","last_change":"2024-04-23T19:10:09.487095+0000","last_active":"2024-04-23T19:13:16.489503+0000","last_peered":"2024-04-23T19:13:16.489503+0000","last_clean":"2024-04-23T19:13:16.489503+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:13:16.489503+0000","last_undegraded":"2024-04-23T19:13:16.489503+0000","last_fullsized":"2024-04-23T19:13:16.489503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719376+0000","last_change":"2024-04-23T19:10:07.504154+0000","last_active":"2024-04-23T19:12:26.719376+0000","last_peered":"2024-04-23T19:12:26.719376+0000","last_clean":"2024-04-23T19:12:26.719376+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:12:26.719376+0000","last_undegraded":"2024-04-23T19:12:26.719376+0000","last_fullsized":"2024-04-23T19:12:26.719376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.940488+0000","last_change":"2024-04-23T19:13:34.940545+0000","last_active":"2024-04-23T19:13:34.940488+0000","last_peered":"2024-04-23T19:13:34.940488+0000","last_clean":"2024-04-23T19:13:34.940488+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:13:34.940488+0000","last_undegraded":"2024-04-23T19:13:34.940488+0000","last_fullsized":"2024-04-23T19:13:34.940488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.940371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:19.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.236705+0000","last_change":"2024-04-23T19:13:34.236743+0000","last_active":"2024-04-23T19:13:34.236705+0000","last_peered":"2024-04-23T19:13:34.236705+0000","last_clean":"2024-04-23T19:13:34.236705+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:13:34.236705+0000","last_undegraded":"2024-04-23T19:13:34.236705+0000","last_fullsized":"2024-04-23T19:13:34.236705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.236613+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:39.299958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'285","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.684726+0000","last_change":"2024-04-23T19:13:32.684754+0000","last_active":"2024-04-23T19:13:32.684726+0000","last_peered":"2024-04-23T19:13:32.684726+0000","last_clean":"2024-04-23T19:13:32.684726+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:32.684726+0000","last_undegraded":"2024-04-23T19:13:32.684726+0000","last_fullsized":"2024-04-23T19:13:32.684726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.684622+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:16.152805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.458795+0000","last_change":"2024-04-23T19:10:47.730482+0000","last_active":"2024-04-23T19:13:16.458795+0000","last_peered":"2024-04-23T19:13:16.458795+0000","last_clean":"2024-04-23T19:13:16.458795+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:13:16.458795+0000","last_undegraded":"2024-04-23T19:13:16.458795+0000","last_fullsized":"2024-04-23T19:13:16.458795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.717521+0000","last_change":"2024-04-23T19:10:07.492480+0000","last_active":"2024-04-23T19:12:25.717521+0000","last_peered":"2024-04-23T19:12:25.717521+0000","last_clean":"2024-04-23T19:12:25.717521+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:12:25.717521+0000","last_undegraded":"2024-04-23T19:12:25.717521+0000","last_fullsized":"2024-04-23T19:12:25.717521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.919126+0000","last_change":"2024-04-23T19:13:40.919152+0000","last_active":"2024-04-23T19:13:40.919126+0000","last_peered":"2024-04-23T19:13:40.919126+0000","last_clean":"2024-04-23T19:13:40.919126+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:13:40.919126+0000","last_undegraded":"2024-04-23T19:13:40.919126+0000","last_fullsized":"2024-04-23T19:13:40.919126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.919038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:35.160636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'51","reported_seq":111,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.205995+0000","last_change":"2024-04-23T19:13:39.206079+0000","last_active":"2024-04-23T19:13:39.205995+0000","last_peered":"2024-04-23T19:13:39.205995+0000","last_clean":"2024-04-23T19:13:39.205995+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:13:39.205995+0000","last_undegraded":"2024-04-23T19:13:39.205995+0000","last_fullsized":"2024-04-23T19:13:39.205995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'51","last_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_deep_scrub":"32'51","last_deep_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.205895+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:03.299394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.646105+0000","last_change":"2024-04-23T19:13:43.646133+0000","last_active":"2024-04-23T19:13:43.646105+0000","last_peered":"2024-04-23T19:13:43.646105+0000","last_clean":"2024-04-23T19:13:43.646105+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:13:43.646105+0000","last_undegraded":"2024-04-23T19:13:43.646105+0000","last_fullsized":"2024-04-23T19:13:43.646105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.646006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:44.344504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.717405+0000","last_change":"2024-04-23T19:10:17.519710+0000","last_active":"2024-04-23T19:12:25.717405+0000","last_peered":"2024-04-23T19:12:25.717405+0000","last_clean":"2024-04-23T19:12:25.717405+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:12:25.717405+0000","last_undegraded":"2024-04-23T19:12:25.717405+0000","last_fullsized":"2024-04-23T19:12:25.717405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.835784+0000","last_change":"2024-04-23T19:13:58.835852+0000","last_active":"2024-04-23T19:13:58.835784+0000","last_peered":"2024-04-23T19:13:58.835784+0000","last_clean":"2024-04-23T19:13:58.835784+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:58.835784+0000","last_undegraded":"2024-04-23T19:13:58.835784+0000","last_fullsized":"2024-04-23T19:13:58.835784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.835665+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:05.136073+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.024239+0000","last_change":"2024-04-23T19:13:54.024266+0000","last_active":"2024-04-23T19:13:54.024239+0000","last_peered":"2024-04-23T19:13:54.024239+0000","last_clean":"2024-04-23T19:13:54.024239+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:13:54.024239+0000","last_undegraded":"2024-04-23T19:13:54.024239+0000","last_fullsized":"2024-04-23T19:13:54.024239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.024146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:24.982777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.680623+0000","last_change":"2024-04-23T19:13:42.680645+0000","last_active":"2024-04-23T19:13:42.680623+0000","last_peered":"2024-04-23T19:13:42.680623+0000","last_clean":"2024-04-23T19:13:42.680623+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:13:42.680623+0000","last_undegraded":"2024-04-23T19:13:42.680623+0000","last_fullsized":"2024-04-23T19:13:42.680623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.680534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:03.703466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'38","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.936866+0000","last_change":"2024-04-23T19:13:58.936900+0000","last_active":"2024-04-23T19:13:58.936866+0000","last_peered":"2024-04-23T19:13:58.936866+0000","last_clean":"2024-04-23T19:13:58.936866+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:13:58.936866+0000","last_undegraded":"2024-04-23T19:13:58.936866+0000","last_fullsized":"2024-04-23T19:13:58.936866+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.936780+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:49.313649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.585103+0000","last_change":"2024-04-23T19:13:28.585132+0000","last_active":"2024-04-23T19:13:28.585103+0000","last_peered":"2024-04-23T19:13:28.585103+0000","last_clean":"2024-04-23T19:13:28.585103+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:13:28.585103+0000","last_undegraded":"2024-04-23T19:13:28.585103+0000","last_fullsized":"2024-04-23T19:13:28.585103+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.585004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:05.071466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'5","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.267889+0000","last_change":"2024-04-23T19:10:17.518314+0000","last_active":"2024-04-23T19:13:16.267889+0000","last_peered":"2024-04-23T19:13:16.267889+0000","last_clean":"2024-04-23T19:13:16.267889+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:13:16.267889+0000","last_undegraded":"2024-04-23T19:13:16.267889+0000","last_fullsized":"2024-04-23T19:13:16.267889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":88,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.098620+0000","last_change":"2024-04-23T19:14:01.098652+0000","last_active":"2024-04-23T19:14:01.098620+0000","last_peered":"2024-04-23T19:14:01.098620+0000","last_clean":"2024-04-23T19:14:01.098620+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:14:01.098620+0000","last_undegraded":"2024-04-23T19:14:01.098620+0000","last_fullsized":"2024-04-23T19:14:01.098620+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.098506+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:43.162965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719245+0000","last_change":"2024-04-23T19:10:07.492637+0000","last_active":"2024-04-23T19:12:21.719245+0000","last_peered":"2024-04-23T19:12:21.719245+0000","last_clean":"2024-04-23T19:12:21.719245+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:12:21.719245+0000","last_undegraded":"2024-04-23T19:12:21.719245+0000","last_fullsized":"2024-04-23T19:12:21.719245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.573918+0000","last_change":"2024-04-23T19:13:45.573946+0000","last_active":"2024-04-23T19:13:45.573918+0000","last_peered":"2024-04-23T19:13:45.573918+0000","last_clean":"2024-04-23T19:13:45.573918+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:13:45.573918+0000","last_undegraded":"2024-04-23T19:13:45.573918+0000","last_fullsized":"2024-04-23T19:13:45.573918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.573803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:16.532433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'45","reported_seq":103,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.889391+0000","last_change":"2024-04-23T19:13:50.889437+0000","last_active":"2024-04-23T19:13:50.889391+0000","last_peered":"2024-04-23T19:13:50.889391+0000","last_clean":"2024-04-23T19:13:50.889391+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:13:50.889391+0000","last_undegraded":"2024-04-23T19:13:50.889391+0000","last_fullsized":"2024-04-23T19:13:50.889391+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.889290+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:52.811988+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'285","reported_seq":360,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.512179+0000","last_change":"2024-04-23T19:13:51.512239+0000","last_active":"2024-04-23T19:13:51.512179+0000","last_peered":"2024-04-23T19:13:51.512179+0000","last_clean":"2024-04-23T19:13:51.512179+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:51.512179+0000","last_undegraded":"2024-04-23T19:13:51.512179+0000","last_fullsized":"2024-04-23T19:13:51.512179+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.512072+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:26.148887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:30.713313+0000","last_change":"2024-04-23T19:10:17.520791+0000","last_active":"2024-04-23T19:12:30.713313+0000","last_peered":"2024-04-23T19:12:30.713313+0000","last_clean":"2024-04-23T19:12:30.713313+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:12:30.713313+0000","last_undegraded":"2024-04-23T19:12:30.713313+0000","last_fullsized":"2024-04-23T19:12:30.713313+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.092362+0000","last_change":"2024-04-23T19:13:51.092424+0000","last_active":"2024-04-23T19:13:51.092362+0000","last_peered":"2024-04-23T19:13:51.092362+0000","last_clean":"2024-04-23T19:13:51.092362+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:51.092362+0000","last_undegraded":"2024-04-23T19:13:51.092362+0000","last_fullsized":"2024-04-23T19:13:51.092362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.092261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:12.115193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719160+0000","last_change":"2024-04-23T19:10:07.501083+0000","last_active":"2024-04-23T19:12:21.719160+0000","last_peered":"2024-04-23T19:12:21.719160+0000","last_clean":"2024-04-23T19:12:21.719160+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:12:21.719160+0000","last_undegraded":"2024-04-23T19:12:21.719160+0000","last_fullsized":"2024-04-23T19:12:21.719160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.667015+0000","last_change":"2024-04-23T19:13:41.667080+0000","last_active":"2024-04-23T19:13:41.667015+0000","last_peered":"2024-04-23T19:13:41.667015+0000","last_clean":"2024-04-23T19:13:41.667015+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:13:41.667015+0000","last_undegraded":"2024-04-23T19:13:41.667015+0000","last_fullsized":"2024-04-23T19:13:41.667015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.666920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:52.673059+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.975145+0000","last_change":"2024-04-23T19:13:33.975191+0000","last_active":"2024-04-23T19:13:33.975145+0000","last_peered":"2024-04-23T19:13:33.975145+0000","last_clean":"2024-04-23T19:13:33.975145+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:13:33.975145+0000","last_undegraded":"2024-04-23T19:13:33.975145+0000","last_fullsized":"2024-04-23T19:13:33.975145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.975049+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:07.187015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.695421+0000","last_change":"2024-04-23T19:13:35.695444+0000","last_active":"2024-04-23T19:13:35.695421+0000","last_peered":"2024-04-23T19:13:35.695421+0000","last_clean":"2024-04-23T19:13:35.695421+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:13:35.695421+0000","last_undegraded":"2024-04-23T19:13:35.695421+0000","last_fullsized":"2024-04-23T19:13:35.695421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.695326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:43.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'18","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.587428+0000","last_change":"2024-04-23T19:10:17.513990+0000","last_active":"2024-04-23T19:13:16.587428+0000","last_peered":"2024-04-23T19:13:16.587428+0000","last_clean":"2024-04-23T19:13:16.587428+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:13:16.587428+0000","last_undegraded":"2024-04-23T19:13:16.587428+0000","last_fullsized":"2024-04-23T19:13:16.587428+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.796778+0000","last_change":"2024-04-23T19:13:55.796803+0000","last_active":"2024-04-23T19:13:55.796778+0000","last_peered":"2024-04-23T19:13:55.796778+0000","last_clean":"2024-04-23T19:13:55.796778+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:55.796778+0000","last_undegraded":"2024-04-23T19:13:55.796778+0000","last_fullsized":"2024-04-23T19:13:55.796778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.796686+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:51:24.793617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712349+0000","last_change":"2024-04-23T19:10:07.490182+0000","last_active":"2024-04-23T19:12:25.712349+0000","last_peered":"2024-04-23T19:12:25.712349+0000","last_clean":"2024-04-23T19:12:25.712349+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:12:25.712349+0000","last_undegraded":"2024-04-23T19:12:25.712349+0000","last_fullsized":"2024-04-23T19:12:25.712349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.190333+0000","last_change":"2024-04-23T19:13:44.190374+0000","last_active":"2024-04-23T19:13:44.190333+0000","last_peered":"2024-04-23T19:13:44.190333+0000","last_clean":"2024-04-23T19:13:44.190333+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:13:44.190333+0000","last_undegraded":"2024-04-23T19:13:44.190333+0000","last_fullsized":"2024-04-23T19:13:44.190333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.190242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:32.140502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.877432+0000","last_change":"2024-04-23T19:13:33.877469+0000","last_active":"2024-04-23T19:13:33.877432+0000","last_peered":"2024-04-23T19:13:33.877432+0000","last_clean":"2024-04-23T19:13:33.877432+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:13:33.877432+0000","last_undegraded":"2024-04-23T19:13:33.877432+0000","last_fullsized":"2024-04-23T19:13:33.877432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.877330+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:17.345513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.951003+0000","last_change":"2024-04-23T19:13:48.951037+0000","last_active":"2024-04-23T19:13:48.951003+0000","last_peered":"2024-04-23T19:13:48.951003+0000","last_clean":"2024-04-23T19:13:48.951003+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:13:48.951003+0000","last_undegraded":"2024-04-23T19:13:48.951003+0000","last_fullsized":"2024-04-23T19:13:48.951003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.950905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:41.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.540893+0000","last_change":"2024-04-23T19:10:17.511337+0000","last_active":"2024-04-23T19:13:16.540893+0000","last_peered":"2024-04-23T19:13:16.540893+0000","last_clean":"2024-04-23T19:13:16.540893+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:13:16.540893+0000","last_undegraded":"2024-04-23T19:13:16.540893+0000","last_fullsized":"2024-04-23T19:13:16.540893+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.919986+0000","last_change":"2024-04-23T19:13:42.920017+0000","last_active":"2024-04-23T19:13:42.919986+0000","last_peered":"2024-04-23T19:13:42.919986+0000","last_clean":"2024-04-23T19:13:42.919986+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:42.919986+0000","last_undegraded":"2024-04-23T19:13:42.919986+0000","last_fullsized":"2024-04-23T19:13:42.919986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.919888+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:33.023562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718347+0000","last_change":"2024-04-23T19:10:07.494969+0000","last_active":"2024-04-23T19:12:16.718347+0000","last_peered":"2024-04-23T19:12:16.718347+0000","last_clean":"2024-04-23T19:12:16.718347+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:12:16.718347+0000","last_undegraded":"2024-04-23T19:12:16.718347+0000","last_fullsized":"2024-04-23T19:12:16.718347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.565484+0000","last_change":"2024-04-23T19:13:46.565530+0000","last_active":"2024-04-23T19:13:46.565484+0000","last_peered":"2024-04-23T19:13:46.565484+0000","last_clean":"2024-04-23T19:13:46.565484+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:13:46.565484+0000","last_undegraded":"2024-04-23T19:13:46.565484+0000","last_fullsized":"2024-04-23T19:13:46.565484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.565380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:54.525905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'45","reported_seq":103,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.188689+0000","last_change":"2024-04-23T19:13:40.188736+0000","last_active":"2024-04-23T19:13:40.188689+0000","last_peered":"2024-04-23T19:13:40.188689+0000","last_clean":"2024-04-23T19:13:40.188689+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:13:40.188689+0000","last_undegraded":"2024-04-23T19:13:40.188689+0000","last_fullsized":"2024-04-23T19:13:40.188689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.188598+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:54.836675+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'285","reported_seq":415,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.128944+0000","last_change":"2024-04-23T19:13:28.128972+0000","last_active":"2024-04-23T19:13:28.128944+0000","last_peered":"2024-04-23T19:13:28.128944+0000","last_clean":"2024-04-23T19:13:28.128944+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:28.128944+0000","last_undegraded":"2024-04-23T19:13:28.128944+0000","last_fullsized":"2024-04-23T19:13:28.128944+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.128856+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:09.022827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'12","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.542668+0000","last_change":"2024-04-23T19:10:17.518152+0000","last_active":"2024-04-23T19:13:16.542668+0000","last_peered":"2024-04-23T19:13:16.542668+0000","last_clean":"2024-04-23T19:13:16.542668+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:13:16.542668+0000","last_undegraded":"2024-04-23T19:13:16.542668+0000","last_fullsized":"2024-04-23T19:13:16.542668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.008589+0000","last_change":"2024-04-23T19:14:05.008630+0000","last_active":"2024-04-23T19:14:05.008589+0000","last_peered":"2024-04-23T19:14:05.008589+0000","last_clean":"2024-04-23T19:14:05.008589+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:14:05.008589+0000","last_undegraded":"2024-04-23T19:14:05.008589+0000","last_fullsized":"2024-04-23T19:14:05.008589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'20","last_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_deep_scrub":"32'20","last_deep_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.008496+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:11.308903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718301+0000","last_change":"2024-04-23T19:10:07.508434+0000","last_active":"2024-04-23T19:12:16.718301+0000","last_peered":"2024-04-23T19:12:16.718301+0000","last_clean":"2024-04-23T19:12:16.718301+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:12:16.718301+0000","last_undegraded":"2024-04-23T19:12:16.718301+0000","last_fullsized":"2024-04-23T19:12:16.718301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'5","reported_seq":428,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.911991+0000","last_change":"2024-04-23T19:13:37.912027+0000","last_active":"2024-04-23T19:13:37.911991+0000","last_peered":"2024-04-23T19:13:37.911991+0000","last_clean":"2024-04-23T19:13:37.911991+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:13:37.911991+0000","last_undegraded":"2024-04-23T19:13:37.911991+0000","last_fullsized":"2024-04-23T19:13:37.911991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.911892+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:23.295978+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"32'15","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.923002+0000","last_change":"2024-04-23T19:13:45.923039+0000","last_active":"2024-04-23T19:13:45.923002+0000","last_peered":"2024-04-23T19:13:45.923002+0000","last_clean":"2024-04-23T19:13:45.923002+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:13:45.923002+0000","last_undegraded":"2024-04-23T19:13:45.923002+0000","last_fullsized":"2024-04-23T19:13:45.923002+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.922918+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:04.148244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'285","reported_seq":477,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.784157+0000","last_change":"2024-04-23T19:13:26.784184+0000","last_active":"2024-04-23T19:13:26.784157+0000","last_peered":"2024-04-23T19:13:26.784157+0000","last_clean":"2024-04-23T19:13:26.784157+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:26.784157+0000","last_undegraded":"2024-04-23T19:13:26.784157+0000","last_fullsized":"2024-04-23T19:13:26.784157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.784064+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:31.847408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.211829+0000","last_change":"2024-04-23T19:13:41.211854+0000","last_active":"2024-04-23T19:13:41.211829+0000","last_peered":"2024-04-23T19:13:41.211829+0000","last_clean":"2024-04-23T19:13:41.211829+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:13:41.211829+0000","last_undegraded":"2024-04-23T19:13:41.211829+0000","last_fullsized":"2024-04-23T19:13:41.211829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.211740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.450923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.738274+0000","last_change":"2024-04-23T19:13:48.738341+0000","last_active":"2024-04-23T19:13:48.738274+0000","last_peered":"2024-04-23T19:13:48.738274+0000","last_clean":"2024-04-23T19:13:48.738274+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:13:48.738274+0000","last_undegraded":"2024-04-23T19:13:48.738274+0000","last_fullsized":"2024-04-23T19:13:48.738274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.738148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:12.860218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.388835+0000","last_change":"2024-04-23T19:10:07.507496+0000","last_active":"2024-04-23T19:13:16.388835+0000","last_peered":"2024-04-23T19:13:16.388835+0000","last_clean":"2024-04-23T19:13:16.388835+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:13:16.388835+0000","last_undegraded":"2024-04-23T19:13:16.388835+0000","last_fullsized":"2024-04-23T19:13:16.388835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.111154+0000","last_change":"2024-04-23T19:13:28.111184+0000","last_active":"2024-04-23T19:13:28.111154+0000","last_peered":"2024-04-23T19:13:28.111154+0000","last_clean":"2024-04-23T19:13:28.111154+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:13:28.111154+0000","last_undegraded":"2024-04-23T19:13:28.111154+0000","last_fullsized":"2024-04-23T19:13:28.111154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.111056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:18.387489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'48","reported_seq":107,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.948761+0000","last_change":"2024-04-23T19:13:35.948796+0000","last_active":"2024-04-23T19:13:35.948761+0000","last_peered":"2024-04-23T19:13:35.948761+0000","last_clean":"2024-04-23T19:13:35.948761+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:13:35.948761+0000","last_undegraded":"2024-04-23T19:13:35.948761+0000","last_fullsized":"2024-04-23T19:13:35.948761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'48","last_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_deep_scrub":"32'48","last_deep_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.948676+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:52.110221+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.116201+0000","last_change":"2024-04-23T19:13:30.116258+0000","last_active":"2024-04-23T19:13:30.116201+0000","last_peered":"2024-04-23T19:13:30.116201+0000","last_clean":"2024-04-23T19:13:30.116201+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:13:30.116201+0000","last_undegraded":"2024-04-23T19:13:30.116201+0000","last_fullsized":"2024-04-23T19:13:30.116201+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.116108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:58.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'7","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.724981+0000","last_change":"2024-04-23T19:13:46.725037+0000","last_active":"2024-04-23T19:13:46.724981+0000","last_peered":"2024-04-23T19:13:46.724981+0000","last_clean":"2024-04-23T19:13:46.724981+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:46.724981+0000","last_undegraded":"2024-04-23T19:13:46.724981+0000","last_fullsized":"2024-04-23T19:13:46.724981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'7","last_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_deep_scrub":"32'7","last_deep_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.724881+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:17.683511+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.894478+0000","last_change":"2024-04-23T19:13:40.894509+0000","last_active":"2024-04-23T19:13:40.894478+0000","last_peered":"2024-04-23T19:13:40.894478+0000","last_clean":"2024-04-23T19:13:40.894478+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:13:40.894478+0000","last_undegraded":"2024-04-23T19:13:40.894478+0000","last_fullsized":"2024-04-23T19:13:40.894478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.894372+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:11.403485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":203,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.513823+0000","last_change":"2024-04-23T19:09:52.518180+0000","last_active":"2024-04-23T19:13:26.513823+0000","last_peered":"2024-04-23T19:13:26.513823+0000","last_clean":"2024-04-23T19:13:26.513823+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:13:26.513823+0000","last_undegraded":"2024-04-23T19:13:26.513823+0000","last_fullsized":"2024-04-23T19:13:26.513823+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:51.428209+0000","last_clean_scrub_stamp":"2024-04-23T19:09:51.428209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:06.716831+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:12:06.716831+0000","last_peered":"2024-04-23T19:12:06.716831+0000","last_clean":"2024-04-23T19:12:06.716831+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:12:06.716831+0000","last_undegraded":"2024-04-23T19:12:06.716831+0000","last_fullsized":"2024-04-23T19:12:06.716831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717583+0000","last_change":"2024-04-23T19:10:07.508995+0000","last_active":"2024-04-23T19:12:11.717583+0000","last_peered":"2024-04-23T19:12:11.717583+0000","last_clean":"2024-04-23T19:12:11.717583+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:12:11.717583+0000","last_undegraded":"2024-04-23T19:12:11.717583+0000","last_fullsized":"2024-04-23T19:12:11.717583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.842193+0000","last_change":"2024-04-23T19:13:43.842231+0000","last_active":"2024-04-23T19:13:43.842193+0000","last_peered":"2024-04-23T19:13:43.842193+0000","last_clean":"2024-04-23T19:13:43.842193+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:13:43.842193+0000","last_undegraded":"2024-04-23T19:13:43.842193+0000","last_fullsized":"2024-04-23T19:13:43.842193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.842100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:04.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'88","reported_seq":178,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.876527+0000","last_change":"2024-04-23T19:13:37.876563+0000","last_active":"2024-04-23T19:13:37.876527+0000","last_peered":"2024-04-23T19:13:37.876527+0000","last_clean":"2024-04-23T19:13:37.876527+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:13:37.876527+0000","last_undegraded":"2024-04-23T19:13:37.876527+0000","last_fullsized":"2024-04-23T19:13:37.876527+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'88","last_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_deep_scrub":"32'88","last_deep_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.876439+0000","objects_scrubbed":5,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:33:54.212735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.621369+0000","last_change":"2024-04-23T19:13:29.621413+0000","last_active":"2024-04-23T19:13:29.621369+0000","last_peered":"2024-04-23T19:13:29.621369+0000","last_clean":"2024-04-23T19:13:29.621369+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:13:29.621369+0000","last_undegraded":"2024-04-23T19:13:29.621369+0000","last_fullsized":"2024-04-23T19:13:29.621369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.621264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:03.242868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":171,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.101439+0000","last_change":"2024-04-23T19:13:49.101471+0000","last_active":"2024-04-23T19:13:49.101439+0000","last_peered":"2024-04-23T19:13:49.101439+0000","last_clean":"2024-04-23T19:13:49.101439+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:49.101439+0000","last_undegraded":"2024-04-23T19:13:49.101439+0000","last_fullsized":"2024-04-23T19:13:49.101439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'81","last_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_deep_scrub":"32'81","last_deep_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.101341+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:49.219878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719274+0000","last_change":"2024-04-23T19:10:09.488098+0000","last_active":"2024-04-23T19:12:21.719274+0000","last_peered":"2024-04-23T19:12:21.719274+0000","last_clean":"2024-04-23T19:12:21.719274+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:12:21.719274+0000","last_undegraded":"2024-04-23T19:12:21.719274+0000","last_fullsized":"2024-04-23T19:12:21.719274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400043+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:13:27.400043+0000","last_peered":"2024-04-23T19:13:27.400043+0000","last_clean":"2024-04-23T19:13:27.400043+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:13:27.400043+0000","last_undegraded":"2024-04-23T19:13:27.400043+0000","last_fullsized":"2024-04-23T19:13:27.400043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710873+0000","last_change":"2024-04-23T19:10:07.486483+0000","last_active":"2024-04-23T19:12:15.710873+0000","last_peered":"2024-04-23T19:12:15.710873+0000","last_clean":"2024-04-23T19:12:15.710873+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:12:15.710873+0000","last_undegraded":"2024-04-23T19:12:15.710873+0000","last_fullsized":"2024-04-23T19:12:15.710873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.891584+0000","last_change":"2024-04-23T19:13:36.891613+0000","last_active":"2024-04-23T19:13:36.891584+0000","last_peered":"2024-04-23T19:13:36.891584+0000","last_clean":"2024-04-23T19:13:36.891584+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:13:36.891584+0000","last_undegraded":"2024-04-23T19:13:36.891584+0000","last_fullsized":"2024-04-23T19:13:36.891584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.891497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:43.191904+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'290","reported_seq":515,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.753074+0000","last_change":"2024-04-23T19:13:28.753118+0000","last_active":"2024-04-23T19:13:28.753074+0000","last_peered":"2024-04-23T19:13:28.753074+0000","last_clean":"2024-04-23T19:13:28.753074+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:28.753074+0000","last_undegraded":"2024-04-23T19:13:28.753074+0000","last_fullsized":"2024-04-23T19:13:28.753074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'290","last_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_deep_scrub":"32'290","last_deep_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.752962+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:14.080605+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.883822+0000","last_change":"2024-04-23T19:13:53.883869+0000","last_active":"2024-04-23T19:13:53.883822+0000","last_peered":"2024-04-23T19:13:53.883822+0000","last_clean":"2024-04-23T19:13:53.883822+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:13:53.883822+0000","last_undegraded":"2024-04-23T19:13:53.883822+0000","last_fullsized":"2024-04-23T19:13:53.883822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.883735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:38.160620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719457+0000","last_change":"2024-04-23T19:10:17.518750+0000","last_active":"2024-04-23T19:12:26.719457+0000","last_peered":"2024-04-23T19:12:26.719457+0000","last_clean":"2024-04-23T19:12:26.719457+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:12:26.719457+0000","last_undegraded":"2024-04-23T19:12:26.719457+0000","last_fullsized":"2024-04-23T19:12:26.719457+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489441+0000","last_change":"2024-04-23T19:10:09.486782+0000","last_active":"2024-04-23T19:13:16.489441+0000","last_peered":"2024-04-23T19:13:16.489441+0000","last_clean":"2024-04-23T19:13:16.489441+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:13:16.489441+0000","last_undegraded":"2024-04-23T19:13:16.489441+0000","last_fullsized":"2024-04-23T19:13:16.489441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:05.714047+0000","last_change":"2024-04-23T19:09:59.449720+0000","last_active":"2024-04-23T19:12:05.714047+0000","last_peered":"2024-04-23T19:12:05.714047+0000","last_clean":"2024-04-23T19:12:05.714047+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:12:05.714047+0000","last_undegraded":"2024-04-23T19:12:05.714047+0000","last_fullsized":"2024-04-23T19:12:05.714047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.504970+0000","last_change":"2024-04-23T19:13:58.504995+0000","last_active":"2024-04-23T19:13:58.504970+0000","last_peered":"2024-04-23T19:13:58.504970+0000","last_clean":"2024-04-23T19:13:58.504970+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:13:58.504970+0000","last_undegraded":"2024-04-23T19:13:58.504970+0000","last_fullsized":"2024-04-23T19:13:58.504970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.504872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:38.028243+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"23'1","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.716581+0000","last_change":"2024-04-23T19:10:47.736369+0000","last_active":"2024-04-23T19:12:20.716581+0000","last_peered":"2024-04-23T19:12:20.716581+0000","last_clean":"2024-04-23T19:12:20.716581+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:12:20.716581+0000","last_undegraded":"2024-04-23T19:12:20.716581+0000","last_fullsized":"2024-04-23T19:12:20.716581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'23","reported_seq":100,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.228683+0000","last_change":"2024-04-23T19:13:38.228773+0000","last_active":"2024-04-23T19:13:38.228683+0000","last_peered":"2024-04-23T19:13:38.228683+0000","last_clean":"2024-04-23T19:13:38.228683+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:38.228683+0000","last_undegraded":"2024-04-23T19:13:38.228683+0000","last_fullsized":"2024-04-23T19:13:38.228683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'23","last_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_deep_scrub":"32'23","last_deep_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.228582+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:14.715044+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.555803+0000","last_change":"2024-04-23T19:13:26.555830+0000","last_active":"2024-04-23T19:13:26.555803+0000","last_peered":"2024-04-23T19:13:26.555803+0000","last_clean":"2024-04-23T19:13:26.555803+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:13:26.555803+0000","last_undegraded":"2024-04-23T19:13:26.555803+0000","last_fullsized":"2024-04-23T19:13:26.555803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.555714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:11.883356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720790+0000","last_change":"2024-04-23T19:10:17.517093+0000","last_active":"2024-04-23T19:12:31.720790+0000","last_peered":"2024-04-23T19:12:31.720790+0000","last_clean":"2024-04-23T19:12:31.720790+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:12:31.720790+0000","last_undegraded":"2024-04-23T19:12:31.720790+0000","last_fullsized":"2024-04-23T19:12:31.720790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.526287+0000","last_change":"2024-04-23T19:13:59.526311+0000","last_active":"2024-04-23T19:13:59.526287+0000","last_peered":"2024-04-23T19:13:59.526287+0000","last_clean":"2024-04-23T19:13:59.526287+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:59.526287+0000","last_undegraded":"2024-04-23T19:13:59.526287+0000","last_fullsized":"2024-04-23T19:13:59.526287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.526195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:39.758894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:06.716784+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:12:06.716784+0000","last_peered":"2024-04-23T19:12:06.716784+0000","last_clean":"2024-04-23T19:12:06.716784+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:12:06.716784+0000","last_undegraded":"2024-04-23T19:12:06.716784+0000","last_fullsized":"2024-04-23T19:12:06.716784+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.715645+0000","last_change":"2024-04-23T19:10:07.482292+0000","last_active":"2024-04-23T19:12:15.715645+0000","last_peered":"2024-04-23T19:12:15.715645+0000","last_clean":"2024-04-23T19:12:15.715645+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:12:15.715645+0000","last_undegraded":"2024-04-23T19:12:15.715645+0000","last_fullsized":"2024-04-23T19:12:15.715645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.501429+0000","last_change":"2024-04-23T19:13:49.501453+0000","last_active":"2024-04-23T19:13:49.501429+0000","last_peered":"2024-04-23T19:13:49.501429+0000","last_clean":"2024-04-23T19:13:49.501429+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:13:49.501429+0000","last_undegraded":"2024-04-23T19:13:49.501429+0000","last_fullsized":"2024-04-23T19:13:49.501429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.501336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:21.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'21","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.866530+0000","last_change":"2024-04-23T19:13:39.866575+0000","last_active":"2024-04-23T19:13:39.866530+0000","last_peered":"2024-04-23T19:13:39.866530+0000","last_clean":"2024-04-23T19:13:39.866530+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:39.866530+0000","last_undegraded":"2024-04-23T19:13:39.866530+0000","last_fullsized":"2024-04-23T19:13:39.866530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.866421+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:20.099121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.919503+0000","last_change":"2024-04-23T19:13:47.919527+0000","last_active":"2024-04-23T19:13:47.919503+0000","last_peered":"2024-04-23T19:13:47.919503+0000","last_clean":"2024-04-23T19:13:47.919503+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:13:47.919503+0000","last_undegraded":"2024-04-23T19:13:47.919503+0000","last_fullsized":"2024-04-23T19:13:47.919503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.919412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:48.916350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.403673+0000","last_change":"2024-04-23T19:10:17.515693+0000","last_active":"2024-04-23T19:13:16.403673+0000","last_peered":"2024-04-23T19:13:16.403673+0000","last_clean":"2024-04-23T19:13:16.403673+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:13:16.403673+0000","last_undegraded":"2024-04-23T19:13:16.403673+0000","last_fullsized":"2024-04-23T19:13:16.403673+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.694239+0000","last_change":"2024-04-23T19:13:47.694263+0000","last_active":"2024-04-23T19:13:47.694239+0000","last_peered":"2024-04-23T19:13:47.694239+0000","last_clean":"2024-04-23T19:13:47.694239+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:47.694239+0000","last_undegraded":"2024-04-23T19:13:47.694239+0000","last_fullsized":"2024-04-23T19:13:47.694239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.694146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:55.654671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.385922+0000","last_change":"2024-04-23T19:10:07.483919+0000","last_active":"2024-04-23T19:13:16.385922+0000","last_peered":"2024-04-23T19:13:16.385922+0000","last_clean":"2024-04-23T19:13:16.385922+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:13:16.385922+0000","last_undegraded":"2024-04-23T19:13:16.385922+0000","last_fullsized":"2024-04-23T19:13:16.385922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:05.709506+0000","last_change":"2024-04-23T19:09:59.450931+0000","last_active":"2024-04-23T19:12:05.709506+0000","last_peered":"2024-04-23T19:12:05.709506+0000","last_clean":"2024-04-23T19:12:05.709506+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:12:05.709506+0000","last_undegraded":"2024-04-23T19:12:05.709506+0000","last_fullsized":"2024-04-23T19:12:05.709506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.017789+0000","last_change":"2024-04-23T19:10:47.730021+0000","last_active":"2024-04-23T19:13:23.017789+0000","last_peered":"2024-04-23T19:13:23.017789+0000","last_clean":"2024-04-23T19:13:23.017789+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:13:23.017789+0000","last_undegraded":"2024-04-23T19:13:23.017789+0000","last_fullsized":"2024-04-23T19:13:23.017789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'75","reported_seq":151,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.702503+0000","last_change":"2024-04-23T19:13:34.702548+0000","last_active":"2024-04-23T19:13:34.702503+0000","last_peered":"2024-04-23T19:13:34.702503+0000","last_clean":"2024-04-23T19:13:34.702503+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:13:34.702503+0000","last_undegraded":"2024-04-23T19:13:34.702503+0000","last_fullsized":"2024-04-23T19:13:34.702503+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'75","last_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_deep_scrub":"32'75","last_deep_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.702393+0000","objects_scrubbed":9,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:50.863938+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.162541+0000","last_change":"2024-04-23T19:13:31.162563+0000","last_active":"2024-04-23T19:13:31.162541+0000","last_peered":"2024-04-23T19:13:31.162541+0000","last_clean":"2024-04-23T19:13:31.162541+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:13:31.162541+0000","last_undegraded":"2024-04-23T19:13:31.162541+0000","last_fullsized":"2024-04-23T19:13:31.162541+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.162449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:47.422034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.538697+0000","last_change":"2024-04-23T19:13:55.538734+0000","last_active":"2024-04-23T19:13:55.538697+0000","last_peered":"2024-04-23T19:13:55.538697+0000","last_clean":"2024-04-23T19:13:55.538697+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:55.538697+0000","last_undegraded":"2024-04-23T19:13:55.538697+0000","last_fullsized":"2024-04-23T19:13:55.538697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.538607+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:42.408183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.742336+0000","last_change":"2024-04-23T19:13:51.742366+0000","last_active":"2024-04-23T19:13:51.742336+0000","last_peered":"2024-04-23T19:13:51.742336+0000","last_clean":"2024-04-23T19:13:51.742336+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:51.742336+0000","last_undegraded":"2024-04-23T19:13:51.742336+0000","last_fullsized":"2024-04-23T19:13:51.742336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.742223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:22.634791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:15.710939+0000","last_change":"2024-04-23T19:10:07.486351+0000","last_active":"2024-04-23T19:12:15.710939+0000","last_peered":"2024-04-23T19:12:15.710939+0000","last_clean":"2024-04-23T19:12:15.710939+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:12:15.710939+0000","last_undegraded":"2024-04-23T19:12:15.710939+0000","last_fullsized":"2024-04-23T19:12:15.710939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:06.716874+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:12:06.716874+0000","last_peered":"2024-04-23T19:12:06.716874+0000","last_clean":"2024-04-23T19:12:06.716874+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:12:06.716874+0000","last_undegraded":"2024-04-23T19:12:06.716874+0000","last_fullsized":"2024-04-23T19:12:06.716874+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.046332+0000","last_change":"2024-04-23T19:13:31.046354+0000","last_active":"2024-04-23T19:13:31.046332+0000","last_peered":"2024-04-23T19:13:31.046332+0000","last_clean":"2024-04-23T19:13:31.046332+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:13:31.046332+0000","last_undegraded":"2024-04-23T19:13:31.046332+0000","last_fullsized":"2024-04-23T19:13:31.046332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.046239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:01.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'22","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.762294+0000","last_change":"2024-04-23T19:13:29.762358+0000","last_active":"2024-04-23T19:13:29.762294+0000","last_peered":"2024-04-23T19:13:29.762294+0000","last_clean":"2024-04-23T19:13:29.762294+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:13:29.762294+0000","last_undegraded":"2024-04-23T19:13:29.762294+0000","last_fullsized":"2024-04-23T19:13:29.762294+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'22","last_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_deep_scrub":"32'22","last_deep_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.762203+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:37.849358+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'285","reported_seq":367,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.782353+0000","last_change":"2024-04-23T19:13:30.782396+0000","last_active":"2024-04-23T19:13:30.782353+0000","last_peered":"2024-04-23T19:13:30.782353+0000","last_clean":"2024-04-23T19:13:30.782353+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:30.782353+0000","last_undegraded":"2024-04-23T19:13:30.782353+0000","last_fullsized":"2024-04-23T19:13:30.782353+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.782266+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:07.268727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720909+0000","last_change":"2024-04-23T19:10:17.513890+0000","last_active":"2024-04-23T19:12:31.720909+0000","last_peered":"2024-04-23T19:12:31.720909+0000","last_clean":"2024-04-23T19:12:31.720909+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:12:31.720909+0000","last_undegraded":"2024-04-23T19:12:31.720909+0000","last_fullsized":"2024-04-23T19:12:31.720909+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.489441+0000","last_change":"2024-04-23T19:10:09.496860+0000","last_active":"2024-04-23T19:13:16.489441+0000","last_peered":"2024-04-23T19:13:16.489441+0000","last_clean":"2024-04-23T19:13:16.489441+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:13:16.489441+0000","last_undegraded":"2024-04-23T19:13:16.489441+0000","last_fullsized":"2024-04-23T19:13:16.489441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718497+0000","last_change":"2024-04-23T19:10:07.493532+0000","last_active":"2024-04-23T19:12:16.718497+0000","last_peered":"2024-04-23T19:12:16.718497+0000","last_clean":"2024-04-23T19:12:16.718497+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:12:16.718497+0000","last_undegraded":"2024-04-23T19:12:16.718497+0000","last_fullsized":"2024-04-23T19:12:16.718497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.710374+0000","last_change":"2024-04-23T19:09:59.451107+0000","last_active":"2024-04-23T19:12:10.710374+0000","last_peered":"2024-04-23T19:12:10.710374+0000","last_clean":"2024-04-23T19:12:10.710374+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:12:10.710374+0000","last_undegraded":"2024-04-23T19:12:10.710374+0000","last_fullsized":"2024-04-23T19:12:10.710374+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.862675+0000","last_change":"2024-04-23T19:13:38.862700+0000","last_active":"2024-04-23T19:13:38.862675+0000","last_peered":"2024-04-23T19:13:38.862675+0000","last_clean":"2024-04-23T19:13:38.862675+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:13:38.862675+0000","last_undegraded":"2024-04-23T19:13:38.862675+0000","last_fullsized":"2024-04-23T19:13:38.862675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.862584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:18.385955+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"32'16","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.889003+0000","last_change":"2024-04-23T19:13:51.889045+0000","last_active":"2024-04-23T19:13:51.889003+0000","last_peered":"2024-04-23T19:13:51.889003+0000","last_clean":"2024-04-23T19:13:51.889003+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:13:51.889003+0000","last_undegraded":"2024-04-23T19:13:51.889003+0000","last_fullsized":"2024-04-23T19:13:51.889003+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.888897+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:47:41.106501+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'285","reported_seq":441,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.522614+0000","last_change":"2024-04-23T19:13:47.522645+0000","last_active":"2024-04-23T19:13:47.522614+0000","last_peered":"2024-04-23T19:13:47.522614+0000","last_clean":"2024-04-23T19:13:47.522614+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:47.522614+0000","last_undegraded":"2024-04-23T19:13:47.522614+0000","last_fullsized":"2024-04-23T19:13:47.522614+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.522518+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:37.798951+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720369+0000","last_change":"2024-04-23T19:10:17.516808+0000","last_active":"2024-04-23T19:12:31.720369+0000","last_peered":"2024-04-23T19:12:31.720369+0000","last_clean":"2024-04-23T19:12:31.720369+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:12:31.720369+0000","last_undegraded":"2024-04-23T19:12:31.720369+0000","last_fullsized":"2024-04-23T19:12:31.720369+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.083324+0000","last_change":"2024-04-23T19:13:50.083351+0000","last_active":"2024-04-23T19:13:50.083324+0000","last_peered":"2024-04-23T19:13:50.083324+0000","last_clean":"2024-04-23T19:13:50.083324+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:50.083324+0000","last_undegraded":"2024-04-23T19:13:50.083324+0000","last_fullsized":"2024-04-23T19:13:50.083324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.083230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:01.089369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.717876+0000","last_change":"2024-04-23T19:10:07.500858+0000","last_active":"2024-04-23T19:12:16.717876+0000","last_peered":"2024-04-23T19:12:16.717876+0000","last_clean":"2024-04-23T19:12:16.717876+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:12:16.717876+0000","last_undegraded":"2024-04-23T19:12:16.717876+0000","last_fullsized":"2024-04-23T19:12:16.717876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717695+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:12:11.717695+0000","last_peered":"2024-04-23T19:12:11.717695+0000","last_clean":"2024-04-23T19:12:11.717695+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:12:11.717695+0000","last_undegraded":"2024-04-23T19:12:11.717695+0000","last_fullsized":"2024-04-23T19:12:11.717695+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.119744+0000","last_change":"2024-04-23T19:13:29.119771+0000","last_active":"2024-04-23T19:13:29.119744+0000","last_peered":"2024-04-23T19:13:29.119744+0000","last_clean":"2024-04-23T19:13:29.119744+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:13:29.119744+0000","last_undegraded":"2024-04-23T19:13:29.119744+0000","last_fullsized":"2024-04-23T19:13:29.119744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.119655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:06.217082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'98","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.195210+0000","last_change":"2024-04-23T19:13:35.195253+0000","last_active":"2024-04-23T19:13:35.195210+0000","last_peered":"2024-04-23T19:13:35.195210+0000","last_clean":"2024-04-23T19:13:35.195210+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:13:35.195210+0000","last_undegraded":"2024-04-23T19:13:35.195210+0000","last_fullsized":"2024-04-23T19:13:35.195210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'98","last_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_deep_scrub":"32'98","last_deep_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.195120+0000","objects_scrubbed":8,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:11.071311+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4316,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":98,"num_read_kb":92,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.664505+0000","last_change":"2024-04-23T19:13:30.664532+0000","last_active":"2024-04-23T19:13:30.664505+0000","last_peered":"2024-04-23T19:13:30.664505+0000","last_clean":"2024-04-23T19:13:30.664505+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:13:30.664505+0000","last_undegraded":"2024-04-23T19:13:30.664505+0000","last_fullsized":"2024-04-23T19:13:30.664505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.664414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:40.569067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.161812+0000","last_change":"2024-04-23T19:13:45.161859+0000","last_active":"2024-04-23T19:13:45.161812+0000","last_peered":"2024-04-23T19:13:45.161812+0000","last_clean":"2024-04-23T19:13:45.161812+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:13:45.161812+0000","last_undegraded":"2024-04-23T19:13:45.161812+0000","last_fullsized":"2024-04-23T19:13:45.161812+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.161710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:30.545797+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.187419+0000","last_change":"2024-04-23T19:13:37.187444+0000","last_active":"2024-04-23T19:13:37.187419+0000","last_peered":"2024-04-23T19:13:37.187419+0000","last_clean":"2024-04-23T19:13:37.187419+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:13:37.187419+0000","last_undegraded":"2024-04-23T19:13:37.187419+0000","last_fullsized":"2024-04-23T19:13:37.187419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.187313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.274468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.039233+0000","last_change":"2024-04-23T19:13:59.039257+0000","last_active":"2024-04-23T19:13:59.039233+0000","last_peered":"2024-04-23T19:13:59.039233+0000","last_clean":"2024-04-23T19:13:59.039233+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:13:59.039233+0000","last_undegraded":"2024-04-23T19:13:59.039233+0000","last_fullsized":"2024-04-23T19:13:59.039233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.039137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:29.931705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720068+0000","last_change":"2024-04-23T19:10:07.500107+0000","last_active":"2024-04-23T19:12:26.720068+0000","last_peered":"2024-04-23T19:12:26.720068+0000","last_clean":"2024-04-23T19:12:26.720068+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:12:26.720068+0000","last_undegraded":"2024-04-23T19:12:26.720068+0000","last_fullsized":"2024-04-23T19:12:26.720068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720337+0000","last_change":"2024-04-23T19:10:47.725041+0000","last_active":"2024-04-23T19:12:31.720337+0000","last_peered":"2024-04-23T19:12:31.720337+0000","last_clean":"2024-04-23T19:12:31.720337+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:12:31.720337+0000","last_undegraded":"2024-04-23T19:12:31.720337+0000","last_fullsized":"2024-04-23T19:12:31.720337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:10.467223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:10.467223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.033330+0000","last_change":"2024-04-23T19:13:32.033364+0000","last_active":"2024-04-23T19:13:32.033330+0000","last_peered":"2024-04-23T19:13:32.033330+0000","last_clean":"2024-04-23T19:13:32.033330+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:13:32.033330+0000","last_undegraded":"2024-04-23T19:13:32.033330+0000","last_fullsized":"2024-04-23T19:13:32.033330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.033246+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:16.932749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.897278+0000","last_change":"2024-04-23T19:13:57.897299+0000","last_active":"2024-04-23T19:13:57.897278+0000","last_peered":"2024-04-23T19:13:57.897278+0000","last_clean":"2024-04-23T19:13:57.897278+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:13:57.897278+0000","last_undegraded":"2024-04-23T19:13:57.897278+0000","last_fullsized":"2024-04-23T19:13:57.897278+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.897188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:37.553642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:20.711683+0000","last_change":"2024-04-23T19:10:17.514011+0000","last_active":"2024-04-23T19:12:20.711683+0000","last_peered":"2024-04-23T19:12:20.711683+0000","last_clean":"2024-04-23T19:12:20.711683+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:12:20.711683+0000","last_undegraded":"2024-04-23T19:12:20.711683+0000","last_fullsized":"2024-04-23T19:12:20.711683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720290+0000","last_change":"2024-04-23T19:10:47.732302+0000","last_active":"2024-04-23T19:12:31.720290+0000","last_peered":"2024-04-23T19:12:31.720290+0000","last_clean":"2024-04-23T19:12:31.720290+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:12:31.720290+0000","last_undegraded":"2024-04-23T19:12:31.720290+0000","last_fullsized":"2024-04-23T19:12:31.720290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.058576+0000","last_change":"2024-04-23T19:14:02.058602+0000","last_active":"2024-04-23T19:14:02.058576+0000","last_peered":"2024-04-23T19:14:02.058576+0000","last_clean":"2024-04-23T19:14:02.058576+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:14:02.058576+0000","last_undegraded":"2024-04-23T19:14:02.058576+0000","last_fullsized":"2024-04-23T19:14:02.058576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.058488+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:35.270454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.891432+0000","last_change":"2024-04-23T19:13:35.891471+0000","last_active":"2024-04-23T19:13:35.891432+0000","last_peered":"2024-04-23T19:13:35.891432+0000","last_clean":"2024-04-23T19:13:35.891432+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:13:35.891432+0000","last_undegraded":"2024-04-23T19:13:35.891432+0000","last_fullsized":"2024-04-23T19:13:35.891432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.891324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:22.760900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.738794+0000","last_change":"2024-04-23T19:13:37.738834+0000","last_active":"2024-04-23T19:13:37.738794+0000","last_peered":"2024-04-23T19:13:37.738794+0000","last_clean":"2024-04-23T19:13:37.738794+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:37.738794+0000","last_undegraded":"2024-04-23T19:13:37.738794+0000","last_fullsized":"2024-04-23T19:13:37.738794+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.738703+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:59.027468+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.218628+0000","last_change":"2024-04-23T19:13:33.218654+0000","last_active":"2024-04-23T19:13:33.218628+0000","last_peered":"2024-04-23T19:13:33.218628+0000","last_clean":"2024-04-23T19:13:33.218628+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:13:33.218628+0000","last_undegraded":"2024-04-23T19:13:33.218628+0000","last_fullsized":"2024-04-23T19:13:33.218628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.218534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:31.120188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:50.715910+0000","last_change":"2024-04-23T19:10:17.520788+0000","last_active":"2024-04-23T19:12:50.715910+0000","last_peered":"2024-04-23T19:12:50.715910+0000","last_clean":"2024-04-23T19:12:50.715910+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:12:50.715910+0000","last_undegraded":"2024-04-23T19:12:50.715910+0000","last_fullsized":"2024-04-23T19:12:50.715910+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.581715+0000","last_change":"2024-04-23T19:13:53.581741+0000","last_active":"2024-04-23T19:13:53.581715+0000","last_peered":"2024-04-23T19:13:53.581715+0000","last_clean":"2024-04-23T19:13:53.581715+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:53.581715+0000","last_undegraded":"2024-04-23T19:13:53.581715+0000","last_fullsized":"2024-04-23T19:13:53.581715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.581624+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:26.793590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.067768+0000","last_change":"2024-04-23T19:13:56.067792+0000","last_active":"2024-04-23T19:13:56.067768+0000","last_peered":"2024-04-23T19:13:56.067768+0000","last_clean":"2024-04-23T19:13:56.067768+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:13:56.067768+0000","last_undegraded":"2024-04-23T19:13:56.067768+0000","last_fullsized":"2024-04-23T19:13:56.067768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.067677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:46.344110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.223615+0000","last_change":"2024-04-23T19:13:43.223641+0000","last_active":"2024-04-23T19:13:43.223615+0000","last_peered":"2024-04-23T19:13:43.223615+0000","last_clean":"2024-04-23T19:13:43.223615+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:13:43.223615+0000","last_undegraded":"2024-04-23T19:13:43.223615+0000","last_fullsized":"2024-04-23T19:13:43.223615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.223515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:59.385060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'47","reported_seq":119,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.709428+0000","last_change":"2024-04-23T19:13:40.709462+0000","last_active":"2024-04-23T19:13:40.709428+0000","last_peered":"2024-04-23T19:13:40.709428+0000","last_clean":"2024-04-23T19:13:40.709428+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:40.709428+0000","last_undegraded":"2024-04-23T19:13:40.709428+0000","last_fullsized":"2024-04-23T19:13:40.709428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'47","last_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_deep_scrub":"32'47","last_deep_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.709338+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:40.827875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.737111+0000","last_change":"2024-04-23T19:13:27.737136+0000","last_active":"2024-04-23T19:13:27.737111+0000","last_peered":"2024-04-23T19:13:27.737111+0000","last_clean":"2024-04-23T19:13:27.737111+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:13:27.737111+0000","last_undegraded":"2024-04-23T19:13:27.737111+0000","last_fullsized":"2024-04-23T19:13:27.737111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.737016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:03.613207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:46.723450+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:12:46.723450+0000","last_peered":"2024-04-23T19:12:46.723450+0000","last_clean":"2024-04-23T19:12:46.723450+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:12:46.723450+0000","last_undegraded":"2024-04-23T19:12:46.723450+0000","last_fullsized":"2024-04-23T19:12:46.723450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'9","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536654+0000","last_change":"2024-04-23T19:10:09.498393+0000","last_active":"2024-04-23T19:13:16.536654+0000","last_peered":"2024-04-23T19:13:16.536654+0000","last_clean":"2024-04-23T19:13:16.536654+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:13:16.536654+0000","last_undegraded":"2024-04-23T19:13:16.536654+0000","last_fullsized":"2024-04-23T19:13:16.536654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720518+0000","last_change":"2024-04-23T19:10:07.496451+0000","last_active":"2024-04-23T19:12:31.720518+0000","last_peered":"2024-04-23T19:12:31.720518+0000","last_clean":"2024-04-23T19:12:31.720518+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:12:31.720518+0000","last_undegraded":"2024-04-23T19:12:31.720518+0000","last_fullsized":"2024-04-23T19:12:31.720518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.092812+0000","last_change":"2024-04-23T19:13:30.092835+0000","last_active":"2024-04-23T19:13:30.092812+0000","last_peered":"2024-04-23T19:13:30.092812+0000","last_clean":"2024-04-23T19:13:30.092812+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:30.092812+0000","last_undegraded":"2024-04-23T19:13:30.092812+0000","last_fullsized":"2024-04-23T19:13:30.092812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.092707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:02.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'38","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.254489+0000","last_change":"2024-04-23T19:13:42.254523+0000","last_active":"2024-04-23T19:13:42.254489+0000","last_peered":"2024-04-23T19:13:42.254489+0000","last_clean":"2024-04-23T19:13:42.254489+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:13:42.254489+0000","last_undegraded":"2024-04-23T19:13:42.254489+0000","last_fullsized":"2024-04-23T19:13:42.254489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.254400+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:59.255554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":53,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.124596+0000","last_change":"2024-04-23T19:13:27.124622+0000","last_active":"2024-04-23T19:13:27.124596+0000","last_peered":"2024-04-23T19:13:27.124596+0000","last_clean":"2024-04-23T19:13:27.124596+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:13:27.124596+0000","last_undegraded":"2024-04-23T19:13:27.124596+0000","last_fullsized":"2024-04-23T19:13:27.124596+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.124488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:56.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:55.721838+0000","last_change":"2024-04-23T19:10:47.728408+0000","last_active":"2024-04-23T19:12:55.721838+0000","last_peered":"2024-04-23T19:12:55.721838+0000","last_clean":"2024-04-23T19:12:55.721838+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:12:55.721838+0000","last_undegraded":"2024-04-23T19:12:55.721838+0000","last_fullsized":"2024-04-23T19:12:55.721838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720691+0000","last_change":"2024-04-23T19:10:07.503907+0000","last_active":"2024-04-23T19:12:31.720691+0000","last_peered":"2024-04-23T19:12:31.720691+0000","last_clean":"2024-04-23T19:12:31.720691+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:12:31.720691+0000","last_undegraded":"2024-04-23T19:12:31.720691+0000","last_fullsized":"2024-04-23T19:12:31.720691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.511187+0000","last_change":"2024-04-23T19:13:48.511211+0000","last_active":"2024-04-23T19:13:48.511187+0000","last_peered":"2024-04-23T19:13:48.511187+0000","last_clean":"2024-04-23T19:13:48.511187+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:13:48.511187+0000","last_undegraded":"2024-04-23T19:13:48.511187+0000","last_fullsized":"2024-04-23T19:13:48.511187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.511103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:25.608530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.697721+0000","last_change":"2024-04-23T19:13:38.697755+0000","last_active":"2024-04-23T19:13:38.697721+0000","last_peered":"2024-04-23T19:13:38.697721+0000","last_clean":"2024-04-23T19:13:38.697721+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:13:38.697721+0000","last_undegraded":"2024-04-23T19:13:38.697721+0000","last_fullsized":"2024-04-23T19:13:38.697721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.697637+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:22.296607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.201793+0000","last_change":"2024-04-23T19:13:32.201821+0000","last_active":"2024-04-23T19:13:32.201793+0000","last_peered":"2024-04-23T19:13:32.201793+0000","last_clean":"2024-04-23T19:13:32.201793+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:13:32.201793+0000","last_undegraded":"2024-04-23T19:13:32.201793+0000","last_fullsized":"2024-04-23T19:13:32.201793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.201694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:43.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.714595+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:40.714595+0000","last_peered":"2024-04-23T19:12:40.714595+0000","last_clean":"2024-04-23T19:12:40.714595+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:12:40.714595+0000","last_undegraded":"2024-04-23T19:12:40.714595+0000","last_fullsized":"2024-04-23T19:12:40.714595+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.612325+0000","last_change":"2024-04-23T19:14:05.612354+0000","last_active":"2024-04-23T19:14:05.612325+0000","last_peered":"2024-04-23T19:14:05.612325+0000","last_clean":"2024-04-23T19:14:05.612325+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:14:05.612325+0000","last_undegraded":"2024-04-23T19:14:05.612325+0000","last_fullsized":"2024-04-23T19:14:05.612325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.612239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:04.922704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'4","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.105129+0000","last_change":"2024-04-23T19:10:07.493942+0000","last_active":"2024-04-23T19:13:23.105129+0000","last_peered":"2024-04-23T19:13:23.105129+0000","last_clean":"2024-04-23T19:13:23.105129+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:13:23.105129+0000","last_undegraded":"2024-04-23T19:13:23.105129+0000","last_fullsized":"2024-04-23T19:13:23.105129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.142190+0000","last_change":"2024-04-23T19:13:48.142214+0000","last_active":"2024-04-23T19:13:48.142190+0000","last_peered":"2024-04-23T19:13:48.142190+0000","last_clean":"2024-04-23T19:13:48.142190+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:13:48.142190+0000","last_undegraded":"2024-04-23T19:13:48.142190+0000","last_fullsized":"2024-04-23T19:13:48.142190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.142093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:17.068685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.927235+0000","last_change":"2024-04-23T19:13:54.927318+0000","last_active":"2024-04-23T19:13:54.927235+0000","last_peered":"2024-04-23T19:13:54.927235+0000","last_clean":"2024-04-23T19:13:54.927235+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:13:54.927235+0000","last_undegraded":"2024-04-23T19:13:54.927235+0000","last_fullsized":"2024-04-23T19:13:54.927235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.927131+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:23.021585+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.866434+0000","last_change":"2024-04-23T19:13:42.866499+0000","last_active":"2024-04-23T19:13:42.866434+0000","last_peered":"2024-04-23T19:13:42.866434+0000","last_clean":"2024-04-23T19:13:42.866434+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:13:42.866434+0000","last_undegraded":"2024-04-23T19:13:42.866434+0000","last_fullsized":"2024-04-23T19:13:42.866434+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.866306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.872444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721579+0000","last_change":"2024-04-23T19:10:17.517391+0000","last_active":"2024-04-23T19:12:36.721579+0000","last_peered":"2024-04-23T19:12:36.721579+0000","last_clean":"2024-04-23T19:12:36.721579+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:12:36.721579+0000","last_undegraded":"2024-04-23T19:12:36.721579+0000","last_fullsized":"2024-04-23T19:12:36.721579+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:20.724924+0000","last_change":"2024-04-23T19:10:47.733611+0000","last_active":"2024-04-23T19:13:20.724924+0000","last_peered":"2024-04-23T19:13:20.724924+0000","last_clean":"2024-04-23T19:13:20.724924+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:13:20.724924+0000","last_undegraded":"2024-04-23T19:13:20.724924+0000","last_fullsized":"2024-04-23T19:13:20.724924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721300+0000","last_change":"2024-04-23T19:10:07.502477+0000","last_active":"2024-04-23T19:12:36.721300+0000","last_peered":"2024-04-23T19:12:36.721300+0000","last_clean":"2024-04-23T19:12:36.721300+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:12:36.721300+0000","last_undegraded":"2024-04-23T19:12:36.721300+0000","last_fullsized":"2024-04-23T19:12:36.721300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.129709+0000","last_change":"2024-04-23T19:13:47.129736+0000","last_active":"2024-04-23T19:13:47.129709+0000","last_peered":"2024-04-23T19:13:47.129709+0000","last_clean":"2024-04-23T19:13:47.129709+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:13:47.129709+0000","last_undegraded":"2024-04-23T19:13:47.129709+0000","last_fullsized":"2024-04-23T19:13:47.129709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.129623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:16:50.873891+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.723348+0000","last_change":"2024-04-23T19:13:36.723412+0000","last_active":"2024-04-23T19:13:36.723348+0000","last_peered":"2024-04-23T19:13:36.723348+0000","last_clean":"2024-04-23T19:13:36.723348+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:13:36.723348+0000","last_undegraded":"2024-04-23T19:13:36.723348+0000","last_fullsized":"2024-04-23T19:13:36.723348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.723239+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:22.107325+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.719921+0000","last_change":"2024-04-23T19:13:39.719943+0000","last_active":"2024-04-23T19:13:39.719921+0000","last_peered":"2024-04-23T19:13:39.719921+0000","last_clean":"2024-04-23T19:13:39.719921+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:13:39.719921+0000","last_undegraded":"2024-04-23T19:13:39.719921+0000","last_fullsized":"2024-04-23T19:13:39.719921+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:33.961424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.462496+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.462496+0000","last_peered":"2024-04-23T19:13:16.462496+0000","last_clean":"2024-04-23T19:13:16.462496+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.462496+0000","last_undegraded":"2024-04-23T19:13:16.462496+0000","last_fullsized":"2024-04-23T19:13:16.462496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.526283+0000","last_change":"2024-04-23T19:13:57.526310+0000","last_active":"2024-04-23T19:13:57.526283+0000","last_peered":"2024-04-23T19:13:57.526283+0000","last_clean":"2024-04-23T19:13:57.526283+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:57.526283+0000","last_undegraded":"2024-04-23T19:13:57.526283+0000","last_fullsized":"2024-04-23T19:13:57.526283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.526187+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:13.862482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.121043+0000","last_change":"2024-04-23T19:10:07.504027+0000","last_active":"2024-04-23T19:13:16.121043+0000","last_peered":"2024-04-23T19:13:16.121043+0000","last_clean":"2024-04-23T19:13:16.121043+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:13:16.121043+0000","last_undegraded":"2024-04-23T19:13:16.121043+0000","last_fullsized":"2024-04-23T19:13:16.121043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'5","reported_seq":238,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.794993+0000","last_change":"2024-04-23T19:13:44.795028+0000","last_active":"2024-04-23T19:13:44.794993+0000","last_peered":"2024-04-23T19:13:44.794993+0000","last_clean":"2024-04-23T19:13:44.794993+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:13:44.794993+0000","last_undegraded":"2024-04-23T19:13:44.794993+0000","last_fullsized":"2024-04-23T19:13:44.794993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.794905+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:45.493403+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"32'15","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.880779+0000","last_change":"2024-04-23T19:13:56.880831+0000","last_active":"2024-04-23T19:13:56.880779+0000","last_peered":"2024-04-23T19:13:56.880779+0000","last_clean":"2024-04-23T19:13:56.880779+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:13:56.880779+0000","last_undegraded":"2024-04-23T19:13:56.880779+0000","last_fullsized":"2024-04-23T19:13:56.880779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.880679+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:26.722280+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'285","reported_seq":429,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.137923+0000","last_change":"2024-04-23T19:13:29.137953+0000","last_active":"2024-04-23T19:13:29.137923+0000","last_peered":"2024-04-23T19:13:29.137923+0000","last_clean":"2024-04-23T19:13:29.137923+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:29.137923+0000","last_undegraded":"2024-04-23T19:13:29.137923+0000","last_fullsized":"2024-04-23T19:13:29.137923+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.137819+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:17.728927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721419+0000","last_change":"2024-04-23T19:10:17.513830+0000","last_active":"2024-04-23T19:12:36.721419+0000","last_peered":"2024-04-23T19:12:36.721419+0000","last_clean":"2024-04-23T19:12:36.721419+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:12:36.721419+0000","last_undegraded":"2024-04-23T19:12:36.721419+0000","last_fullsized":"2024-04-23T19:12:36.721419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.032334+0000","last_change":"2024-04-23T19:13:53.032361+0000","last_active":"2024-04-23T19:13:53.032334+0000","last_peered":"2024-04-23T19:13:53.032334+0000","last_clean":"2024-04-23T19:13:53.032334+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:13:53.032334+0000","last_undegraded":"2024-04-23T19:13:53.032334+0000","last_fullsized":"2024-04-23T19:13:53.032334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.032245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:08.378799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721362+0000","last_change":"2024-04-23T19:10:07.497992+0000","last_active":"2024-04-23T19:12:36.721362+0000","last_peered":"2024-04-23T19:12:36.721362+0000","last_clean":"2024-04-23T19:12:36.721362+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:12:36.721362+0000","last_undegraded":"2024-04-23T19:12:36.721362+0000","last_fullsized":"2024-04-23T19:12:36.721362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.890930+0000","last_change":"2024-04-23T19:13:41.890978+0000","last_active":"2024-04-23T19:13:41.890930+0000","last_peered":"2024-04-23T19:13:41.890930+0000","last_clean":"2024-04-23T19:13:41.890930+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:13:41.890930+0000","last_undegraded":"2024-04-23T19:13:41.890930+0000","last_fullsized":"2024-04-23T19:13:41.890930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.890830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:42.009368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.806873+0000","last_change":"2024-04-23T19:13:31.806949+0000","last_active":"2024-04-23T19:13:31.806873+0000","last_peered":"2024-04-23T19:13:31.806873+0000","last_clean":"2024-04-23T19:13:31.806873+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:13:31.806873+0000","last_undegraded":"2024-04-23T19:13:31.806873+0000","last_fullsized":"2024-04-23T19:13:31.806873+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.806771+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:55.900270+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.655391+0000","last_change":"2024-04-23T19:13:31.655412+0000","last_active":"2024-04-23T19:13:31.655391+0000","last_peered":"2024-04-23T19:13:31.655391+0000","last_clean":"2024-04-23T19:13:31.655391+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:13:31.655391+0000","last_undegraded":"2024-04-23T19:13:31.655391+0000","last_fullsized":"2024-04-23T19:13:31.655391+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.655302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:46.303379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722147+0000","last_change":"2024-04-23T19:10:17.518482+0000","last_active":"2024-04-23T19:12:41.722147+0000","last_peered":"2024-04-23T19:12:41.722147+0000","last_clean":"2024-04-23T19:12:41.722147+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:12:41.722147+0000","last_undegraded":"2024-04-23T19:12:41.722147+0000","last_fullsized":"2024-04-23T19:12:41.722147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.023793+0000","last_change":"2024-04-23T19:13:55.023820+0000","last_active":"2024-04-23T19:13:55.023793+0000","last_peered":"2024-04-23T19:13:55.023793+0000","last_clean":"2024-04-23T19:13:55.023793+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:55.023793+0000","last_undegraded":"2024-04-23T19:13:55.023793+0000","last_fullsized":"2024-04-23T19:13:55.023793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.023704+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:02.984229+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721726+0000","last_change":"2024-04-23T19:10:07.503802+0000","last_active":"2024-04-23T19:12:36.721726+0000","last_peered":"2024-04-23T19:12:36.721726+0000","last_clean":"2024-04-23T19:12:36.721726+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:12:36.721726+0000","last_undegraded":"2024-04-23T19:12:36.721726+0000","last_fullsized":"2024-04-23T19:12:36.721726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.149582+0000","last_change":"2024-04-23T19:13:46.149624+0000","last_active":"2024-04-23T19:13:46.149582+0000","last_peered":"2024-04-23T19:13:46.149582+0000","last_clean":"2024-04-23T19:13:46.149582+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:13:46.149582+0000","last_undegraded":"2024-04-23T19:13:46.149582+0000","last_fullsized":"2024-04-23T19:13:46.149582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.149461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:44.705262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.718061+0000","last_change":"2024-04-23T19:13:33.718094+0000","last_active":"2024-04-23T19:13:33.718061+0000","last_peered":"2024-04-23T19:13:33.718061+0000","last_clean":"2024-04-23T19:13:33.718061+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:13:33.718061+0000","last_undegraded":"2024-04-23T19:13:33.718061+0000","last_fullsized":"2024-04-23T19:13:33.718061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.717974+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:50.719128+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722435+0000","last_change":"2024-04-23T19:10:15.514687+0000","last_active":"2024-04-23T19:12:41.722435+0000","last_peered":"2024-04-23T19:12:41.722435+0000","last_clean":"2024-04-23T19:12:41.722435+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:12:41.722435+0000","last_undegraded":"2024-04-23T19:12:41.722435+0000","last_fullsized":"2024-04-23T19:12:41.722435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:14.481013+0000","last_clean_scrub_stamp":"2024-04-23T19:10:14.481013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722640+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:41.722640+0000","last_peered":"2024-04-23T19:12:41.722640+0000","last_clean":"2024-04-23T19:12:41.722640+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:41.722640+0000","last_undegraded":"2024-04-23T19:12:41.722640+0000","last_fullsized":"2024-04-23T19:12:41.722640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'11","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.537050+0000","last_change":"2024-04-23T19:10:09.490516+0000","last_active":"2024-04-23T19:13:16.537050+0000","last_peered":"2024-04-23T19:13:16.537050+0000","last_clean":"2024-04-23T19:13:16.537050+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:13:16.537050+0000","last_undegraded":"2024-04-23T19:13:16.537050+0000","last_fullsized":"2024-04-23T19:13:16.537050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:10.710387+0000","last_change":"2024-04-23T19:10:07.486770+0000","last_active":"2024-04-23T19:12:10.710387+0000","last_peered":"2024-04-23T19:12:10.710387+0000","last_clean":"2024-04-23T19:12:10.710387+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:12:10.710387+0000","last_undegraded":"2024-04-23T19:12:10.710387+0000","last_fullsized":"2024-04-23T19:12:10.710387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.929784+0000","last_change":"2024-04-23T19:13:36.929808+0000","last_active":"2024-04-23T19:13:36.929784+0000","last_peered":"2024-04-23T19:13:36.929784+0000","last_clean":"2024-04-23T19:13:36.929784+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:13:36.929784+0000","last_undegraded":"2024-04-23T19:13:36.929784+0000","last_fullsized":"2024-04-23T19:13:36.929784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.929690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:45.019264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'30","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.925641+0000","last_change":"2024-04-23T19:13:52.925676+0000","last_active":"2024-04-23T19:13:52.925641+0000","last_peered":"2024-04-23T19:13:52.925641+0000","last_clean":"2024-04-23T19:13:52.925641+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:13:52.925641+0000","last_undegraded":"2024-04-23T19:13:52.925641+0000","last_fullsized":"2024-04-23T19:13:52.925641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'30","last_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_deep_scrub":"32'30","last_deep_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.925542+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.397817+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.909679+0000","last_change":"2024-04-23T19:13:49.909700+0000","last_active":"2024-04-23T19:13:49.909679+0000","last_peered":"2024-04-23T19:13:49.909679+0000","last_clean":"2024-04-23T19:13:49.909679+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:13:49.909679+0000","last_undegraded":"2024-04-23T19:13:49.909679+0000","last_fullsized":"2024-04-23T19:13:49.909679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.909590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:01:35.495993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722581+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:41.722581+0000","last_peered":"2024-04-23T19:12:41.722581+0000","last_clean":"2024-04-23T19:12:41.722581+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:41.722581+0000","last_undegraded":"2024-04-23T19:12:41.722581+0000","last_fullsized":"2024-04-23T19:12:41.722581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.048809+0000","last_change":"2024-04-23T19:13:52.048855+0000","last_active":"2024-04-23T19:13:52.048809+0000","last_peered":"2024-04-23T19:13:52.048809+0000","last_clean":"2024-04-23T19:13:52.048809+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:52.048809+0000","last_undegraded":"2024-04-23T19:13:52.048809+0000","last_fullsized":"2024-04-23T19:13:52.048809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.048718+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:52.747216+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.104929+0000","last_change":"2024-04-23T19:10:07.501211+0000","last_active":"2024-04-23T19:13:23.104929+0000","last_peered":"2024-04-23T19:13:23.104929+0000","last_clean":"2024-04-23T19:13:23.104929+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:13:23.104929+0000","last_undegraded":"2024-04-23T19:13:23.104929+0000","last_fullsized":"2024-04-23T19:13:23.104929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'3","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.954199+0000","last_change":"2024-04-23T19:13:39.954265+0000","last_active":"2024-04-23T19:13:39.954199+0000","last_peered":"2024-04-23T19:13:39.954199+0000","last_clean":"2024-04-23T19:13:39.954199+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:39.954199+0000","last_undegraded":"2024-04-23T19:13:39.954199+0000","last_fullsized":"2024-04-23T19:13:39.954199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.954105+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:23.553074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'40","reported_seq":97,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.877187+0000","last_change":"2024-04-23T19:13:55.877248+0000","last_active":"2024-04-23T19:13:55.877187+0000","last_peered":"2024-04-23T19:13:55.877187+0000","last_clean":"2024-04-23T19:13:55.877187+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:55.877187+0000","last_undegraded":"2024-04-23T19:13:55.877187+0000","last_fullsized":"2024-04-23T19:13:55.877187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'40","last_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_deep_scrub":"32'40","last_deep_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.877078+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:08.472673+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.549277+0000","last_change":"2024-04-23T19:13:27.549295+0000","last_active":"2024-04-23T19:13:27.549277+0000","last_peered":"2024-04-23T19:13:27.549277+0000","last_clean":"2024-04-23T19:13:27.549277+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:13:27.549277+0000","last_undegraded":"2024-04-23T19:13:27.549277+0000","last_fullsized":"2024-04-23T19:13:27.549277+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.549205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:33.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.486600+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.486600+0000","last_peered":"2024-04-23T19:13:16.486600+0000","last_clean":"2024-04-23T19:13:16.486600+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.486600+0000","last_undegraded":"2024-04-23T19:13:16.486600+0000","last_fullsized":"2024-04-23T19:13:16.486600+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.719319+0000","last_change":"2024-04-23T19:13:49.719401+0000","last_active":"2024-04-23T19:13:49.719319+0000","last_peered":"2024-04-23T19:13:49.719319+0000","last_clean":"2024-04-23T19:13:49.719319+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:49.719319+0000","last_undegraded":"2024-04-23T19:13:49.719319+0000","last_fullsized":"2024-04-23T19:13:49.719319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.719206+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:11:27.945878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.480071+0000","last_change":"2024-04-23T19:10:07.492825+0000","last_active":"2024-04-23T19:13:16.480071+0000","last_peered":"2024-04-23T19:13:16.480071+0000","last_clean":"2024-04-23T19:13:16.480071+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:13:16.480071+0000","last_undegraded":"2024-04-23T19:13:16.480071+0000","last_fullsized":"2024-04-23T19:13:16.480071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.541764+0000","last_change":"2024-04-23T19:14:03.541792+0000","last_active":"2024-04-23T19:14:03.541764+0000","last_peered":"2024-04-23T19:14:03.541764+0000","last_clean":"2024-04-23T19:14:03.541764+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:14:03.541764+0000","last_undegraded":"2024-04-23T19:14:03.541764+0000","last_fullsized":"2024-04-23T19:14:03.541764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.541669+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:29.952719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'2","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.104929+0000","last_change":"2024-04-23T19:10:07.489213+0000","last_active":"2024-04-23T19:13:23.104929+0000","last_peered":"2024-04-23T19:13:23.104929+0000","last_clean":"2024-04-23T19:13:23.104929+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:13:23.104929+0000","last_undegraded":"2024-04-23T19:13:23.104929+0000","last_fullsized":"2024-04-23T19:13:23.104929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.266042+0000","last_change":"2024-04-23T19:10:09.495393+0000","last_active":"2024-04-23T19:13:16.266042+0000","last_peered":"2024-04-23T19:13:16.266042+0000","last_clean":"2024-04-23T19:13:16.266042+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:13:16.266042+0000","last_undegraded":"2024-04-23T19:13:16.266042+0000","last_fullsized":"2024-04-23T19:13:16.266042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:11.717563+0000","last_change":"2024-04-23T19:10:07.506697+0000","last_active":"2024-04-23T19:12:11.717563+0000","last_peered":"2024-04-23T19:12:11.717563+0000","last_clean":"2024-04-23T19:12:11.717563+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:12:11.717563+0000","last_undegraded":"2024-04-23T19:12:11.717563+0000","last_fullsized":"2024-04-23T19:12:11.717563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.509098+0000","last_change":"2024-04-23T19:14:01.509140+0000","last_active":"2024-04-23T19:14:01.509098+0000","last_peered":"2024-04-23T19:14:01.509098+0000","last_clean":"2024-04-23T19:14:01.509098+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:14:01.509098+0000","last_undegraded":"2024-04-23T19:14:01.509098+0000","last_fullsized":"2024-04-23T19:14:01.509098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.508998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:10.244783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:23.107204+0000","last_change":"2024-04-23T19:10:07.490623+0000","last_active":"2024-04-23T19:13:23.107204+0000","last_peered":"2024-04-23T19:13:23.107204+0000","last_clean":"2024-04-23T19:13:23.107204+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:13:23.107204+0000","last_undegraded":"2024-04-23T19:13:23.107204+0000","last_fullsized":"2024-04-23T19:13:23.107204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.554203+0000","last_change":"2024-04-23T19:13:54.554231+0000","last_active":"2024-04-23T19:13:54.554203+0000","last_peered":"2024-04-23T19:13:54.554203+0000","last_clean":"2024-04-23T19:13:54.554203+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:13:54.554203+0000","last_undegraded":"2024-04-23T19:13:54.554203+0000","last_fullsized":"2024-04-23T19:13:54.554203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.554115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:38.772240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.518578+0000","last_change":"2024-04-23T19:14:00.518605+0000","last_active":"2024-04-23T19:14:00.518578+0000","last_peered":"2024-04-23T19:14:00.518578+0000","last_clean":"2024-04-23T19:14:00.518578+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:14:00.518578+0000","last_undegraded":"2024-04-23T19:14:00.518578+0000","last_fullsized":"2024-04-23T19:14:00.518578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.518487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:31.027600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.766225+0000","last_change":"2024-04-23T19:13:45.766254+0000","last_active":"2024-04-23T19:13:45.766225+0000","last_peered":"2024-04-23T19:13:45.766225+0000","last_clean":"2024-04-23T19:13:45.766225+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:45.766225+0000","last_undegraded":"2024-04-23T19:13:45.766225+0000","last_fullsized":"2024-04-23T19:13:45.766225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.766137+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:33.256585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.717850+0000","last_change":"2024-04-23T19:10:07.490753+0000","last_active":"2024-04-23T19:12:16.717850+0000","last_peered":"2024-04-23T19:12:16.717850+0000","last_clean":"2024-04-23T19:12:16.717850+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:12:16.717850+0000","last_undegraded":"2024-04-23T19:12:16.717850+0000","last_fullsized":"2024-04-23T19:12:16.717850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.165756+0000","last_change":"2024-04-23T19:10:09.482692+0000","last_active":"2024-04-23T19:13:16.165756+0000","last_peered":"2024-04-23T19:13:16.165756+0000","last_clean":"2024-04-23T19:13:16.165756+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:13:16.165756+0000","last_undegraded":"2024-04-23T19:13:16.165756+0000","last_fullsized":"2024-04-23T19:13:16.165756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.079231+0000","last_change":"2024-04-23T19:13:58.079291+0000","last_active":"2024-04-23T19:13:58.079231+0000","last_peered":"2024-04-23T19:13:58.079231+0000","last_clean":"2024-04-23T19:13:58.079231+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:58.079231+0000","last_undegraded":"2024-04-23T19:13:58.079231+0000","last_fullsized":"2024-04-23T19:13:58.079231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.079130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:35.830705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.265314+0000","last_change":"2024-04-23T19:10:09.484981+0000","last_active":"2024-04-23T19:13:16.265314+0000","last_peered":"2024-04-23T19:13:16.265314+0000","last_clean":"2024-04-23T19:13:16.265314+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:13:16.265314+0000","last_undegraded":"2024-04-23T19:13:16.265314+0000","last_fullsized":"2024-04-23T19:13:16.265314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.554467+0000","last_change":"2024-04-23T19:14:02.554519+0000","last_active":"2024-04-23T19:14:02.554467+0000","last_peered":"2024-04-23T19:14:02.554467+0000","last_clean":"2024-04-23T19:14:02.554467+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:14:02.554467+0000","last_undegraded":"2024-04-23T19:14:02.554467+0000","last_fullsized":"2024-04-23T19:14:02.554467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.554376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:52.658050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720563+0000","last_change":"2024-04-23T19:10:09.484268+0000","last_active":"2024-04-23T19:12:31.720563+0000","last_peered":"2024-04-23T19:12:31.720563+0000","last_clean":"2024-04-23T19:12:31.720563+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:12:31.720563+0000","last_undegraded":"2024-04-23T19:12:31.720563+0000","last_fullsized":"2024-04-23T19:12:31.720563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:16.718248+0000","last_change":"2024-04-23T19:10:07.507068+0000","last_active":"2024-04-23T19:12:16.718248+0000","last_peered":"2024-04-23T19:12:16.718248+0000","last_clean":"2024-04-23T19:12:16.718248+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:12:16.718248+0000","last_undegraded":"2024-04-23T19:12:16.718248+0000","last_fullsized":"2024-04-23T19:12:16.718248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.936668+0000","last_change":"2024-04-23T19:14:00.936694+0000","last_active":"2024-04-23T19:14:00.936668+0000","last_peered":"2024-04-23T19:14:00.936668+0000","last_clean":"2024-04-23T19:14:00.936668+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:14:00.936668+0000","last_undegraded":"2024-04-23T19:14:00.936668+0000","last_fullsized":"2024-04-23T19:14:00.936668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.936576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:19.351629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:25.712299+0000","last_change":"2024-04-23T19:10:07.489441+0000","last_active":"2024-04-23T19:12:25.712299+0000","last_peered":"2024-04-23T19:12:25.712299+0000","last_clean":"2024-04-23T19:12:25.712299+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:12:25.712299+0000","last_undegraded":"2024-04-23T19:12:25.712299+0000","last_fullsized":"2024-04-23T19:12:25.712299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.538431+0000","last_change":"2024-04-23T19:13:52.538459+0000","last_active":"2024-04-23T19:13:52.538431+0000","last_peered":"2024-04-23T19:13:52.538431+0000","last_clean":"2024-04-23T19:13:52.538431+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:13:52.538431+0000","last_undegraded":"2024-04-23T19:13:52.538431+0000","last_fullsized":"2024-04-23T19:13:52.538431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.538343+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:34.602802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.718693+0000","last_change":"2024-04-23T19:10:07.490880+0000","last_active":"2024-04-23T19:12:21.718693+0000","last_peered":"2024-04-23T19:12:21.718693+0000","last_clean":"2024-04-23T19:12:21.718693+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:12:21.718693+0000","last_undegraded":"2024-04-23T19:12:21.718693+0000","last_fullsized":"2024-04-23T19:12:21.718693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.865209+0000","last_change":"2024-04-23T19:13:59.865265+0000","last_active":"2024-04-23T19:13:59.865209+0000","last_peered":"2024-04-23T19:13:59.865209+0000","last_clean":"2024-04-23T19:13:59.865209+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:13:59.865209+0000","last_undegraded":"2024-04-23T19:13:59.865209+0000","last_fullsized":"2024-04-23T19:13:59.865209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.865099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:18:21.512041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.718719+0000","last_change":"2024-04-23T19:10:07.502896+0000","last_active":"2024-04-23T19:12:21.718719+0000","last_peered":"2024-04-23T19:12:21.718719+0000","last_clean":"2024-04-23T19:12:21.718719+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:12:21.718719+0000","last_undegraded":"2024-04-23T19:12:21.718719+0000","last_fullsized":"2024-04-23T19:12:21.718719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.731878+0000","last_change":"2024-04-23T19:13:52.731942+0000","last_active":"2024-04-23T19:13:52.731878+0000","last_peered":"2024-04-23T19:13:52.731878+0000","last_clean":"2024-04-23T19:13:52.731878+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:13:52.731878+0000","last_undegraded":"2024-04-23T19:13:52.731878+0000","last_fullsized":"2024-04-23T19:13:52.731878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.731777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:37.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719175+0000","last_change":"2024-04-23T19:10:07.490658+0000","last_active":"2024-04-23T19:12:21.719175+0000","last_peered":"2024-04-23T19:12:21.719175+0000","last_clean":"2024-04-23T19:12:21.719175+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:12:21.719175+0000","last_undegraded":"2024-04-23T19:12:21.719175+0000","last_fullsized":"2024-04-23T19:12:21.719175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719471+0000","last_change":"2024-04-23T19:10:09.483025+0000","last_active":"2024-04-23T19:12:26.719471+0000","last_peered":"2024-04-23T19:12:26.719471+0000","last_clean":"2024-04-23T19:12:26.719471+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:12:26.719471+0000","last_undegraded":"2024-04-23T19:12:26.719471+0000","last_fullsized":"2024-04-23T19:12:26.719471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719234+0000","last_change":"2024-04-23T19:10:07.502808+0000","last_active":"2024-04-23T19:12:21.719234+0000","last_peered":"2024-04-23T19:12:21.719234+0000","last_clean":"2024-04-23T19:12:21.719234+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:12:21.719234+0000","last_undegraded":"2024-04-23T19:12:21.719234+0000","last_fullsized":"2024-04-23T19:12:21.719234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'5","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.536612+0000","last_change":"2024-04-23T19:10:09.495999+0000","last_active":"2024-04-23T19:13:16.536612+0000","last_peered":"2024-04-23T19:13:16.536612+0000","last_clean":"2024-04-23T19:13:16.536612+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:13:16.536612+0000","last_undegraded":"2024-04-23T19:13:16.536612+0000","last_fullsized":"2024-04-23T19:13:16.536612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.057731+0000","last_change":"2024-04-23T19:14:00.057782+0000","last_active":"2024-04-23T19:14:00.057731+0000","last_peered":"2024-04-23T19:14:00.057731+0000","last_clean":"2024-04-23T19:14:00.057731+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:14:00.057731+0000","last_undegraded":"2024-04-23T19:14:00.057731+0000","last_fullsized":"2024-04-23T19:14:00.057731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.057618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:34.694433+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.265739+0000","last_change":"2024-04-23T19:10:09.489995+0000","last_active":"2024-04-23T19:13:16.265739+0000","last_peered":"2024-04-23T19:13:16.265739+0000","last_clean":"2024-04-23T19:13:16.265739+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:13:16.265739+0000","last_undegraded":"2024-04-23T19:13:16.265739+0000","last_fullsized":"2024-04-23T19:13:16.265739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720175+0000","last_change":"2024-04-23T19:10:07.507910+0000","last_active":"2024-04-23T19:12:26.720175+0000","last_peered":"2024-04-23T19:12:26.720175+0000","last_clean":"2024-04-23T19:12:26.720175+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:12:26.720175+0000","last_undegraded":"2024-04-23T19:12:26.720175+0000","last_fullsized":"2024-04-23T19:12:26.720175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.719909+0000","last_change":"2024-04-23T19:10:07.503476+0000","last_active":"2024-04-23T19:12:26.719909+0000","last_peered":"2024-04-23T19:12:26.719909+0000","last_clean":"2024-04-23T19:12:26.719909+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:12:26.719909+0000","last_undegraded":"2024-04-23T19:12:26.719909+0000","last_fullsized":"2024-04-23T19:12:26.719909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.407993+0000","last_change":"2024-04-23T19:10:09.487033+0000","last_active":"2024-04-23T19:13:16.407993+0000","last_peered":"2024-04-23T19:13:16.407993+0000","last_clean":"2024-04-23T19:13:16.407993+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:13:16.407993+0000","last_undegraded":"2024-04-23T19:13:16.407993+0000","last_fullsized":"2024-04-23T19:13:16.407993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.395378+0000","last_change":"2024-04-23T19:10:07.486697+0000","last_active":"2024-04-23T19:13:16.395378+0000","last_peered":"2024-04-23T19:13:16.395378+0000","last_clean":"2024-04-23T19:13:16.395378+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:13:16.395378+0000","last_undegraded":"2024-04-23T19:13:16.395378+0000","last_fullsized":"2024-04-23T19:13:16.395378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.753730+0000","last_change":"2024-04-23T19:13:53.753760+0000","last_active":"2024-04-23T19:13:53.753730+0000","last_peered":"2024-04-23T19:13:53.753730+0000","last_clean":"2024-04-23T19:13:53.753730+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:53.753730+0000","last_undegraded":"2024-04-23T19:13:53.753730+0000","last_fullsized":"2024-04-23T19:13:53.753730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.753615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:13.659941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1524,"ondisk_log_size":1524,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":477,"ondisk_log_size":477,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673015,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39948,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331892,"statfs":{"total":96636764160,"available":96595857408,"internally_reserved":0,"allocated":462848,"data_stored":144617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673014,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335504,"statfs":{"total":96636764160,"available":96599556096,"internally_reserved":0,"allocated":1024000,"data_stored":734097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40468,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331372,"statfs":{"total":96636764160,"available":96595324928,"internally_reserved":0,"allocated":995328,"data_stored":732924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705719,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":425984,"data_stored":144396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:14:08.558 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.558 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.558 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.559 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.560 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.560 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.560 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.560 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.560 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.560 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.561 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.562 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.562 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.562 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.562 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.562 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.562 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.562 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.563 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.563 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.563 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.563 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.563 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.563 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.563 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.564 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.564 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.564 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T19:09:51.428209+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=51, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.564 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.564 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.564 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.564 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.565 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.566 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.566 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.566 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.566 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.566 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.566 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.566 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.567 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-23T19:10:10.467223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=10, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.568 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.568 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.568 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.568 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.568 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.569 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.569 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.569 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.569 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.570 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.570 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.570 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.570 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.570 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.570 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.570 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-23T19:10:14.481013+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.571 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.572 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.573 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:08.574 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T19:14:28.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:14:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.688+0000 7fbea0d2e640 1 Processor -- start 2024-04-23T19:14:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.688+0000 7fbea0d2e640 1 -- start start 2024-04-23T19:14:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbea0d2e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe9c12ec80 0x7fbe9c131070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbea0d2e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9c128410 con 0x7fbe9c1288d0 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbea0d2e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9c1286b0 con 0x7fbe9c12ec80 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbe99d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe9c12ec80 0x7fbe9c131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbe9a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbe9c1288d0 0x7fbe9c128cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33676/0 (socket says 172.21.15.16:33676) 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbe9a59c640 1 -- 172.21.15.16:0/3006630798 learned_addr learned my addr 172.21.15.16:0/3006630798 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbe99d9b640 1 -- 172.21.15.16:0/3006630798 >> v1:172.21.15.16:6789/0 conn(0x7fbe9c1288d0 legacy=0x7fbe9c128cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.689+0000 7fbe99d9b640 1 -- 172.21.15.16:0/3006630798 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c06de40 con 0x7fbe9c12ec80 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.690+0000 7fbe99d9b640 1 --2- 172.21.15.16:0/3006630798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c12ec80 0x7fbe9c131070 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009720 tx=0x7fbe9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3552c041424dd16e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.690+0000 7fbe9959a640 1 -- 172.21.15.16:0/3006630798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe900316d0 con 0x7fbe9c12ec80 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.690+0000 7fbe9959a640 1 -- 172.21.15.16:0/3006630798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe90037070 con 0x7fbe9c12ec80 2024-04-23T19:14:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.690+0000 7fbe9959a640 1 -- 172.21.15.16:0/3006630798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe900319c0 con 0x7fbe9c12ec80 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.691+0000 7fbea0d2e640 1 -- 172.21.15.16:0/3006630798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c12ec80 msgr2=0x7fbe9c131070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.691+0000 7fbea0d2e640 1 --2- 172.21.15.16:0/3006630798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c12ec80 0x7fbe9c131070 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009720 tx=0x7fbe9002b840 comp rx=0 tx=0).stop 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.691+0000 7fbea0d2e640 1 -- 172.21.15.16:0/3006630798 shutdown_connections 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.691+0000 7fbea0d2e640 1 --2- 172.21.15.16:0/3006630798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c12ec80 0x7fbe9c131070 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.691+0000 7fbea0d2e640 1 -- 172.21.15.16:0/3006630798 >> 172.21.15.16:0/3006630798 conn(0x7fbe9c09a200 msgr2=0x7fbe9c097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.691+0000 7fbea0d2e640 1 -- 172.21.15.16:0/3006630798 shutdown_connections 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.691+0000 7fbea0d2e640 1 -- 172.21.15.16:0/3006630798 wait complete. 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.692+0000 7fbea0d2e640 1 Processor -- start 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.692+0000 7fbea0d2e640 1 -- start start 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.692+0000 7fbea0d2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c1288d0 0x7fbe9c070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.692+0000 7fbea0d2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c0945f0 con 0x7fbe9c1288d0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe9a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c1288d0 0x7fbe9c070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe9a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c1288d0 0x7fbe9c070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38654/0 (socket says 172.21.15.16:38654) 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe9a59c640 1 -- 172.21.15.16:0/2087788707 learned_addr learned my addr 172.21.15.16:0/2087788707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe9a59c640 1 -- 172.21.15.16:0/2087788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c129df0 con 0x7fbe9c1288d0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe9a59c640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c1288d0 0x7fbe9c070f00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c4a0 tx=0x7fbe8c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe8affd640 1 -- 172.21.15.16:0/2087788707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c017070 con 0x7fbe9c1288d0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe8affd640 1 -- 172.21.15.16:0/2087788707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c005c10 con 0x7fbe9c1288d0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbe8affd640 1 -- 172.21.15.16:0/2087788707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c007070 con 0x7fbe9c1288d0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c1298c0 con 0x7fbe9c1288d0 2024-04-23T19:14:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.693+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c071620 con 0x7fbe9c1288d0 2024-04-23T19:14:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.694+0000 7fbe8affd640 1 -- 172.21.15.16:0/2087788707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fbe8c01a070 con 0x7fbe9c1288d0 2024-04-23T19:14:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.694+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c12a0d0 con 0x7fbe9c1288d0 2024-04-23T19:14:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.694+0000 7fbe8affd640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe7c0385c0 0x7fbe7c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.694+0000 7fbe8affd640 1 -- 172.21.15.16:0/2087788707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fbe8c04b3a0 con 0x7fbe9c1288d0 2024-04-23T19:14:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.695+0000 7fbe99d9b640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe7c0385c0 0x7fbe7c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.695+0000 7fbe99d9b640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe7c0385c0 0x7fbe7c03aa80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbe90036000 tx=0x7fbe90008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.699+0000 7fbe8affd640 1 -- 172.21.15.16:0/2087788707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe9c12a0d0 con 0x7fbe9c1288d0 2024-04-23T19:14:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.882+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe9c072ab0 con 0x7fbe7c0385c0 2024-04-23T19:14:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.898+0000 7fbe8affd640 1 -- 172.21.15.16:0/2087788707 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+664934 (secure 0 0 0) 0x7fbe9c072ab0 con 0x7fbe7c0385c0 2024-04-23T19:14:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:14:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe7c0385c0 msgr2=0x7fbe7c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe7c0385c0 0x7fbe7c03aa80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbe90036000 tx=0x7fbe90008030 comp rx=0 tx=0).stop 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c1288d0 msgr2=0x7fbe9c070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c1288d0 0x7fbe9c070f00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c4a0 tx=0x7fbe8c00c960 comp rx=0 tx=0).stop 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 shutdown_connections 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fbe7c0385c0 0x7fbe7c03aa80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 --2- 172.21.15.16:0/2087788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c1288d0 0x7fbe9c070f00 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 >> 172.21.15.16:0/2087788707 conn(0x7fbe9c09a200 msgr2=0x7fbe9c126c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.903+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 shutdown_connections 2024-04-23T19:14:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:28.904+0000 7fbea0d2e640 1 -- 172.21.15.16:0/2087788707 wait complete. 2024-04-23T19:14:28.923 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-04-23T19:14:28.723047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5092,"num_read_kb":4673,"num_write":2177,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4783,"ondisk_log_size":4783,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":152504,"kb_used_data":2840,"kb_used_omap":6,"kb_used_meta":149625,"kb_avail":377334856,"statfs":{"total":386547056640,"available":386390892544,"internally_reserved":0,"allocated":2908160,"data_stored":1756034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":153216791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003987"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.042420+0000","last_change":"2024-04-23T19:14:14.042420+0000","last_active":"2024-04-23T19:14:14.042420+0000","last_peered":"2024-04-23T19:14:14.042420+0000","last_clean":"2024-04-23T19:14:14.042420+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:14:14.042420+0000","last_undegraded":"2024-04-23T19:14:14.042420+0000","last_fullsized":"2024-04-23T19:14:14.042420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.042287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.042287+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.042287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:13.352752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.897072+0000","last_change":"2024-04-23T19:14:10.897102+0000","last_active":"2024-04-23T19:14:10.897072+0000","last_peered":"2024-04-23T19:14:10.897072+0000","last_clean":"2024-04-23T19:14:10.897072+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:14:10.897072+0000","last_undegraded":"2024-04-23T19:14:10.897072+0000","last_fullsized":"2024-04-23T19:14:10.897072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:10.896986+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:10.896986+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.896986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:27.637541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.564794+0000","last_change":"2024-04-23T19:14:04.564794+0000","last_active":"2024-04-23T19:14:04.564794+0000","last_peered":"2024-04-23T19:14:04.564794+0000","last_clean":"2024-04-23T19:14:04.564794+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:14:04.564794+0000","last_undegraded":"2024-04-23T19:14:04.564794+0000","last_fullsized":"2024-04-23T19:14:04.564794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.564668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:53.911600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.937584+0000","last_change":"2024-04-23T19:13:59.937609+0000","last_active":"2024-04-23T19:13:59.937584+0000","last_peered":"2024-04-23T19:13:59.937584+0000","last_clean":"2024-04-23T19:13:59.937584+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:59.937584+0000","last_undegraded":"2024-04-23T19:13:59.937584+0000","last_fullsized":"2024-04-23T19:13:59.937584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.937477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:01.611995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.987105+0000","last_change":"2024-04-23T19:14:18.987130+0000","last_active":"2024-04-23T19:14:18.987105+0000","last_peered":"2024-04-23T19:14:18.987105+0000","last_clean":"2024-04-23T19:14:18.987105+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:14:18.987105+0000","last_undegraded":"2024-04-23T19:14:18.987105+0000","last_fullsized":"2024-04-23T19:14:18.987105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.986988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.986988+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.986988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:58.819070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.946559+0000","last_change":"2024-04-23T19:13:46.946583+0000","last_active":"2024-04-23T19:13:46.946559+0000","last_peered":"2024-04-23T19:13:46.946559+0000","last_clean":"2024-04-23T19:13:46.946559+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:13:46.946559+0000","last_undegraded":"2024-04-23T19:13:46.946559+0000","last_fullsized":"2024-04-23T19:13:46.946559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.946470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:19.468921+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.055999+0000","last_change":"2024-04-23T19:14:11.056028+0000","last_active":"2024-04-23T19:14:11.055999+0000","last_peered":"2024-04-23T19:14:11.055999+0000","last_clean":"2024-04-23T19:14:11.055999+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:14:11.055999+0000","last_undegraded":"2024-04-23T19:14:11.055999+0000","last_fullsized":"2024-04-23T19:14:11.055999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:11.055907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:11.055907+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.055907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:44:01.159581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.985264+0000","last_change":"2024-04-23T19:14:11.985294+0000","last_active":"2024-04-23T19:14:11.985264+0000","last_peered":"2024-04-23T19:14:11.985264+0000","last_clean":"2024-04-23T19:14:11.985264+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:14:11.985264+0000","last_undegraded":"2024-04-23T19:14:11.985264+0000","last_fullsized":"2024-04-23T19:14:11.985264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:11.985174+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:11.985174+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.985174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:26.660339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.878169+0000","last_change":"2024-04-23T19:14:25.878199+0000","last_active":"2024-04-23T19:14:25.878169+0000","last_peered":"2024-04-23T19:14:25.878169+0000","last_clean":"2024-04-23T19:14:25.878169+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:14:25.878169+0000","last_undegraded":"2024-04-23T19:14:25.878169+0000","last_fullsized":"2024-04-23T19:14:25.878169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:25.878068+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:25.878068+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.878068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:20.382452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.789955+0000","last_change":"2024-04-23T19:13:57.789955+0000","last_active":"2024-04-23T19:13:57.789955+0000","last_peered":"2024-04-23T19:13:57.789955+0000","last_clean":"2024-04-23T19:13:57.789955+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:13:57.789955+0000","last_undegraded":"2024-04-23T19:13:57.789955+0000","last_fullsized":"2024-04-23T19:13:57.789955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.789839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:44.659415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.007123+0000","last_change":"2024-04-23T19:14:12.007150+0000","last_active":"2024-04-23T19:14:12.007123+0000","last_peered":"2024-04-23T19:14:12.007123+0000","last_clean":"2024-04-23T19:14:12.007123+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:14:12.007123+0000","last_undegraded":"2024-04-23T19:14:12.007123+0000","last_fullsized":"2024-04-23T19:14:12.007123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:12.007017+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:12.007017+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.007017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:38.418068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":86,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.905886+0000","last_change":"2024-04-23T19:13:44.905886+0000","last_active":"2024-04-23T19:13:44.905886+0000","last_peered":"2024-04-23T19:13:44.905886+0000","last_clean":"2024-04-23T19:13:44.905886+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:44.905886+0000","last_undegraded":"2024-04-23T19:13:44.905886+0000","last_fullsized":"2024-04-23T19:13:44.905886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.905747+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.252680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721533+0000","last_change":"2024-04-23T19:10:07.489309+0000","last_active":"2024-04-23T19:12:36.721533+0000","last_peered":"2024-04-23T19:12:36.721533+0000","last_clean":"2024-04-23T19:12:36.721533+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:12:36.721533+0000","last_undegraded":"2024-04-23T19:12:36.721533+0000","last_fullsized":"2024-04-23T19:12:36.721533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'24","reported_seq":100,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.035838+0000","last_change":"2024-04-23T19:13:57.035838+0000","last_active":"2024-04-23T19:13:57.035838+0000","last_peered":"2024-04-23T19:13:57.035838+0000","last_clean":"2024-04-23T19:13:57.035838+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:57.035838+0000","last_undegraded":"2024-04-23T19:13:57.035838+0000","last_fullsized":"2024-04-23T19:13:57.035838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'24","last_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_deep_scrub":"32'24","last_deep_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.035696+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:34.133123+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.567190+0000","last_change":"2024-04-23T19:14:19.567220+0000","last_active":"2024-04-23T19:14:19.567190+0000","last_peered":"2024-04-23T19:14:19.567190+0000","last_clean":"2024-04-23T19:14:19.567190+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:14:19.567190+0000","last_undegraded":"2024-04-23T19:14:19.567190+0000","last_fullsized":"2024-04-23T19:14:19.567190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.567093+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.567093+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.567093+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.943962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'18","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.074121+0000","last_change":"2024-04-23T19:14:15.074121+0000","last_active":"2024-04-23T19:14:15.074121+0000","last_peered":"2024-04-23T19:14:15.074121+0000","last_clean":"2024-04-23T19:14:15.074121+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:14:15.074121+0000","last_undegraded":"2024-04-23T19:14:15.074121+0000","last_fullsized":"2024-04-23T19:14:15.074121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'18","last_scrub_stamp":"2024-04-23T19:14:15.073956+0000","last_deep_scrub":"32'18","last_deep_scrub_stamp":"2024-04-23T19:14:15.073956+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.073956+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:47.596407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.943596+0000","last_change":"2024-04-23T19:13:43.943623+0000","last_active":"2024-04-23T19:13:43.943596+0000","last_peered":"2024-04-23T19:13:43.943596+0000","last_clean":"2024-04-23T19:13:43.943596+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:13:43.943596+0000","last_undegraded":"2024-04-23T19:13:43.943596+0000","last_fullsized":"2024-04-23T19:13:43.943596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.943496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:10.354547+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.824684+0000","last_change":"2024-04-23T19:14:15.824733+0000","last_active":"2024-04-23T19:14:15.824684+0000","last_peered":"2024-04-23T19:14:15.824684+0000","last_clean":"2024-04-23T19:14:15.824684+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:14:15.824684+0000","last_undegraded":"2024-04-23T19:14:15.824684+0000","last_fullsized":"2024-04-23T19:14:15.824684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:15.824569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:15.824569+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.824569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:40.296843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.911745+0000","last_change":"2024-04-23T19:14:00.911780+0000","last_active":"2024-04-23T19:14:00.911745+0000","last_peered":"2024-04-23T19:14:00.911745+0000","last_clean":"2024-04-23T19:14:00.911745+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:14:00.911745+0000","last_undegraded":"2024-04-23T19:14:00.911745+0000","last_fullsized":"2024-04-23T19:14:00.911745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.911657+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.206563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.028413+0000","last_change":"2024-04-23T19:14:10.028439+0000","last_active":"2024-04-23T19:14:10.028413+0000","last_peered":"2024-04-23T19:14:10.028413+0000","last_clean":"2024-04-23T19:14:10.028413+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:14:10.028413+0000","last_undegraded":"2024-04-23T19:14:10.028413+0000","last_fullsized":"2024-04-23T19:14:10.028413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:10.028304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:10.028304+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.028304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:18.764089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.839412+0000","last_change":"2024-04-23T19:14:06.839443+0000","last_active":"2024-04-23T19:14:06.839412+0000","last_peered":"2024-04-23T19:14:06.839412+0000","last_clean":"2024-04-23T19:14:06.839412+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:14:06.839412+0000","last_undegraded":"2024-04-23T19:14:06.839412+0000","last_fullsized":"2024-04-23T19:14:06.839412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:06.839307+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:06.839307+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.839307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:24.085334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.796508+0000","last_change":"2024-04-23T19:14:16.796535+0000","last_active":"2024-04-23T19:14:16.796508+0000","last_peered":"2024-04-23T19:14:16.796508+0000","last_clean":"2024-04-23T19:14:16.796508+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:14:16.796508+0000","last_undegraded":"2024-04-23T19:14:16.796508+0000","last_fullsized":"2024-04-23T19:14:16.796508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:16.796417+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:16.796417+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.796417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:01.073302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.526561+0000","last_change":"2024-04-23T19:13:56.526589+0000","last_active":"2024-04-23T19:13:56.526561+0000","last_peered":"2024-04-23T19:13:56.526561+0000","last_clean":"2024-04-23T19:13:56.526561+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:56.526561+0000","last_undegraded":"2024-04-23T19:13:56.526561+0000","last_fullsized":"2024-04-23T19:13:56.526561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.526450+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:02.826857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.855594+0000","last_change":"2024-04-23T19:14:07.855627+0000","last_active":"2024-04-23T19:14:07.855594+0000","last_peered":"2024-04-23T19:14:07.855594+0000","last_clean":"2024-04-23T19:14:07.855594+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:14:07.855594+0000","last_undegraded":"2024-04-23T19:14:07.855594+0000","last_fullsized":"2024-04-23T19:14:07.855594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.855478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.855478+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.855478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:53:54.172855+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.044859+0000","last_change":"2024-04-23T19:14:03.044887+0000","last_active":"2024-04-23T19:14:03.044859+0000","last_peered":"2024-04-23T19:14:03.044859+0000","last_clean":"2024-04-23T19:14:03.044859+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:14:03.044859+0000","last_undegraded":"2024-04-23T19:14:03.044859+0000","last_fullsized":"2024-04-23T19:14:03.044859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.044766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:47.262891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.578908+0000","last_change":"2024-04-23T19:14:06.578937+0000","last_active":"2024-04-23T19:14:06.578908+0000","last_peered":"2024-04-23T19:14:06.578908+0000","last_clean":"2024-04-23T19:14:06.578908+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:14:06.578908+0000","last_undegraded":"2024-04-23T19:14:06.578908+0000","last_fullsized":"2024-04-23T19:14:06.578908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:06.578801+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:06.578801+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.578801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:39.101252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.816135+0000","last_change":"2024-04-23T19:13:56.816160+0000","last_active":"2024-04-23T19:13:56.816135+0000","last_peered":"2024-04-23T19:13:56.816135+0000","last_clean":"2024-04-23T19:13:56.816135+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:56.816135+0000","last_undegraded":"2024-04-23T19:13:56.816135+0000","last_fullsized":"2024-04-23T19:13:56.816135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.816040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:41.034165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720008+0000","last_change":"2024-04-23T19:10:07.493956+0000","last_active":"2024-04-23T19:12:26.720008+0000","last_peered":"2024-04-23T19:12:26.720008+0000","last_clean":"2024-04-23T19:12:26.720008+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:12:26.720008+0000","last_undegraded":"2024-04-23T19:12:26.720008+0000","last_fullsized":"2024-04-23T19:12:26.720008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'36","reported_seq":109,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.799937+0000","last_change":"2024-04-23T19:13:54.799967+0000","last_active":"2024-04-23T19:13:54.799937+0000","last_peered":"2024-04-23T19:13:54.799937+0000","last_clean":"2024-04-23T19:13:54.799937+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:54.799937+0000","last_undegraded":"2024-04-23T19:13:54.799937+0000","last_fullsized":"2024-04-23T19:13:54.799937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'36","last_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_deep_scrub":"32'36","last_deep_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.799848+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:43.632765+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.014609+0000","last_change":"2024-04-23T19:14:15.014637+0000","last_active":"2024-04-23T19:14:15.014609+0000","last_peered":"2024-04-23T19:14:15.014609+0000","last_clean":"2024-04-23T19:14:15.014609+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:14:15.014609+0000","last_undegraded":"2024-04-23T19:14:15.014609+0000","last_fullsized":"2024-04-23T19:14:15.014609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:15.014507+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:15.014507+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.014507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:49.779222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720039+0000","last_change":"2024-04-23T19:10:07.507346+0000","last_active":"2024-04-23T19:12:26.720039+0000","last_peered":"2024-04-23T19:12:26.720039+0000","last_clean":"2024-04-23T19:12:26.720039+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:12:26.720039+0000","last_undegraded":"2024-04-23T19:12:26.720039+0000","last_fullsized":"2024-04-23T19:12:26.720039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.986548+0000","last_change":"2024-04-23T19:13:32.986571+0000","last_active":"2024-04-23T19:13:32.986548+0000","last_peered":"2024-04-23T19:13:32.986548+0000","last_clean":"2024-04-23T19:13:32.986548+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:13:32.986548+0000","last_undegraded":"2024-04-23T19:13:32.986548+0000","last_fullsized":"2024-04-23T19:13:32.986548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.986453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:15.050911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.889110+0000","last_change":"2024-04-23T19:13:41.889150+0000","last_active":"2024-04-23T19:13:41.889110+0000","last_peered":"2024-04-23T19:13:41.889110+0000","last_clean":"2024-04-23T19:13:41.889110+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:13:41.889110+0000","last_undegraded":"2024-04-23T19:13:41.889110+0000","last_fullsized":"2024-04-23T19:13:41.889110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.889007+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:50.624792+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.929910+0000","last_change":"2024-04-23T19:13:38.929939+0000","last_active":"2024-04-23T19:13:38.929910+0000","last_peered":"2024-04-23T19:13:38.929910+0000","last_clean":"2024-04-23T19:13:38.929910+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:13:38.929910+0000","last_undegraded":"2024-04-23T19:13:38.929910+0000","last_fullsized":"2024-04-23T19:13:38.929910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.929807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:00.218571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'10","reported_seq":37,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.758575+0000","last_change":"2024-04-23T19:13:50.758606+0000","last_active":"2024-04-23T19:13:50.758575+0000","last_peered":"2024-04-23T19:13:50.758575+0000","last_clean":"2024-04-23T19:13:50.758575+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:50.758575+0000","last_undegraded":"2024-04-23T19:13:50.758575+0000","last_fullsized":"2024-04-23T19:13:50.758575+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.758473+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:28.510048+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":82,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.892739+0000","last_change":"2024-04-23T19:14:04.892767+0000","last_active":"2024-04-23T19:14:04.892739+0000","last_peered":"2024-04-23T19:14:04.892739+0000","last_clean":"2024-04-23T19:14:04.892739+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:14:04.892739+0000","last_undegraded":"2024-04-23T19:14:04.892739+0000","last_fullsized":"2024-04-23T19:14:04.892739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:14:04.892651+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:14:04.892651+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.892651+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:00.034408+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.010646+0000","last_change":"2024-04-23T19:14:04.010672+0000","last_active":"2024-04-23T19:14:04.010646+0000","last_peered":"2024-04-23T19:14:04.010646+0000","last_clean":"2024-04-23T19:14:04.010646+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:14:04.010646+0000","last_undegraded":"2024-04-23T19:14:04.010646+0000","last_fullsized":"2024-04-23T19:14:04.010646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.010558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:50.880134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":230,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.491387+0000","last_change":"2024-04-23T19:13:50.491419+0000","last_active":"2024-04-23T19:13:50.491387+0000","last_peered":"2024-04-23T19:13:50.491387+0000","last_clean":"2024-04-23T19:13:50.491387+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:13:50.491387+0000","last_undegraded":"2024-04-23T19:13:50.491387+0000","last_fullsized":"2024-04-23T19:13:50.491387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.491297+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:21.383865+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"32'9","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.851896+0000","last_change":"2024-04-23T19:13:32.851932+0000","last_active":"2024-04-23T19:13:32.851896+0000","last_peered":"2024-04-23T19:13:32.851896+0000","last_clean":"2024-04-23T19:13:32.851896+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:13:32.851896+0000","last_undegraded":"2024-04-23T19:13:32.851896+0000","last_fullsized":"2024-04-23T19:13:32.851896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'9","last_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_deep_scrub":"32'9","last_deep_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.851806+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:35.714512+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.911789+0000","last_change":"2024-04-23T19:13:34.911815+0000","last_active":"2024-04-23T19:13:34.911789+0000","last_peered":"2024-04-23T19:13:34.911789+0000","last_clean":"2024-04-23T19:13:34.911789+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:13:34.911789+0000","last_undegraded":"2024-04-23T19:13:34.911789+0000","last_fullsized":"2024-04-23T19:13:34.911789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.911697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:51.912852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.426732+0000","last_change":"2024-04-23T19:14:25.426761+0000","last_active":"2024-04-23T19:14:25.426732+0000","last_peered":"2024-04-23T19:14:25.426732+0000","last_clean":"2024-04-23T19:14:25.426732+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:14:25.426732+0000","last_undegraded":"2024-04-23T19:14:25.426732+0000","last_fullsized":"2024-04-23T19:14:25.426732+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:25.426641+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:25.426641+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.426641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:20.568398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.976956+0000","last_change":"2024-04-23T19:14:10.976979+0000","last_active":"2024-04-23T19:14:10.976956+0000","last_peered":"2024-04-23T19:14:10.976956+0000","last_clean":"2024-04-23T19:14:10.976956+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:14:10.976956+0000","last_undegraded":"2024-04-23T19:14:10.976956+0000","last_fullsized":"2024-04-23T19:14:10.976956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:10.976867+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:10.976867+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.976867+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:05.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.919662+0000","last_change":"2024-04-23T19:14:23.919693+0000","last_active":"2024-04-23T19:14:23.919662+0000","last_peered":"2024-04-23T19:14:23.919662+0000","last_clean":"2024-04-23T19:14:23.919662+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:14:23.919662+0000","last_undegraded":"2024-04-23T19:14:23.919662+0000","last_fullsized":"2024-04-23T19:14:23.919662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:23.919566+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:23.919566+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.919566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:55.156177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.612267+0000","last_change":"2024-04-23T19:13:44.612314+0000","last_active":"2024-04-23T19:13:44.612267+0000","last_peered":"2024-04-23T19:13:44.612267+0000","last_clean":"2024-04-23T19:13:44.612267+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:13:44.612267+0000","last_undegraded":"2024-04-23T19:13:44.612267+0000","last_fullsized":"2024-04-23T19:13:44.612267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.612159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:59.958714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'16","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.181820+0000","last_change":"2024-04-23T19:13:36.181858+0000","last_active":"2024-04-23T19:13:36.181820+0000","last_peered":"2024-04-23T19:13:36.181820+0000","last_clean":"2024-04-23T19:13:36.181820+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:13:36.181820+0000","last_undegraded":"2024-04-23T19:13:36.181820+0000","last_fullsized":"2024-04-23T19:13:36.181820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.181735+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:21.509377+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'286","reported_seq":398,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.939272+0000","last_change":"2024-04-23T19:14:05.939305+0000","last_active":"2024-04-23T19:14:05.939272+0000","last_peered":"2024-04-23T19:14:05.939272+0000","last_clean":"2024-04-23T19:14:05.939272+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:14:05.939272+0000","last_undegraded":"2024-04-23T19:14:05.939272+0000","last_fullsized":"2024-04-23T19:14:05.939272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'286","last_scrub_stamp":"2024-04-23T19:14:05.939186+0000","last_deep_scrub":"32'286","last_deep_scrub_stamp":"2024-04-23T19:14:05.939186+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.939186+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:33.131024+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.127003+0000","last_change":"2024-04-23T19:14:17.127033+0000","last_active":"2024-04-23T19:14:17.127003+0000","last_peered":"2024-04-23T19:14:17.127003+0000","last_clean":"2024-04-23T19:14:17.127003+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:14:17.127003+0000","last_undegraded":"2024-04-23T19:14:17.127003+0000","last_fullsized":"2024-04-23T19:14:17.127003+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:17.126915+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:17.126915+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.126915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:46:04.993448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.491090+0000","last_change":"2024-04-23T19:14:21.491116+0000","last_active":"2024-04-23T19:14:21.491090+0000","last_peered":"2024-04-23T19:14:21.491090+0000","last_clean":"2024-04-23T19:14:21.491090+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:14:21.491090+0000","last_undegraded":"2024-04-23T19:14:21.491090+0000","last_fullsized":"2024-04-23T19:14:21.491090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:21.490995+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:21.490995+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.490995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:39.906047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.907417+0000","last_change":"2024-04-23T19:14:01.907441+0000","last_active":"2024-04-23T19:14:01.907417+0000","last_peered":"2024-04-23T19:14:01.907417+0000","last_clean":"2024-04-23T19:14:01.907417+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:14:01.907417+0000","last_undegraded":"2024-04-23T19:14:01.907417+0000","last_fullsized":"2024-04-23T19:14:01.907417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:01.907314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:01.907314+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.907314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:33:49.391209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.940488+0000","last_change":"2024-04-23T19:13:34.940545+0000","last_active":"2024-04-23T19:13:34.940488+0000","last_peered":"2024-04-23T19:13:34.940488+0000","last_clean":"2024-04-23T19:13:34.940488+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:13:34.940488+0000","last_undegraded":"2024-04-23T19:13:34.940488+0000","last_fullsized":"2024-04-23T19:13:34.940488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.940371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:19.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.236705+0000","last_change":"2024-04-23T19:13:34.236743+0000","last_active":"2024-04-23T19:13:34.236705+0000","last_peered":"2024-04-23T19:13:34.236705+0000","last_clean":"2024-04-23T19:13:34.236705+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:13:34.236705+0000","last_undegraded":"2024-04-23T19:13:34.236705+0000","last_fullsized":"2024-04-23T19:13:34.236705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.236613+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:39.299958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'285","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.684726+0000","last_change":"2024-04-23T19:13:32.684754+0000","last_active":"2024-04-23T19:13:32.684726+0000","last_peered":"2024-04-23T19:13:32.684726+0000","last_clean":"2024-04-23T19:13:32.684726+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:32.684726+0000","last_undegraded":"2024-04-23T19:13:32.684726+0000","last_fullsized":"2024-04-23T19:13:32.684726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.684622+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:16.152805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.163019+0000","last_change":"2024-04-23T19:14:20.163047+0000","last_active":"2024-04-23T19:14:20.163019+0000","last_peered":"2024-04-23T19:14:20.163019+0000","last_clean":"2024-04-23T19:14:20.163019+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:14:20.163019+0000","last_undegraded":"2024-04-23T19:14:20.163019+0000","last_fullsized":"2024-04-23T19:14:20.163019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:20.162930+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:20.162930+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.162930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:09.380533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.053066+0000","last_change":"2024-04-23T19:14:13.053095+0000","last_active":"2024-04-23T19:14:13.053066+0000","last_peered":"2024-04-23T19:14:13.053066+0000","last_clean":"2024-04-23T19:14:13.053066+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:14:13.053066+0000","last_undegraded":"2024-04-23T19:14:13.053066+0000","last_fullsized":"2024-04-23T19:14:13.053066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:13.052973+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:13.052973+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.052973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:02.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.919126+0000","last_change":"2024-04-23T19:13:40.919152+0000","last_active":"2024-04-23T19:13:40.919126+0000","last_peered":"2024-04-23T19:13:40.919126+0000","last_clean":"2024-04-23T19:13:40.919126+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:13:40.919126+0000","last_undegraded":"2024-04-23T19:13:40.919126+0000","last_fullsized":"2024-04-23T19:13:40.919126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.919038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:35.160636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'51","reported_seq":111,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.205995+0000","last_change":"2024-04-23T19:13:39.206079+0000","last_active":"2024-04-23T19:13:39.205995+0000","last_peered":"2024-04-23T19:13:39.205995+0000","last_clean":"2024-04-23T19:13:39.205995+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:13:39.205995+0000","last_undegraded":"2024-04-23T19:13:39.205995+0000","last_fullsized":"2024-04-23T19:13:39.205995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'51","last_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_deep_scrub":"32'51","last_deep_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.205895+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:03.299394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.646105+0000","last_change":"2024-04-23T19:13:43.646133+0000","last_active":"2024-04-23T19:13:43.646105+0000","last_peered":"2024-04-23T19:13:43.646105+0000","last_clean":"2024-04-23T19:13:43.646105+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:13:43.646105+0000","last_undegraded":"2024-04-23T19:13:43.646105+0000","last_fullsized":"2024-04-23T19:13:43.646105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.646006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:44.344504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.099191+0000","last_change":"2024-04-23T19:14:16.099221+0000","last_active":"2024-04-23T19:14:16.099191+0000","last_peered":"2024-04-23T19:14:16.099191+0000","last_clean":"2024-04-23T19:14:16.099191+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:14:16.099191+0000","last_undegraded":"2024-04-23T19:14:16.099191+0000","last_fullsized":"2024-04-23T19:14:16.099191+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:16.099098+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:16.099098+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.099098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:32.839653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.835784+0000","last_change":"2024-04-23T19:13:58.835852+0000","last_active":"2024-04-23T19:13:58.835784+0000","last_peered":"2024-04-23T19:13:58.835784+0000","last_clean":"2024-04-23T19:13:58.835784+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:58.835784+0000","last_undegraded":"2024-04-23T19:13:58.835784+0000","last_fullsized":"2024-04-23T19:13:58.835784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.835665+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:05.136073+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.024239+0000","last_change":"2024-04-23T19:13:54.024266+0000","last_active":"2024-04-23T19:13:54.024239+0000","last_peered":"2024-04-23T19:13:54.024239+0000","last_clean":"2024-04-23T19:13:54.024239+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:13:54.024239+0000","last_undegraded":"2024-04-23T19:13:54.024239+0000","last_fullsized":"2024-04-23T19:13:54.024239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.024146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:24.982777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.680623+0000","last_change":"2024-04-23T19:13:42.680645+0000","last_active":"2024-04-23T19:13:42.680623+0000","last_peered":"2024-04-23T19:13:42.680623+0000","last_clean":"2024-04-23T19:13:42.680623+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:13:42.680623+0000","last_undegraded":"2024-04-23T19:13:42.680623+0000","last_fullsized":"2024-04-23T19:13:42.680623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.680534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:03.703466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'38","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.936866+0000","last_change":"2024-04-23T19:13:58.936900+0000","last_active":"2024-04-23T19:13:58.936866+0000","last_peered":"2024-04-23T19:13:58.936866+0000","last_clean":"2024-04-23T19:13:58.936866+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:13:58.936866+0000","last_undegraded":"2024-04-23T19:13:58.936866+0000","last_fullsized":"2024-04-23T19:13:58.936866+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.936780+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:49.313649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.585103+0000","last_change":"2024-04-23T19:13:28.585132+0000","last_active":"2024-04-23T19:13:28.585103+0000","last_peered":"2024-04-23T19:13:28.585103+0000","last_clean":"2024-04-23T19:13:28.585103+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:13:28.585103+0000","last_undegraded":"2024-04-23T19:13:28.585103+0000","last_fullsized":"2024-04-23T19:13:28.585103+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.585004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:05.071466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'6","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.768856+0000","last_change":"2024-04-23T19:14:24.768856+0000","last_active":"2024-04-23T19:14:24.768856+0000","last_peered":"2024-04-23T19:14:24.768856+0000","last_clean":"2024-04-23T19:14:24.768856+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:14:24.768856+0000","last_undegraded":"2024-04-23T19:14:24.768856+0000","last_fullsized":"2024-04-23T19:14:24.768856+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:24.768667+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:24.768667+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.768667+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:28.438866+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":88,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.098620+0000","last_change":"2024-04-23T19:14:01.098652+0000","last_active":"2024-04-23T19:14:01.098620+0000","last_peered":"2024-04-23T19:14:01.098620+0000","last_clean":"2024-04-23T19:14:01.098620+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:14:01.098620+0000","last_undegraded":"2024-04-23T19:14:01.098620+0000","last_fullsized":"2024-04-23T19:14:01.098620+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.098506+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:43.162965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719245+0000","last_change":"2024-04-23T19:10:07.492637+0000","last_active":"2024-04-23T19:12:21.719245+0000","last_peered":"2024-04-23T19:12:21.719245+0000","last_clean":"2024-04-23T19:12:21.719245+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:12:21.719245+0000","last_undegraded":"2024-04-23T19:12:21.719245+0000","last_fullsized":"2024-04-23T19:12:21.719245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.573918+0000","last_change":"2024-04-23T19:13:45.573946+0000","last_active":"2024-04-23T19:13:45.573918+0000","last_peered":"2024-04-23T19:13:45.573918+0000","last_clean":"2024-04-23T19:13:45.573918+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:13:45.573918+0000","last_undegraded":"2024-04-23T19:13:45.573918+0000","last_fullsized":"2024-04-23T19:13:45.573918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.573803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:16.532433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'45","reported_seq":103,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.889391+0000","last_change":"2024-04-23T19:13:50.889437+0000","last_active":"2024-04-23T19:13:50.889391+0000","last_peered":"2024-04-23T19:13:50.889391+0000","last_clean":"2024-04-23T19:13:50.889391+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:13:50.889391+0000","last_undegraded":"2024-04-23T19:13:50.889391+0000","last_fullsized":"2024-04-23T19:13:50.889391+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.889290+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:52.811988+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'285","reported_seq":360,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.512179+0000","last_change":"2024-04-23T19:13:51.512239+0000","last_active":"2024-04-23T19:13:51.512179+0000","last_peered":"2024-04-23T19:13:51.512179+0000","last_clean":"2024-04-23T19:13:51.512179+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:51.512179+0000","last_undegraded":"2024-04-23T19:13:51.512179+0000","last_fullsized":"2024-04-23T19:13:51.512179+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.512072+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:26.148887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:30.713313+0000","last_change":"2024-04-23T19:10:17.520791+0000","last_active":"2024-04-23T19:12:30.713313+0000","last_peered":"2024-04-23T19:12:30.713313+0000","last_clean":"2024-04-23T19:12:30.713313+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:12:30.713313+0000","last_undegraded":"2024-04-23T19:12:30.713313+0000","last_fullsized":"2024-04-23T19:12:30.713313+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.092362+0000","last_change":"2024-04-23T19:13:51.092424+0000","last_active":"2024-04-23T19:13:51.092362+0000","last_peered":"2024-04-23T19:13:51.092362+0000","last_clean":"2024-04-23T19:13:51.092362+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:51.092362+0000","last_undegraded":"2024-04-23T19:13:51.092362+0000","last_fullsized":"2024-04-23T19:13:51.092362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.092261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:12.115193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719160+0000","last_change":"2024-04-23T19:10:07.501083+0000","last_active":"2024-04-23T19:12:21.719160+0000","last_peered":"2024-04-23T19:12:21.719160+0000","last_clean":"2024-04-23T19:12:21.719160+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:12:21.719160+0000","last_undegraded":"2024-04-23T19:12:21.719160+0000","last_fullsized":"2024-04-23T19:12:21.719160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.667015+0000","last_change":"2024-04-23T19:13:41.667080+0000","last_active":"2024-04-23T19:13:41.667015+0000","last_peered":"2024-04-23T19:13:41.667015+0000","last_clean":"2024-04-23T19:13:41.667015+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:13:41.667015+0000","last_undegraded":"2024-04-23T19:13:41.667015+0000","last_fullsized":"2024-04-23T19:13:41.667015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.666920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:52.673059+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.975145+0000","last_change":"2024-04-23T19:13:33.975191+0000","last_active":"2024-04-23T19:13:33.975145+0000","last_peered":"2024-04-23T19:13:33.975145+0000","last_clean":"2024-04-23T19:13:33.975145+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:13:33.975145+0000","last_undegraded":"2024-04-23T19:13:33.975145+0000","last_fullsized":"2024-04-23T19:13:33.975145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.975049+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:07.187015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.695421+0000","last_change":"2024-04-23T19:13:35.695444+0000","last_active":"2024-04-23T19:13:35.695421+0000","last_peered":"2024-04-23T19:13:35.695421+0000","last_clean":"2024-04-23T19:13:35.695421+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:13:35.695421+0000","last_undegraded":"2024-04-23T19:13:35.695421+0000","last_fullsized":"2024-04-23T19:13:35.695421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.695326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:43.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'21","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.533999+0000","last_change":"2024-04-23T19:14:22.534034+0000","last_active":"2024-04-23T19:14:22.533999+0000","last_peered":"2024-04-23T19:14:22.533999+0000","last_clean":"2024-04-23T19:14:22.533999+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:14:22.533999+0000","last_undegraded":"2024-04-23T19:14:22.533999+0000","last_fullsized":"2024-04-23T19:14:22.533999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:14:22.533903+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:14:22.533903+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.533903+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:26.204102+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.796778+0000","last_change":"2024-04-23T19:13:55.796803+0000","last_active":"2024-04-23T19:13:55.796778+0000","last_peered":"2024-04-23T19:13:55.796778+0000","last_clean":"2024-04-23T19:13:55.796778+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:55.796778+0000","last_undegraded":"2024-04-23T19:13:55.796778+0000","last_fullsized":"2024-04-23T19:13:55.796778+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.796686+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:51:24.793617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.565727+0000","last_change":"2024-04-23T19:14:10.565751+0000","last_active":"2024-04-23T19:14:10.565727+0000","last_peered":"2024-04-23T19:14:10.565727+0000","last_clean":"2024-04-23T19:14:10.565727+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:14:10.565727+0000","last_undegraded":"2024-04-23T19:14:10.565727+0000","last_fullsized":"2024-04-23T19:14:10.565727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:10.565616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:10.565616+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.565616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:31.337130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.190333+0000","last_change":"2024-04-23T19:13:44.190374+0000","last_active":"2024-04-23T19:13:44.190333+0000","last_peered":"2024-04-23T19:13:44.190333+0000","last_clean":"2024-04-23T19:13:44.190333+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:13:44.190333+0000","last_undegraded":"2024-04-23T19:13:44.190333+0000","last_fullsized":"2024-04-23T19:13:44.190333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.190242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:32.140502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.877432+0000","last_change":"2024-04-23T19:13:33.877469+0000","last_active":"2024-04-23T19:13:33.877432+0000","last_peered":"2024-04-23T19:13:33.877432+0000","last_clean":"2024-04-23T19:13:33.877432+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:13:33.877432+0000","last_undegraded":"2024-04-23T19:13:33.877432+0000","last_fullsized":"2024-04-23T19:13:33.877432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.877330+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:17.345513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.951003+0000","last_change":"2024-04-23T19:13:48.951037+0000","last_active":"2024-04-23T19:13:48.951003+0000","last_peered":"2024-04-23T19:13:48.951003+0000","last_clean":"2024-04-23T19:13:48.951003+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:13:48.951003+0000","last_undegraded":"2024-04-23T19:13:48.951003+0000","last_fullsized":"2024-04-23T19:13:48.951003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.950905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:41.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'4","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.014390+0000","last_change":"2024-04-23T19:14:10.014390+0000","last_active":"2024-04-23T19:14:10.014390+0000","last_peered":"2024-04-23T19:14:10.014390+0000","last_clean":"2024-04-23T19:14:10.014390+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:14:10.014390+0000","last_undegraded":"2024-04-23T19:14:10.014390+0000","last_fullsized":"2024-04-23T19:14:10.014390+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:10.014222+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:10.014222+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.014222+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:00:32.382010+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.919986+0000","last_change":"2024-04-23T19:13:42.920017+0000","last_active":"2024-04-23T19:13:42.919986+0000","last_peered":"2024-04-23T19:13:42.919986+0000","last_clean":"2024-04-23T19:13:42.919986+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:42.919986+0000","last_undegraded":"2024-04-23T19:13:42.919986+0000","last_fullsized":"2024-04-23T19:13:42.919986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.919888+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:33.023562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400279+0000","last_change":"2024-04-23T19:10:07.494969+0000","last_active":"2024-04-23T19:13:27.400279+0000","last_peered":"2024-04-23T19:13:27.400279+0000","last_clean":"2024-04-23T19:13:27.400279+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:13:27.400279+0000","last_undegraded":"2024-04-23T19:13:27.400279+0000","last_fullsized":"2024-04-23T19:13:27.400279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.565484+0000","last_change":"2024-04-23T19:13:46.565530+0000","last_active":"2024-04-23T19:13:46.565484+0000","last_peered":"2024-04-23T19:13:46.565484+0000","last_clean":"2024-04-23T19:13:46.565484+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:13:46.565484+0000","last_undegraded":"2024-04-23T19:13:46.565484+0000","last_fullsized":"2024-04-23T19:13:46.565484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.565380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:54.525905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'45","reported_seq":103,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.188689+0000","last_change":"2024-04-23T19:13:40.188736+0000","last_active":"2024-04-23T19:13:40.188689+0000","last_peered":"2024-04-23T19:13:40.188689+0000","last_clean":"2024-04-23T19:13:40.188689+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:13:40.188689+0000","last_undegraded":"2024-04-23T19:13:40.188689+0000","last_fullsized":"2024-04-23T19:13:40.188689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.188598+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:54.836675+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'285","reported_seq":415,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.128944+0000","last_change":"2024-04-23T19:13:28.128972+0000","last_active":"2024-04-23T19:13:28.128944+0000","last_peered":"2024-04-23T19:13:28.128944+0000","last_clean":"2024-04-23T19:13:28.128944+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:28.128944+0000","last_undegraded":"2024-04-23T19:13:28.128944+0000","last_fullsized":"2024-04-23T19:13:28.128944+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.128856+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:09.022827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'13","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.464595+0000","last_change":"2024-04-23T19:14:24.464629+0000","last_active":"2024-04-23T19:14:24.464595+0000","last_peered":"2024-04-23T19:14:24.464595+0000","last_clean":"2024-04-23T19:14:24.464595+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:14:24.464595+0000","last_undegraded":"2024-04-23T19:14:24.464595+0000","last_fullsized":"2024-04-23T19:14:24.464595+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'13","last_scrub_stamp":"2024-04-23T19:14:24.464510+0000","last_deep_scrub":"32'13","last_deep_scrub_stamp":"2024-04-23T19:14:24.464510+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.464510+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:40:14.783844+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.008589+0000","last_change":"2024-04-23T19:14:05.008630+0000","last_active":"2024-04-23T19:14:05.008589+0000","last_peered":"2024-04-23T19:14:05.008589+0000","last_clean":"2024-04-23T19:14:05.008589+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:14:05.008589+0000","last_undegraded":"2024-04-23T19:14:05.008589+0000","last_fullsized":"2024-04-23T19:14:05.008589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'20","last_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_deep_scrub":"32'20","last_deep_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.008496+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:11.308903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.997873+0000","last_change":"2024-04-23T19:14:16.997873+0000","last_active":"2024-04-23T19:14:16.997873+0000","last_peered":"2024-04-23T19:14:16.997873+0000","last_clean":"2024-04-23T19:14:16.997873+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:14:16.997873+0000","last_undegraded":"2024-04-23T19:14:16.997873+0000","last_fullsized":"2024-04-23T19:14:16.997873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:16.997750+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:16.997750+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.997750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:57:56.974216+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'5","reported_seq":428,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.911991+0000","last_change":"2024-04-23T19:13:37.912027+0000","last_active":"2024-04-23T19:13:37.911991+0000","last_peered":"2024-04-23T19:13:37.911991+0000","last_clean":"2024-04-23T19:13:37.911991+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:13:37.911991+0000","last_undegraded":"2024-04-23T19:13:37.911991+0000","last_fullsized":"2024-04-23T19:13:37.911991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.911892+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:23.295978+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"32'15","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.923002+0000","last_change":"2024-04-23T19:13:45.923039+0000","last_active":"2024-04-23T19:13:45.923002+0000","last_peered":"2024-04-23T19:13:45.923002+0000","last_clean":"2024-04-23T19:13:45.923002+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:13:45.923002+0000","last_undegraded":"2024-04-23T19:13:45.923002+0000","last_fullsized":"2024-04-23T19:13:45.923002+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.922918+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:04.148244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'285","reported_seq":477,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.784157+0000","last_change":"2024-04-23T19:13:26.784184+0000","last_active":"2024-04-23T19:13:26.784157+0000","last_peered":"2024-04-23T19:13:26.784157+0000","last_clean":"2024-04-23T19:13:26.784157+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:26.784157+0000","last_undegraded":"2024-04-23T19:13:26.784157+0000","last_fullsized":"2024-04-23T19:13:26.784157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.784064+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:31.847408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.211829+0000","last_change":"2024-04-23T19:13:41.211854+0000","last_active":"2024-04-23T19:13:41.211829+0000","last_peered":"2024-04-23T19:13:41.211829+0000","last_clean":"2024-04-23T19:13:41.211829+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:13:41.211829+0000","last_undegraded":"2024-04-23T19:13:41.211829+0000","last_fullsized":"2024-04-23T19:13:41.211829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.211740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.450923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.738274+0000","last_change":"2024-04-23T19:13:48.738341+0000","last_active":"2024-04-23T19:13:48.738274+0000","last_peered":"2024-04-23T19:13:48.738274+0000","last_clean":"2024-04-23T19:13:48.738274+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:13:48.738274+0000","last_undegraded":"2024-04-23T19:13:48.738274+0000","last_fullsized":"2024-04-23T19:13:48.738274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.738148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:12.860218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400081+0000","last_change":"2024-04-23T19:10:07.507496+0000","last_active":"2024-04-23T19:13:27.400081+0000","last_peered":"2024-04-23T19:13:27.400081+0000","last_clean":"2024-04-23T19:13:27.400081+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:13:27.400081+0000","last_undegraded":"2024-04-23T19:13:27.400081+0000","last_fullsized":"2024-04-23T19:13:27.400081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.111154+0000","last_change":"2024-04-23T19:13:28.111184+0000","last_active":"2024-04-23T19:13:28.111154+0000","last_peered":"2024-04-23T19:13:28.111154+0000","last_clean":"2024-04-23T19:13:28.111154+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:13:28.111154+0000","last_undegraded":"2024-04-23T19:13:28.111154+0000","last_fullsized":"2024-04-23T19:13:28.111154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.111056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:18.387489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'48","reported_seq":107,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.948761+0000","last_change":"2024-04-23T19:13:35.948796+0000","last_active":"2024-04-23T19:13:35.948761+0000","last_peered":"2024-04-23T19:13:35.948761+0000","last_clean":"2024-04-23T19:13:35.948761+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:13:35.948761+0000","last_undegraded":"2024-04-23T19:13:35.948761+0000","last_fullsized":"2024-04-23T19:13:35.948761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'48","last_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_deep_scrub":"32'48","last_deep_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.948676+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:52.110221+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.116201+0000","last_change":"2024-04-23T19:13:30.116258+0000","last_active":"2024-04-23T19:13:30.116201+0000","last_peered":"2024-04-23T19:13:30.116201+0000","last_clean":"2024-04-23T19:13:30.116201+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:13:30.116201+0000","last_undegraded":"2024-04-23T19:13:30.116201+0000","last_fullsized":"2024-04-23T19:13:30.116201+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.116108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:58.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'7","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.725037+0000","last_change":"2024-04-23T19:13:46.725037+0000","last_active":"2024-04-23T19:13:46.725037+0000","last_peered":"2024-04-23T19:13:46.725037+0000","last_clean":"2024-04-23T19:13:46.725037+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:46.725037+0000","last_undegraded":"2024-04-23T19:13:46.725037+0000","last_fullsized":"2024-04-23T19:13:46.725037+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'7","last_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_deep_scrub":"32'7","last_deep_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.724881+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:17.683511+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.894509+0000","last_change":"2024-04-23T19:13:40.894509+0000","last_active":"2024-04-23T19:13:40.894509+0000","last_peered":"2024-04-23T19:13:40.894509+0000","last_clean":"2024-04-23T19:13:40.894509+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:13:40.894509+0000","last_undegraded":"2024-04-23T19:13:40.894509+0000","last_fullsized":"2024-04-23T19:13:40.894509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.894372+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:11.403485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"32'119","reported_seq":213,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.926283+0000","last_change":"2024-04-23T19:14:12.926325+0000","last_active":"2024-04-23T19:14:12.926283+0000","last_peered":"2024-04-23T19:14:12.926283+0000","last_clean":"2024-04-23T19:14:12.926283+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:14:12.926283+0000","last_undegraded":"2024-04-23T19:14:12.926283+0000","last_fullsized":"2024-04-23T19:14:12.926283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'119","last_scrub_stamp":"2024-04-23T19:14:12.926188+0000","last_deep_scrub":"32'119","last_deep_scrub_stamp":"2024-04-23T19:14:12.926188+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.926188+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:18.589862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400006+0000","last_change":"2024-04-23T19:09:59.449299+0000","last_active":"2024-04-23T19:13:27.400006+0000","last_peered":"2024-04-23T19:13:27.400006+0000","last_clean":"2024-04-23T19:13:27.400006+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:13:27.400006+0000","last_undegraded":"2024-04-23T19:13:27.400006+0000","last_fullsized":"2024-04-23T19:13:27.400006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.987464+0000","last_change":"2024-04-23T19:14:15.987464+0000","last_active":"2024-04-23T19:14:15.987464+0000","last_peered":"2024-04-23T19:14:15.987464+0000","last_clean":"2024-04-23T19:14:15.987464+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:14:15.987464+0000","last_undegraded":"2024-04-23T19:14:15.987464+0000","last_fullsized":"2024-04-23T19:14:15.987464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:15.987336+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:15.987336+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.987336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:54.753507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.842231+0000","last_change":"2024-04-23T19:13:43.842231+0000","last_active":"2024-04-23T19:13:43.842231+0000","last_peered":"2024-04-23T19:13:43.842231+0000","last_clean":"2024-04-23T19:13:43.842231+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:13:43.842231+0000","last_undegraded":"2024-04-23T19:13:43.842231+0000","last_fullsized":"2024-04-23T19:13:43.842231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.842100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:04.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'88","reported_seq":178,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.876527+0000","last_change":"2024-04-23T19:13:37.876563+0000","last_active":"2024-04-23T19:13:37.876527+0000","last_peered":"2024-04-23T19:13:37.876527+0000","last_clean":"2024-04-23T19:13:37.876527+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:13:37.876527+0000","last_undegraded":"2024-04-23T19:13:37.876527+0000","last_fullsized":"2024-04-23T19:13:37.876527+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'88","last_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_deep_scrub":"32'88","last_deep_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.876439+0000","objects_scrubbed":5,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:33:54.212735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.621369+0000","last_change":"2024-04-23T19:13:29.621413+0000","last_active":"2024-04-23T19:13:29.621369+0000","last_peered":"2024-04-23T19:13:29.621369+0000","last_clean":"2024-04-23T19:13:29.621369+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:13:29.621369+0000","last_undegraded":"2024-04-23T19:13:29.621369+0000","last_fullsized":"2024-04-23T19:13:29.621369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.621264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:03.242868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":171,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.101439+0000","last_change":"2024-04-23T19:13:49.101471+0000","last_active":"2024-04-23T19:13:49.101439+0000","last_peered":"2024-04-23T19:13:49.101439+0000","last_clean":"2024-04-23T19:13:49.101439+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:49.101439+0000","last_undegraded":"2024-04-23T19:13:49.101439+0000","last_fullsized":"2024-04-23T19:13:49.101439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'81","last_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_deep_scrub":"32'81","last_deep_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.101341+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:49.219878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.030866+0000","last_change":"2024-04-23T19:14:13.030892+0000","last_active":"2024-04-23T19:14:13.030866+0000","last_peered":"2024-04-23T19:14:13.030866+0000","last_clean":"2024-04-23T19:14:13.030866+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:14:13.030866+0000","last_undegraded":"2024-04-23T19:14:13.030866+0000","last_fullsized":"2024-04-23T19:14:13.030866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:13.030773+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:13.030773+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.030773+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:24.199482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400043+0000","last_change":"2024-04-23T19:09:59.449756+0000","last_active":"2024-04-23T19:13:27.400043+0000","last_peered":"2024-04-23T19:13:27.400043+0000","last_clean":"2024-04-23T19:13:27.400043+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:13:27.400043+0000","last_undegraded":"2024-04-23T19:13:27.400043+0000","last_fullsized":"2024-04-23T19:13:27.400043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.550545+0000","last_change":"2024-04-23T19:14:16.550545+0000","last_active":"2024-04-23T19:14:16.550545+0000","last_peered":"2024-04-23T19:14:16.550545+0000","last_clean":"2024-04-23T19:14:16.550545+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:14:16.550545+0000","last_undegraded":"2024-04-23T19:14:16.550545+0000","last_fullsized":"2024-04-23T19:14:16.550545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:16.550416+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:16.550416+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.550416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:59.355007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.891584+0000","last_change":"2024-04-23T19:13:36.891613+0000","last_active":"2024-04-23T19:13:36.891584+0000","last_peered":"2024-04-23T19:13:36.891584+0000","last_clean":"2024-04-23T19:13:36.891584+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:13:36.891584+0000","last_undegraded":"2024-04-23T19:13:36.891584+0000","last_fullsized":"2024-04-23T19:13:36.891584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.891497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:43.191904+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'290","reported_seq":516,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.753118+0000","last_change":"2024-04-23T19:13:28.753118+0000","last_active":"2024-04-23T19:13:28.753118+0000","last_peered":"2024-04-23T19:13:28.753118+0000","last_clean":"2024-04-23T19:13:28.753118+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:28.753118+0000","last_undegraded":"2024-04-23T19:13:28.753118+0000","last_fullsized":"2024-04-23T19:13:28.753118+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'290","last_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_deep_scrub":"32'290","last_deep_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.752962+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:14.080605+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.883869+0000","last_change":"2024-04-23T19:13:53.883869+0000","last_active":"2024-04-23T19:13:53.883869+0000","last_peered":"2024-04-23T19:13:53.883869+0000","last_clean":"2024-04-23T19:13:53.883869+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:13:53.883869+0000","last_undegraded":"2024-04-23T19:13:53.883869+0000","last_fullsized":"2024-04-23T19:13:53.883869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.883735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:38.160620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.725674+0000","last_change":"2024-04-23T19:14:25.725710+0000","last_active":"2024-04-23T19:14:25.725674+0000","last_peered":"2024-04-23T19:14:25.725674+0000","last_clean":"2024-04-23T19:14:25.725674+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:14:25.725674+0000","last_undegraded":"2024-04-23T19:14:25.725674+0000","last_fullsized":"2024-04-23T19:14:25.725674+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:25.725582+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:25.725582+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.725582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:55.273423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":75,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.586378+0000","last_change":"2024-04-23T19:14:09.586425+0000","last_active":"2024-04-23T19:14:09.586378+0000","last_peered":"2024-04-23T19:14:09.586378+0000","last_clean":"2024-04-23T19:14:09.586378+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:14:09.586378+0000","last_undegraded":"2024-04-23T19:14:09.586378+0000","last_fullsized":"2024-04-23T19:14:09.586378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:09.586258+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:09.586258+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.586258+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:08.710830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.006366+0000","last_change":"2024-04-23T19:14:07.006366+0000","last_active":"2024-04-23T19:14:07.006366+0000","last_peered":"2024-04-23T19:14:07.006366+0000","last_clean":"2024-04-23T19:14:07.006366+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:14:07.006366+0000","last_undegraded":"2024-04-23T19:14:07.006366+0000","last_fullsized":"2024-04-23T19:14:07.006366+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.006243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.006243+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.006243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:46.529613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.504995+0000","last_change":"2024-04-23T19:13:58.504995+0000","last_active":"2024-04-23T19:13:58.504995+0000","last_peered":"2024-04-23T19:13:58.504995+0000","last_clean":"2024-04-23T19:13:58.504995+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:13:58.504995+0000","last_undegraded":"2024-04-23T19:13:58.504995+0000","last_fullsized":"2024-04-23T19:13:58.504995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.504872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:38.028243+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.171370+0000","last_change":"2024-04-23T19:14:22.171419+0000","last_active":"2024-04-23T19:14:22.171370+0000","last_peered":"2024-04-23T19:14:22.171370+0000","last_clean":"2024-04-23T19:14:22.171370+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:14:22.171370+0000","last_undegraded":"2024-04-23T19:14:22.171370+0000","last_fullsized":"2024-04-23T19:14:22.171370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:22.171266+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:22.171266+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.171266+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:06.448151+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'23","reported_seq":101,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.228773+0000","last_change":"2024-04-23T19:13:38.228773+0000","last_active":"2024-04-23T19:13:38.228773+0000","last_peered":"2024-04-23T19:13:38.228773+0000","last_clean":"2024-04-23T19:13:38.228773+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:38.228773+0000","last_undegraded":"2024-04-23T19:13:38.228773+0000","last_fullsized":"2024-04-23T19:13:38.228773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'23","last_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_deep_scrub":"32'23","last_deep_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.228582+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:14.715044+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.555830+0000","last_change":"2024-04-23T19:13:26.555830+0000","last_active":"2024-04-23T19:13:26.555830+0000","last_peered":"2024-04-23T19:13:26.555830+0000","last_clean":"2024-04-23T19:13:26.555830+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:13:26.555830+0000","last_undegraded":"2024-04-23T19:13:26.555830+0000","last_fullsized":"2024-04-23T19:13:26.555830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.555714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:11.883356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720790+0000","last_change":"2024-04-23T19:10:17.517093+0000","last_active":"2024-04-23T19:12:31.720790+0000","last_peered":"2024-04-23T19:12:31.720790+0000","last_clean":"2024-04-23T19:12:31.720790+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:12:31.720790+0000","last_undegraded":"2024-04-23T19:12:31.720790+0000","last_fullsized":"2024-04-23T19:12:31.720790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.526311+0000","last_change":"2024-04-23T19:13:59.526311+0000","last_active":"2024-04-23T19:13:59.526311+0000","last_peered":"2024-04-23T19:13:59.526311+0000","last_clean":"2024-04-23T19:13:59.526311+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:59.526311+0000","last_undegraded":"2024-04-23T19:13:59.526311+0000","last_fullsized":"2024-04-23T19:13:59.526311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.526195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:39.758894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400056+0000","last_change":"2024-04-23T19:09:59.449687+0000","last_active":"2024-04-23T19:13:27.400056+0000","last_peered":"2024-04-23T19:13:27.400056+0000","last_clean":"2024-04-23T19:13:27.400056+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:13:27.400056+0000","last_undegraded":"2024-04-23T19:13:27.400056+0000","last_fullsized":"2024-04-23T19:13:27.400056+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.981539+0000","last_change":"2024-04-23T19:14:07.981539+0000","last_active":"2024-04-23T19:14:07.981539+0000","last_peered":"2024-04-23T19:14:07.981539+0000","last_clean":"2024-04-23T19:14:07.981539+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:14:07.981539+0000","last_undegraded":"2024-04-23T19:14:07.981539+0000","last_fullsized":"2024-04-23T19:14:07.981539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.981428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.981428+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.981428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:48.214128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.501453+0000","last_change":"2024-04-23T19:13:49.501453+0000","last_active":"2024-04-23T19:13:49.501453+0000","last_peered":"2024-04-23T19:13:49.501453+0000","last_clean":"2024-04-23T19:13:49.501453+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:13:49.501453+0000","last_undegraded":"2024-04-23T19:13:49.501453+0000","last_fullsized":"2024-04-23T19:13:49.501453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.501336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:21.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'21","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.866530+0000","last_change":"2024-04-23T19:13:39.866575+0000","last_active":"2024-04-23T19:13:39.866530+0000","last_peered":"2024-04-23T19:13:39.866530+0000","last_clean":"2024-04-23T19:13:39.866530+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:39.866530+0000","last_undegraded":"2024-04-23T19:13:39.866530+0000","last_fullsized":"2024-04-23T19:13:39.866530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.866421+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:20.099121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.919503+0000","last_change":"2024-04-23T19:13:47.919527+0000","last_active":"2024-04-23T19:13:47.919503+0000","last_peered":"2024-04-23T19:13:47.919503+0000","last_clean":"2024-04-23T19:13:47.919503+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:13:47.919503+0000","last_undegraded":"2024-04-23T19:13:47.919503+0000","last_fullsized":"2024-04-23T19:13:47.919503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.919412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:48.916350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.731424+0000","last_change":"2024-04-23T19:14:21.731452+0000","last_active":"2024-04-23T19:14:21.731424+0000","last_peered":"2024-04-23T19:14:21.731424+0000","last_clean":"2024-04-23T19:14:21.731424+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:14:21.731424+0000","last_undegraded":"2024-04-23T19:14:21.731424+0000","last_fullsized":"2024-04-23T19:14:21.731424+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:14:21.731320+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:14:21.731320+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.731320+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:12.108189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.694263+0000","last_change":"2024-04-23T19:13:47.694263+0000","last_active":"2024-04-23T19:13:47.694263+0000","last_peered":"2024-04-23T19:13:47.694263+0000","last_clean":"2024-04-23T19:13:47.694263+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:47.694263+0000","last_undegraded":"2024-04-23T19:13:47.694263+0000","last_fullsized":"2024-04-23T19:13:47.694263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.694146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:55.654671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.997935+0000","last_change":"2024-04-23T19:14:08.997935+0000","last_active":"2024-04-23T19:14:08.997935+0000","last_peered":"2024-04-23T19:14:08.997935+0000","last_clean":"2024-04-23T19:14:08.997935+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:14:08.997935+0000","last_undegraded":"2024-04-23T19:14:08.997935+0000","last_fullsized":"2024-04-23T19:14:08.997935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:08.997767+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:08.997767+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.997767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:39.506879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.514961+0000","last_change":"2024-04-23T19:14:13.514987+0000","last_active":"2024-04-23T19:14:13.514961+0000","last_peered":"2024-04-23T19:14:13.514961+0000","last_clean":"2024-04-23T19:14:13.514961+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:14:13.514961+0000","last_undegraded":"2024-04-23T19:14:13.514961+0000","last_fullsized":"2024-04-23T19:14:13.514961+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:13.514849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:13.514849+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.514849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:45.028155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'2","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.147860+0000","last_change":"2024-04-23T19:14:19.147888+0000","last_active":"2024-04-23T19:14:19.147860+0000","last_peered":"2024-04-23T19:14:19.147860+0000","last_clean":"2024-04-23T19:14:19.147860+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:14:19.147860+0000","last_undegraded":"2024-04-23T19:14:19.147860+0000","last_fullsized":"2024-04-23T19:14:19.147860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.147776+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.147776+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.147776+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:39.919289+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'75","reported_seq":152,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.702548+0000","last_change":"2024-04-23T19:13:34.702548+0000","last_active":"2024-04-23T19:13:34.702548+0000","last_peered":"2024-04-23T19:13:34.702548+0000","last_clean":"2024-04-23T19:13:34.702548+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:13:34.702548+0000","last_undegraded":"2024-04-23T19:13:34.702548+0000","last_fullsized":"2024-04-23T19:13:34.702548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'75","last_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_deep_scrub":"32'75","last_deep_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.702393+0000","objects_scrubbed":9,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:50.863938+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.162563+0000","last_change":"2024-04-23T19:13:31.162563+0000","last_active":"2024-04-23T19:13:31.162563+0000","last_peered":"2024-04-23T19:13:31.162563+0000","last_clean":"2024-04-23T19:13:31.162563+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:13:31.162563+0000","last_undegraded":"2024-04-23T19:13:31.162563+0000","last_fullsized":"2024-04-23T19:13:31.162563+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.162449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:47.422034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.538697+0000","last_change":"2024-04-23T19:13:55.538734+0000","last_active":"2024-04-23T19:13:55.538697+0000","last_peered":"2024-04-23T19:13:55.538697+0000","last_clean":"2024-04-23T19:13:55.538697+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:55.538697+0000","last_undegraded":"2024-04-23T19:13:55.538697+0000","last_fullsized":"2024-04-23T19:13:55.538697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.538607+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:42.408183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.742366+0000","last_change":"2024-04-23T19:13:51.742366+0000","last_active":"2024-04-23T19:13:51.742366+0000","last_peered":"2024-04-23T19:13:51.742366+0000","last_clean":"2024-04-23T19:13:51.742366+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:51.742366+0000","last_undegraded":"2024-04-23T19:13:51.742366+0000","last_fullsized":"2024-04-23T19:13:51.742366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.742223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:22.634791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.561186+0000","last_change":"2024-04-23T19:14:15.561186+0000","last_active":"2024-04-23T19:14:15.561186+0000","last_peered":"2024-04-23T19:14:15.561186+0000","last_clean":"2024-04-23T19:14:15.561186+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:14:15.561186+0000","last_undegraded":"2024-04-23T19:14:15.561186+0000","last_fullsized":"2024-04-23T19:14:15.561186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:15.561056+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:15.561056+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.561056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:42.489103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400030+0000","last_change":"2024-04-23T19:09:59.450239+0000","last_active":"2024-04-23T19:13:27.400030+0000","last_peered":"2024-04-23T19:13:27.400030+0000","last_clean":"2024-04-23T19:13:27.400030+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:13:27.400030+0000","last_undegraded":"2024-04-23T19:13:27.400030+0000","last_fullsized":"2024-04-23T19:13:27.400030+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.046354+0000","last_change":"2024-04-23T19:13:31.046354+0000","last_active":"2024-04-23T19:13:31.046354+0000","last_peered":"2024-04-23T19:13:31.046354+0000","last_clean":"2024-04-23T19:13:31.046354+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:13:31.046354+0000","last_undegraded":"2024-04-23T19:13:31.046354+0000","last_fullsized":"2024-04-23T19:13:31.046354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.046239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:01.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'22","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.762358+0000","last_change":"2024-04-23T19:13:29.762358+0000","last_active":"2024-04-23T19:13:29.762358+0000","last_peered":"2024-04-23T19:13:29.762358+0000","last_clean":"2024-04-23T19:13:29.762358+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:13:29.762358+0000","last_undegraded":"2024-04-23T19:13:29.762358+0000","last_fullsized":"2024-04-23T19:13:29.762358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'22","last_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_deep_scrub":"32'22","last_deep_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.762203+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:37.849358+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'285","reported_seq":368,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.782396+0000","last_change":"2024-04-23T19:13:30.782396+0000","last_active":"2024-04-23T19:13:30.782396+0000","last_peered":"2024-04-23T19:13:30.782396+0000","last_clean":"2024-04-23T19:13:30.782396+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:30.782396+0000","last_undegraded":"2024-04-23T19:13:30.782396+0000","last_fullsized":"2024-04-23T19:13:30.782396+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.782266+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:07.268727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.981168+0000","last_change":"2024-04-23T19:14:17.981194+0000","last_active":"2024-04-23T19:14:17.981168+0000","last_peered":"2024-04-23T19:14:17.981168+0000","last_clean":"2024-04-23T19:14:17.981168+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:14:17.981168+0000","last_undegraded":"2024-04-23T19:14:17.981168+0000","last_fullsized":"2024-04-23T19:14:17.981168+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:17.981076+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:17.981076+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.981076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:04:30.766806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.940021+0000","last_change":"2024-04-23T19:14:03.940021+0000","last_active":"2024-04-23T19:14:03.940021+0000","last_peered":"2024-04-23T19:14:03.940021+0000","last_clean":"2024-04-23T19:14:03.940021+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:14:03.940021+0000","last_undegraded":"2024-04-23T19:14:03.940021+0000","last_fullsized":"2024-04-23T19:14:03.940021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:03.939847+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:03.939847+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.939847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:54.259182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400231+0000","last_change":"2024-04-23T19:10:07.493532+0000","last_active":"2024-04-23T19:13:27.400231+0000","last_peered":"2024-04-23T19:13:27.400231+0000","last_clean":"2024-04-23T19:13:27.400231+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:13:27.400231+0000","last_undegraded":"2024-04-23T19:13:27.400231+0000","last_fullsized":"2024-04-23T19:13:27.400231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.557659+0000","last_change":"2024-04-23T19:14:14.557659+0000","last_active":"2024-04-23T19:14:14.557659+0000","last_peered":"2024-04-23T19:14:14.557659+0000","last_clean":"2024-04-23T19:14:14.557659+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:14:14.557659+0000","last_undegraded":"2024-04-23T19:14:14.557659+0000","last_fullsized":"2024-04-23T19:14:14.557659+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.557539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.557539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.557539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:50:51.381912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.862700+0000","last_change":"2024-04-23T19:13:38.862700+0000","last_active":"2024-04-23T19:13:38.862700+0000","last_peered":"2024-04-23T19:13:38.862700+0000","last_clean":"2024-04-23T19:13:38.862700+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:13:38.862700+0000","last_undegraded":"2024-04-23T19:13:38.862700+0000","last_fullsized":"2024-04-23T19:13:38.862700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.862584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:18.385955+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"32'16","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.889045+0000","last_change":"2024-04-23T19:13:51.889045+0000","last_active":"2024-04-23T19:13:51.889045+0000","last_peered":"2024-04-23T19:13:51.889045+0000","last_clean":"2024-04-23T19:13:51.889045+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:13:51.889045+0000","last_undegraded":"2024-04-23T19:13:51.889045+0000","last_fullsized":"2024-04-23T19:13:51.889045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.888897+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:47:41.106501+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'285","reported_seq":442,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.522645+0000","last_change":"2024-04-23T19:13:47.522645+0000","last_active":"2024-04-23T19:13:47.522645+0000","last_peered":"2024-04-23T19:13:47.522645+0000","last_clean":"2024-04-23T19:13:47.522645+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:47.522645+0000","last_undegraded":"2024-04-23T19:13:47.522645+0000","last_fullsized":"2024-04-23T19:13:47.522645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.522518+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:37.798951+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.764905+0000","last_change":"2024-04-23T19:14:22.764934+0000","last_active":"2024-04-23T19:14:22.764905+0000","last_peered":"2024-04-23T19:14:22.764905+0000","last_clean":"2024-04-23T19:14:22.764905+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:14:22.764905+0000","last_undegraded":"2024-04-23T19:14:22.764905+0000","last_fullsized":"2024-04-23T19:14:22.764905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'3","last_scrub_stamp":"2024-04-23T19:14:22.764803+0000","last_deep_scrub":"30'3","last_deep_scrub_stamp":"2024-04-23T19:14:22.764803+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.764803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:24.439320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.083351+0000","last_change":"2024-04-23T19:13:50.083351+0000","last_active":"2024-04-23T19:13:50.083351+0000","last_peered":"2024-04-23T19:13:50.083351+0000","last_clean":"2024-04-23T19:13:50.083351+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:50.083351+0000","last_undegraded":"2024-04-23T19:13:50.083351+0000","last_fullsized":"2024-04-23T19:13:50.083351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.083230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:01.089369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.940410+0000","last_change":"2024-04-23T19:14:11.940410+0000","last_active":"2024-04-23T19:14:11.940410+0000","last_peered":"2024-04-23T19:14:11.940410+0000","last_clean":"2024-04-23T19:14:11.940410+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:14:11.940410+0000","last_undegraded":"2024-04-23T19:14:11.940410+0000","last_fullsized":"2024-04-23T19:14:11.940410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:11.940272+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:11.940272+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.940272+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:59.806805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400107+0000","last_change":"2024-04-23T19:09:59.577674+0000","last_active":"2024-04-23T19:13:27.400107+0000","last_peered":"2024-04-23T19:13:27.400107+0000","last_clean":"2024-04-23T19:13:27.400107+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:13:27.400107+0000","last_undegraded":"2024-04-23T19:13:27.400107+0000","last_fullsized":"2024-04-23T19:13:27.400107+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:09:58.435679+0000","last_clean_scrub_stamp":"2024-04-23T19:09:58.435679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.119771+0000","last_change":"2024-04-23T19:13:29.119771+0000","last_active":"2024-04-23T19:13:29.119771+0000","last_peered":"2024-04-23T19:13:29.119771+0000","last_clean":"2024-04-23T19:13:29.119771+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:13:29.119771+0000","last_undegraded":"2024-04-23T19:13:29.119771+0000","last_fullsized":"2024-04-23T19:13:29.119771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.119655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:06.217082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'98","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.195253+0000","last_change":"2024-04-23T19:13:35.195253+0000","last_active":"2024-04-23T19:13:35.195253+0000","last_peered":"2024-04-23T19:13:35.195253+0000","last_clean":"2024-04-23T19:13:35.195253+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:13:35.195253+0000","last_undegraded":"2024-04-23T19:13:35.195253+0000","last_fullsized":"2024-04-23T19:13:35.195253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'98","last_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_deep_scrub":"32'98","last_deep_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.195120+0000","objects_scrubbed":8,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:11.071311+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4316,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":98,"num_read_kb":92,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.664532+0000","last_change":"2024-04-23T19:13:30.664532+0000","last_active":"2024-04-23T19:13:30.664532+0000","last_peered":"2024-04-23T19:13:30.664532+0000","last_clean":"2024-04-23T19:13:30.664532+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:13:30.664532+0000","last_undegraded":"2024-04-23T19:13:30.664532+0000","last_fullsized":"2024-04-23T19:13:30.664532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.664414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:40.569067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.161812+0000","last_change":"2024-04-23T19:13:45.161859+0000","last_active":"2024-04-23T19:13:45.161812+0000","last_peered":"2024-04-23T19:13:45.161812+0000","last_clean":"2024-04-23T19:13:45.161812+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:13:45.161812+0000","last_undegraded":"2024-04-23T19:13:45.161812+0000","last_fullsized":"2024-04-23T19:13:45.161812+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.161710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:30.545797+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.187419+0000","last_change":"2024-04-23T19:13:37.187444+0000","last_active":"2024-04-23T19:13:37.187419+0000","last_peered":"2024-04-23T19:13:37.187419+0000","last_clean":"2024-04-23T19:13:37.187419+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:13:37.187419+0000","last_undegraded":"2024-04-23T19:13:37.187419+0000","last_fullsized":"2024-04-23T19:13:37.187419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.187313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.274468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.039233+0000","last_change":"2024-04-23T19:13:59.039257+0000","last_active":"2024-04-23T19:13:59.039233+0000","last_peered":"2024-04-23T19:13:59.039233+0000","last_clean":"2024-04-23T19:13:59.039233+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:13:59.039233+0000","last_undegraded":"2024-04-23T19:13:59.039233+0000","last_fullsized":"2024-04-23T19:13:59.039233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.039137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:29.931705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720068+0000","last_change":"2024-04-23T19:10:07.500107+0000","last_active":"2024-04-23T19:12:26.720068+0000","last_peered":"2024-04-23T19:12:26.720068+0000","last_clean":"2024-04-23T19:12:26.720068+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:12:26.720068+0000","last_undegraded":"2024-04-23T19:12:26.720068+0000","last_fullsized":"2024-04-23T19:12:26.720068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:26.688297+0000","last_change":"2024-04-23T19:14:26.688322+0000","last_active":"2024-04-23T19:14:26.688297+0000","last_peered":"2024-04-23T19:14:26.688297+0000","last_clean":"2024-04-23T19:14:26.688297+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:14:26.688297+0000","last_undegraded":"2024-04-23T19:14:26.688297+0000","last_fullsized":"2024-04-23T19:14:26.688297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:26.688210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:26.688210+0000","last_clean_scrub_stamp":"2024-04-23T19:14:26.688210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:32.612364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.033330+0000","last_change":"2024-04-23T19:13:32.033364+0000","last_active":"2024-04-23T19:13:32.033330+0000","last_peered":"2024-04-23T19:13:32.033330+0000","last_clean":"2024-04-23T19:13:32.033330+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:13:32.033330+0000","last_undegraded":"2024-04-23T19:13:32.033330+0000","last_fullsized":"2024-04-23T19:13:32.033330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.033246+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:16.932749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.897278+0000","last_change":"2024-04-23T19:13:57.897299+0000","last_active":"2024-04-23T19:13:57.897278+0000","last_peered":"2024-04-23T19:13:57.897278+0000","last_clean":"2024-04-23T19:13:57.897278+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:13:57.897278+0000","last_undegraded":"2024-04-23T19:13:57.897278+0000","last_fullsized":"2024-04-23T19:13:57.897278+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.897188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:37.553642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.482950+0000","last_change":"2024-04-23T19:14:23.482976+0000","last_active":"2024-04-23T19:14:23.482950+0000","last_peered":"2024-04-23T19:14:23.482950+0000","last_clean":"2024-04-23T19:14:23.482950+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:14:23.482950+0000","last_undegraded":"2024-04-23T19:14:23.482950+0000","last_fullsized":"2024-04-23T19:14:23.482950+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:14:23.482860+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:14:23.482860+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.482860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:32.816753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:31.720290+0000","last_change":"2024-04-23T19:10:47.732302+0000","last_active":"2024-04-23T19:12:31.720290+0000","last_peered":"2024-04-23T19:12:31.720290+0000","last_clean":"2024-04-23T19:12:31.720290+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:12:31.720290+0000","last_undegraded":"2024-04-23T19:12:31.720290+0000","last_fullsized":"2024-04-23T19:12:31.720290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:08.463033+0000","last_clean_scrub_stamp":"2024-04-23T19:10:08.463033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.058576+0000","last_change":"2024-04-23T19:14:02.058602+0000","last_active":"2024-04-23T19:14:02.058576+0000","last_peered":"2024-04-23T19:14:02.058576+0000","last_clean":"2024-04-23T19:14:02.058576+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:14:02.058576+0000","last_undegraded":"2024-04-23T19:14:02.058576+0000","last_fullsized":"2024-04-23T19:14:02.058576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.058488+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:35.270454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.891432+0000","last_change":"2024-04-23T19:13:35.891471+0000","last_active":"2024-04-23T19:13:35.891432+0000","last_peered":"2024-04-23T19:13:35.891432+0000","last_clean":"2024-04-23T19:13:35.891432+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:13:35.891432+0000","last_undegraded":"2024-04-23T19:13:35.891432+0000","last_fullsized":"2024-04-23T19:13:35.891432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.891324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:22.760900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.738794+0000","last_change":"2024-04-23T19:13:37.738834+0000","last_active":"2024-04-23T19:13:37.738794+0000","last_peered":"2024-04-23T19:13:37.738794+0000","last_clean":"2024-04-23T19:13:37.738794+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:37.738794+0000","last_undegraded":"2024-04-23T19:13:37.738794+0000","last_fullsized":"2024-04-23T19:13:37.738794+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.738703+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:59.027468+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.218628+0000","last_change":"2024-04-23T19:13:33.218654+0000","last_active":"2024-04-23T19:13:33.218628+0000","last_peered":"2024-04-23T19:13:33.218628+0000","last_clean":"2024-04-23T19:13:33.218628+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:13:33.218628+0000","last_undegraded":"2024-04-23T19:13:33.218628+0000","last_fullsized":"2024-04-23T19:13:33.218628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.218534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:31.120188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:50.715910+0000","last_change":"2024-04-23T19:10:17.520788+0000","last_active":"2024-04-23T19:12:50.715910+0000","last_peered":"2024-04-23T19:12:50.715910+0000","last_clean":"2024-04-23T19:12:50.715910+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:12:50.715910+0000","last_undegraded":"2024-04-23T19:12:50.715910+0000","last_fullsized":"2024-04-23T19:12:50.715910+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.581715+0000","last_change":"2024-04-23T19:13:53.581741+0000","last_active":"2024-04-23T19:13:53.581715+0000","last_peered":"2024-04-23T19:13:53.581715+0000","last_clean":"2024-04-23T19:13:53.581715+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:53.581715+0000","last_undegraded":"2024-04-23T19:13:53.581715+0000","last_fullsized":"2024-04-23T19:13:53.581715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.581624+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:26.793590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.067768+0000","last_change":"2024-04-23T19:13:56.067792+0000","last_active":"2024-04-23T19:13:56.067768+0000","last_peered":"2024-04-23T19:13:56.067768+0000","last_clean":"2024-04-23T19:13:56.067768+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:13:56.067768+0000","last_undegraded":"2024-04-23T19:13:56.067768+0000","last_fullsized":"2024-04-23T19:13:56.067768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.067677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:46.344110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.223615+0000","last_change":"2024-04-23T19:13:43.223641+0000","last_active":"2024-04-23T19:13:43.223615+0000","last_peered":"2024-04-23T19:13:43.223615+0000","last_clean":"2024-04-23T19:13:43.223615+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:13:43.223615+0000","last_undegraded":"2024-04-23T19:13:43.223615+0000","last_fullsized":"2024-04-23T19:13:43.223615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.223515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:59.385060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'47","reported_seq":119,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.709428+0000","last_change":"2024-04-23T19:13:40.709462+0000","last_active":"2024-04-23T19:13:40.709428+0000","last_peered":"2024-04-23T19:13:40.709428+0000","last_clean":"2024-04-23T19:13:40.709428+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:40.709428+0000","last_undegraded":"2024-04-23T19:13:40.709428+0000","last_fullsized":"2024-04-23T19:13:40.709428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'47","last_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_deep_scrub":"32'47","last_deep_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.709338+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:40.827875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.737111+0000","last_change":"2024-04-23T19:13:27.737136+0000","last_active":"2024-04-23T19:13:27.737111+0000","last_peered":"2024-04-23T19:13:27.737111+0000","last_clean":"2024-04-23T19:13:27.737111+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:13:27.737111+0000","last_undegraded":"2024-04-23T19:13:27.737111+0000","last_fullsized":"2024-04-23T19:13:27.737111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.737016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:03.613207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:46.723450+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:12:46.723450+0000","last_peered":"2024-04-23T19:12:46.723450+0000","last_clean":"2024-04-23T19:12:46.723450+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:12:46.723450+0000","last_undegraded":"2024-04-23T19:12:46.723450+0000","last_fullsized":"2024-04-23T19:12:46.723450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'10","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.904017+0000","last_change":"2024-04-23T19:14:02.904055+0000","last_active":"2024-04-23T19:14:02.904017+0000","last_peered":"2024-04-23T19:14:02.904017+0000","last_clean":"2024-04-23T19:14:02.904017+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:14:02.904017+0000","last_undegraded":"2024-04-23T19:14:02.904017+0000","last_fullsized":"2024-04-23T19:14:02.904017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:02.903914+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:02.903914+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.903914+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:12.237807+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.755932+0000","last_change":"2024-04-23T19:14:18.755958+0000","last_active":"2024-04-23T19:14:18.755932+0000","last_peered":"2024-04-23T19:14:18.755932+0000","last_clean":"2024-04-23T19:14:18.755932+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:14:18.755932+0000","last_undegraded":"2024-04-23T19:14:18.755932+0000","last_fullsized":"2024-04-23T19:14:18.755932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.755843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.755843+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.755843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:31.351437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.092812+0000","last_change":"2024-04-23T19:13:30.092835+0000","last_active":"2024-04-23T19:13:30.092812+0000","last_peered":"2024-04-23T19:13:30.092812+0000","last_clean":"2024-04-23T19:13:30.092812+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:30.092812+0000","last_undegraded":"2024-04-23T19:13:30.092812+0000","last_fullsized":"2024-04-23T19:13:30.092812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.092707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:02.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'38","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.254489+0000","last_change":"2024-04-23T19:13:42.254523+0000","last_active":"2024-04-23T19:13:42.254489+0000","last_peered":"2024-04-23T19:13:42.254489+0000","last_clean":"2024-04-23T19:13:42.254489+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:13:42.254489+0000","last_undegraded":"2024-04-23T19:13:42.254489+0000","last_fullsized":"2024-04-23T19:13:42.254489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.254400+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:59.255554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":53,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.124596+0000","last_change":"2024-04-23T19:13:27.124622+0000","last_active":"2024-04-23T19:13:27.124596+0000","last_peered":"2024-04-23T19:13:27.124596+0000","last_clean":"2024-04-23T19:13:27.124596+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:13:27.124596+0000","last_undegraded":"2024-04-23T19:13:27.124596+0000","last_fullsized":"2024-04-23T19:13:27.124596+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.124488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:56.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.130082+0000","last_change":"2024-04-23T19:14:18.130109+0000","last_active":"2024-04-23T19:14:18.130082+0000","last_peered":"2024-04-23T19:14:18.130082+0000","last_clean":"2024-04-23T19:14:18.130082+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:14:18.130082+0000","last_undegraded":"2024-04-23T19:14:18.130082+0000","last_fullsized":"2024-04-23T19:14:18.130082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.129993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.129993+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.129993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:23.793668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.996158+0000","last_change":"2024-04-23T19:14:19.996211+0000","last_active":"2024-04-23T19:14:19.996158+0000","last_peered":"2024-04-23T19:14:19.996158+0000","last_clean":"2024-04-23T19:14:19.996158+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:14:19.996158+0000","last_undegraded":"2024-04-23T19:14:19.996158+0000","last_fullsized":"2024-04-23T19:14:19.996158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.996051+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.996051+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.996051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:19.442125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.511211+0000","last_change":"2024-04-23T19:13:48.511211+0000","last_active":"2024-04-23T19:13:48.511211+0000","last_peered":"2024-04-23T19:13:48.511211+0000","last_clean":"2024-04-23T19:13:48.511211+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:13:48.511211+0000","last_undegraded":"2024-04-23T19:13:48.511211+0000","last_fullsized":"2024-04-23T19:13:48.511211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.511103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:25.608530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.697721+0000","last_change":"2024-04-23T19:13:38.697755+0000","last_active":"2024-04-23T19:13:38.697721+0000","last_peered":"2024-04-23T19:13:38.697721+0000","last_clean":"2024-04-23T19:13:38.697721+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:13:38.697721+0000","last_undegraded":"2024-04-23T19:13:38.697721+0000","last_fullsized":"2024-04-23T19:13:38.697721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.697637+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:22.296607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.201793+0000","last_change":"2024-04-23T19:13:32.201821+0000","last_active":"2024-04-23T19:13:32.201793+0000","last_peered":"2024-04-23T19:13:32.201793+0000","last_clean":"2024-04-23T19:13:32.201793+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:13:32.201793+0000","last_undegraded":"2024-04-23T19:13:32.201793+0000","last_fullsized":"2024-04-23T19:13:32.201793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.201694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:43.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:40.714595+0000","last_change":"2024-04-23T19:10:17.521299+0000","last_active":"2024-04-23T19:12:40.714595+0000","last_peered":"2024-04-23T19:12:40.714595+0000","last_clean":"2024-04-23T19:12:40.714595+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:12:40.714595+0000","last_undegraded":"2024-04-23T19:12:40.714595+0000","last_fullsized":"2024-04-23T19:12:40.714595+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.612354+0000","last_change":"2024-04-23T19:14:05.612354+0000","last_active":"2024-04-23T19:14:05.612354+0000","last_peered":"2024-04-23T19:14:05.612354+0000","last_clean":"2024-04-23T19:14:05.612354+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:14:05.612354+0000","last_undegraded":"2024-04-23T19:14:05.612354+0000","last_fullsized":"2024-04-23T19:14:05.612354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.612239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:04.922704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.782002+0000","last_change":"2024-04-23T19:14:17.782031+0000","last_active":"2024-04-23T19:14:17.782002+0000","last_peered":"2024-04-23T19:14:17.782002+0000","last_clean":"2024-04-23T19:14:17.782002+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:14:17.782002+0000","last_undegraded":"2024-04-23T19:14:17.782002+0000","last_fullsized":"2024-04-23T19:14:17.782002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:17.781913+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:17.781913+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.781913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:45.876367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.142214+0000","last_change":"2024-04-23T19:13:48.142214+0000","last_active":"2024-04-23T19:13:48.142214+0000","last_peered":"2024-04-23T19:13:48.142214+0000","last_clean":"2024-04-23T19:13:48.142214+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:13:48.142214+0000","last_undegraded":"2024-04-23T19:13:48.142214+0000","last_fullsized":"2024-04-23T19:13:48.142214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.142093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:17.068685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.927235+0000","last_change":"2024-04-23T19:13:54.927318+0000","last_active":"2024-04-23T19:13:54.927235+0000","last_peered":"2024-04-23T19:13:54.927235+0000","last_clean":"2024-04-23T19:13:54.927235+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:13:54.927235+0000","last_undegraded":"2024-04-23T19:13:54.927235+0000","last_fullsized":"2024-04-23T19:13:54.927235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.927131+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:23.021585+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.866434+0000","last_change":"2024-04-23T19:13:42.866499+0000","last_active":"2024-04-23T19:13:42.866434+0000","last_peered":"2024-04-23T19:13:42.866434+0000","last_clean":"2024-04-23T19:13:42.866434+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:13:42.866434+0000","last_undegraded":"2024-04-23T19:13:42.866434+0000","last_fullsized":"2024-04-23T19:13:42.866434+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.866306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.872444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:36.721579+0000","last_change":"2024-04-23T19:10:17.517391+0000","last_active":"2024-04-23T19:12:36.721579+0000","last_peered":"2024-04-23T19:12:36.721579+0000","last_clean":"2024-04-23T19:12:36.721579+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:12:36.721579+0000","last_undegraded":"2024-04-23T19:12:36.721579+0000","last_fullsized":"2024-04-23T19:12:36.721579+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.193141+0000","last_change":"2024-04-23T19:14:21.193203+0000","last_active":"2024-04-23T19:14:21.193141+0000","last_peered":"2024-04-23T19:14:21.193141+0000","last_clean":"2024-04-23T19:14:21.193141+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:14:21.193141+0000","last_undegraded":"2024-04-23T19:14:21.193141+0000","last_fullsized":"2024-04-23T19:14:21.193141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:21.193026+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:21.193026+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.193026+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:45.665300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.844740+0000","last_change":"2024-04-23T19:14:05.844813+0000","last_active":"2024-04-23T19:14:05.844740+0000","last_peered":"2024-04-23T19:14:05.844740+0000","last_clean":"2024-04-23T19:14:05.844740+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:14:05.844740+0000","last_undegraded":"2024-04-23T19:14:05.844740+0000","last_fullsized":"2024-04-23T19:14:05.844740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:05.844638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:05.844638+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.844638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:52:20.554775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.129709+0000","last_change":"2024-04-23T19:13:47.129736+0000","last_active":"2024-04-23T19:13:47.129709+0000","last_peered":"2024-04-23T19:13:47.129709+0000","last_clean":"2024-04-23T19:13:47.129709+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:13:47.129709+0000","last_undegraded":"2024-04-23T19:13:47.129709+0000","last_fullsized":"2024-04-23T19:13:47.129709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.129623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:16:50.873891+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'39","reported_seq":96,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.723412+0000","last_change":"2024-04-23T19:13:36.723412+0000","last_active":"2024-04-23T19:13:36.723412+0000","last_peered":"2024-04-23T19:13:36.723412+0000","last_clean":"2024-04-23T19:13:36.723412+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:13:36.723412+0000","last_undegraded":"2024-04-23T19:13:36.723412+0000","last_fullsized":"2024-04-23T19:13:36.723412+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.723239+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:22.107325+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.719943+0000","last_change":"2024-04-23T19:13:39.719943+0000","last_active":"2024-04-23T19:13:39.719943+0000","last_peered":"2024-04-23T19:13:39.719943+0000","last_clean":"2024-04-23T19:13:39.719943+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:13:39.719943+0000","last_undegraded":"2024-04-23T19:13:39.719943+0000","last_fullsized":"2024-04-23T19:13:39.719943+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:33.961424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.462496+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.462496+0000","last_peered":"2024-04-23T19:13:16.462496+0000","last_clean":"2024-04-23T19:13:16.462496+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.462496+0000","last_undegraded":"2024-04-23T19:13:16.462496+0000","last_fullsized":"2024-04-23T19:13:16.462496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.526283+0000","last_change":"2024-04-23T19:13:57.526310+0000","last_active":"2024-04-23T19:13:57.526283+0000","last_peered":"2024-04-23T19:13:57.526283+0000","last_clean":"2024-04-23T19:13:57.526283+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:57.526283+0000","last_undegraded":"2024-04-23T19:13:57.526283+0000","last_fullsized":"2024-04-23T19:13:57.526283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.526187+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:13.862482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.930668+0000","last_change":"2024-04-23T19:14:21.930715+0000","last_active":"2024-04-23T19:14:21.930668+0000","last_peered":"2024-04-23T19:14:21.930668+0000","last_clean":"2024-04-23T19:14:21.930668+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:14:21.930668+0000","last_undegraded":"2024-04-23T19:14:21.930668+0000","last_fullsized":"2024-04-23T19:14:21.930668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:21.930521+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:21.930521+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.930521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:51.962515+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'5","reported_seq":238,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.794993+0000","last_change":"2024-04-23T19:13:44.795028+0000","last_active":"2024-04-23T19:13:44.794993+0000","last_peered":"2024-04-23T19:13:44.794993+0000","last_clean":"2024-04-23T19:13:44.794993+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:13:44.794993+0000","last_undegraded":"2024-04-23T19:13:44.794993+0000","last_fullsized":"2024-04-23T19:13:44.794993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.794905+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:45.493403+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"32'15","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.880831+0000","last_change":"2024-04-23T19:13:56.880831+0000","last_active":"2024-04-23T19:13:56.880831+0000","last_peered":"2024-04-23T19:13:56.880831+0000","last_clean":"2024-04-23T19:13:56.880831+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:13:56.880831+0000","last_undegraded":"2024-04-23T19:13:56.880831+0000","last_fullsized":"2024-04-23T19:13:56.880831+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.880679+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:26.722280+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'285","reported_seq":430,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.137953+0000","last_change":"2024-04-23T19:13:29.137953+0000","last_active":"2024-04-23T19:13:29.137953+0000","last_peered":"2024-04-23T19:13:29.137953+0000","last_clean":"2024-04-23T19:13:29.137953+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:29.137953+0000","last_undegraded":"2024-04-23T19:13:29.137953+0000","last_fullsized":"2024-04-23T19:13:29.137953+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.137819+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:17.728927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.985379+0000","last_change":"2024-04-23T19:14:13.985410+0000","last_active":"2024-04-23T19:14:13.985379+0000","last_peered":"2024-04-23T19:14:13.985379+0000","last_clean":"2024-04-23T19:14:13.985379+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:14:13.985379+0000","last_undegraded":"2024-04-23T19:14:13.985379+0000","last_fullsized":"2024-04-23T19:14:13.985379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:13.985289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:13.985289+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.985289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:36.480554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.032334+0000","last_change":"2024-04-23T19:13:53.032361+0000","last_active":"2024-04-23T19:13:53.032334+0000","last_peered":"2024-04-23T19:13:53.032334+0000","last_clean":"2024-04-23T19:13:53.032334+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:13:53.032334+0000","last_undegraded":"2024-04-23T19:13:53.032334+0000","last_fullsized":"2024-04-23T19:13:53.032334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.032245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:08.378799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.780980+0000","last_change":"2024-04-23T19:14:19.781007+0000","last_active":"2024-04-23T19:14:19.780980+0000","last_peered":"2024-04-23T19:14:19.780980+0000","last_clean":"2024-04-23T19:14:19.780980+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:14:19.780980+0000","last_undegraded":"2024-04-23T19:14:19.780980+0000","last_fullsized":"2024-04-23T19:14:19.780980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:19.780892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:19.780892+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.780892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:49.622493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.890930+0000","last_change":"2024-04-23T19:13:41.890978+0000","last_active":"2024-04-23T19:13:41.890930+0000","last_peered":"2024-04-23T19:13:41.890930+0000","last_clean":"2024-04-23T19:13:41.890930+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:13:41.890930+0000","last_undegraded":"2024-04-23T19:13:41.890930+0000","last_fullsized":"2024-04-23T19:13:41.890930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.890830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:42.009368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.806873+0000","last_change":"2024-04-23T19:13:31.806949+0000","last_active":"2024-04-23T19:13:31.806873+0000","last_peered":"2024-04-23T19:13:31.806873+0000","last_clean":"2024-04-23T19:13:31.806873+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:13:31.806873+0000","last_undegraded":"2024-04-23T19:13:31.806873+0000","last_fullsized":"2024-04-23T19:13:31.806873+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.806771+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:55.900270+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.655391+0000","last_change":"2024-04-23T19:13:31.655412+0000","last_active":"2024-04-23T19:13:31.655391+0000","last_peered":"2024-04-23T19:13:31.655391+0000","last_clean":"2024-04-23T19:13:31.655391+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:13:31.655391+0000","last_undegraded":"2024-04-23T19:13:31.655391+0000","last_fullsized":"2024-04-23T19:13:31.655391+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.655302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:46.303379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.793862+0000","last_change":"2024-04-23T19:14:23.793885+0000","last_active":"2024-04-23T19:14:23.793862+0000","last_peered":"2024-04-23T19:14:23.793862+0000","last_clean":"2024-04-23T19:14:23.793862+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:14:23.793862+0000","last_undegraded":"2024-04-23T19:14:23.793862+0000","last_fullsized":"2024-04-23T19:14:23.793862+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:23.793752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:23.793752+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.793752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:39.597437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.023793+0000","last_change":"2024-04-23T19:13:55.023820+0000","last_active":"2024-04-23T19:13:55.023793+0000","last_peered":"2024-04-23T19:13:55.023793+0000","last_clean":"2024-04-23T19:13:55.023793+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:55.023793+0000","last_undegraded":"2024-04-23T19:13:55.023793+0000","last_fullsized":"2024-04-23T19:13:55.023793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.023704+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:02.984229+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.970911+0000","last_change":"2024-04-23T19:14:20.970941+0000","last_active":"2024-04-23T19:14:20.970911+0000","last_peered":"2024-04-23T19:14:20.970911+0000","last_clean":"2024-04-23T19:14:20.970911+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:14:20.970911+0000","last_undegraded":"2024-04-23T19:14:20.970911+0000","last_fullsized":"2024-04-23T19:14:20.970911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:20.970802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:20.970802+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.970802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:24:34.128767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.149582+0000","last_change":"2024-04-23T19:13:46.149624+0000","last_active":"2024-04-23T19:13:46.149582+0000","last_peered":"2024-04-23T19:13:46.149582+0000","last_clean":"2024-04-23T19:13:46.149582+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:13:46.149582+0000","last_undegraded":"2024-04-23T19:13:46.149582+0000","last_fullsized":"2024-04-23T19:13:46.149582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.149461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:44.705262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.718061+0000","last_change":"2024-04-23T19:13:33.718094+0000","last_active":"2024-04-23T19:13:33.718061+0000","last_peered":"2024-04-23T19:13:33.718061+0000","last_clean":"2024-04-23T19:13:33.718061+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:13:33.718061+0000","last_undegraded":"2024-04-23T19:13:33.718061+0000","last_fullsized":"2024-04-23T19:13:33.718061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.717974+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:50.719128+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.950064+0000","last_change":"2024-04-23T19:14:01.950089+0000","last_active":"2024-04-23T19:14:01.950064+0000","last_peered":"2024-04-23T19:14:01.950064+0000","last_clean":"2024-04-23T19:14:01.950064+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:14:01.950064+0000","last_undegraded":"2024-04-23T19:14:01.950064+0000","last_fullsized":"2024-04-23T19:14:01.950064+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:01.949971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:01.949971+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.949971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:05.620169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722640+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:41.722640+0000","last_peered":"2024-04-23T19:12:41.722640+0000","last_clean":"2024-04-23T19:12:41.722640+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:41.722640+0000","last_undegraded":"2024-04-23T19:12:41.722640+0000","last_fullsized":"2024-04-23T19:12:41.722640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'12","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.000633+0000","last_change":"2024-04-23T19:14:09.000667+0000","last_active":"2024-04-23T19:14:09.000633+0000","last_peered":"2024-04-23T19:14:09.000633+0000","last_clean":"2024-04-23T19:14:09.000633+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:14:09.000633+0000","last_undegraded":"2024-04-23T19:14:09.000633+0000","last_fullsized":"2024-04-23T19:14:09.000633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:09.000539+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:09.000539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.000539+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:32.366969+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.545266+0000","last_change":"2024-04-23T19:14:18.545295+0000","last_active":"2024-04-23T19:14:18.545266+0000","last_peered":"2024-04-23T19:14:18.545266+0000","last_clean":"2024-04-23T19:14:18.545266+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:14:18.545266+0000","last_undegraded":"2024-04-23T19:14:18.545266+0000","last_fullsized":"2024-04-23T19:14:18.545266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.545177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.545177+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.545177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:58.201631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.929784+0000","last_change":"2024-04-23T19:13:36.929808+0000","last_active":"2024-04-23T19:13:36.929784+0000","last_peered":"2024-04-23T19:13:36.929784+0000","last_clean":"2024-04-23T19:13:36.929784+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:13:36.929784+0000","last_undegraded":"2024-04-23T19:13:36.929784+0000","last_fullsized":"2024-04-23T19:13:36.929784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.929690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:45.019264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'30","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.925641+0000","last_change":"2024-04-23T19:13:52.925676+0000","last_active":"2024-04-23T19:13:52.925641+0000","last_peered":"2024-04-23T19:13:52.925641+0000","last_clean":"2024-04-23T19:13:52.925641+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:13:52.925641+0000","last_undegraded":"2024-04-23T19:13:52.925641+0000","last_fullsized":"2024-04-23T19:13:52.925641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'30","last_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_deep_scrub":"32'30","last_deep_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.925542+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.397817+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.909679+0000","last_change":"2024-04-23T19:13:49.909700+0000","last_active":"2024-04-23T19:13:49.909679+0000","last_peered":"2024-04-23T19:13:49.909679+0000","last_clean":"2024-04-23T19:13:49.909679+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:13:49.909679+0000","last_undegraded":"2024-04-23T19:13:49.909679+0000","last_fullsized":"2024-04-23T19:13:49.909679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.909590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:01:35.495993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722581+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:41.722581+0000","last_peered":"2024-04-23T19:12:41.722581+0000","last_clean":"2024-04-23T19:12:41.722581+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:41.722581+0000","last_undegraded":"2024-04-23T19:12:41.722581+0000","last_fullsized":"2024-04-23T19:12:41.722581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.048809+0000","last_change":"2024-04-23T19:13:52.048855+0000","last_active":"2024-04-23T19:13:52.048809+0000","last_peered":"2024-04-23T19:13:52.048809+0000","last_clean":"2024-04-23T19:13:52.048809+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:52.048809+0000","last_undegraded":"2024-04-23T19:13:52.048809+0000","last_fullsized":"2024-04-23T19:13:52.048809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.048718+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:52.747216+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.880060+0000","last_change":"2024-04-23T19:14:09.880060+0000","last_active":"2024-04-23T19:14:09.880060+0000","last_peered":"2024-04-23T19:14:09.880060+0000","last_clean":"2024-04-23T19:14:09.880060+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:14:09.880060+0000","last_undegraded":"2024-04-23T19:14:09.880060+0000","last_fullsized":"2024-04-23T19:14:09.880060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:09.879942+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:09.879942+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.879942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.402393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'3","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.954199+0000","last_change":"2024-04-23T19:13:39.954265+0000","last_active":"2024-04-23T19:13:39.954199+0000","last_peered":"2024-04-23T19:13:39.954199+0000","last_clean":"2024-04-23T19:13:39.954199+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:39.954199+0000","last_undegraded":"2024-04-23T19:13:39.954199+0000","last_fullsized":"2024-04-23T19:13:39.954199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.954105+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:23.553074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'40","reported_seq":97,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.877187+0000","last_change":"2024-04-23T19:13:55.877248+0000","last_active":"2024-04-23T19:13:55.877187+0000","last_peered":"2024-04-23T19:13:55.877187+0000","last_clean":"2024-04-23T19:13:55.877187+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:55.877187+0000","last_undegraded":"2024-04-23T19:13:55.877187+0000","last_fullsized":"2024-04-23T19:13:55.877187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'40","last_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_deep_scrub":"32'40","last_deep_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.877078+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:08.472673+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.549277+0000","last_change":"2024-04-23T19:13:27.549295+0000","last_active":"2024-04-23T19:13:27.549277+0000","last_peered":"2024-04-23T19:13:27.549277+0000","last_clean":"2024-04-23T19:13:27.549277+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:13:27.549277+0000","last_undegraded":"2024-04-23T19:13:27.549277+0000","last_fullsized":"2024-04-23T19:13:27.549277+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.549205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:33.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.486600+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.486600+0000","last_peered":"2024-04-23T19:13:16.486600+0000","last_clean":"2024-04-23T19:13:16.486600+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.486600+0000","last_undegraded":"2024-04-23T19:13:16.486600+0000","last_fullsized":"2024-04-23T19:13:16.486600+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":77,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.719319+0000","last_change":"2024-04-23T19:13:49.719401+0000","last_active":"2024-04-23T19:13:49.719319+0000","last_peered":"2024-04-23T19:13:49.719319+0000","last_clean":"2024-04-23T19:13:49.719319+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:49.719319+0000","last_undegraded":"2024-04-23T19:13:49.719319+0000","last_fullsized":"2024-04-23T19:13:49.719319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.719206+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:11:27.945878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.054867+0000","last_change":"2024-04-23T19:14:06.054867+0000","last_active":"2024-04-23T19:14:06.054867+0000","last_peered":"2024-04-23T19:14:06.054867+0000","last_clean":"2024-04-23T19:14:06.054867+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:14:06.054867+0000","last_undegraded":"2024-04-23T19:14:06.054867+0000","last_fullsized":"2024-04-23T19:14:06.054867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:06.054734+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:06.054734+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.054734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:22.391030+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.541764+0000","last_change":"2024-04-23T19:14:03.541792+0000","last_active":"2024-04-23T19:14:03.541764+0000","last_peered":"2024-04-23T19:14:03.541764+0000","last_clean":"2024-04-23T19:14:03.541764+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:14:03.541764+0000","last_undegraded":"2024-04-23T19:14:03.541764+0000","last_fullsized":"2024-04-23T19:14:03.541764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.541669+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:29.952719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.533755+0000","last_change":"2024-04-23T19:14:20.533755+0000","last_active":"2024-04-23T19:14:20.533755+0000","last_peered":"2024-04-23T19:14:20.533755+0000","last_clean":"2024-04-23T19:14:20.533755+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:14:20.533755+0000","last_undegraded":"2024-04-23T19:14:20.533755+0000","last_fullsized":"2024-04-23T19:14:20.533755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:20.533629+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:20.533629+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.533629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:58.132753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.033561+0000","last_change":"2024-04-23T19:14:08.033589+0000","last_active":"2024-04-23T19:14:08.033561+0000","last_peered":"2024-04-23T19:14:08.033561+0000","last_clean":"2024-04-23T19:14:08.033561+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:14:08.033561+0000","last_undegraded":"2024-04-23T19:14:08.033561+0000","last_fullsized":"2024-04-23T19:14:08.033561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:08.033479+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:08.033479+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.033479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:30:11.119403+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400396+0000","last_change":"2024-04-23T19:10:07.506697+0000","last_active":"2024-04-23T19:13:27.400396+0000","last_peered":"2024-04-23T19:13:27.400396+0000","last_clean":"2024-04-23T19:13:27.400396+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:13:27.400396+0000","last_undegraded":"2024-04-23T19:13:27.400396+0000","last_fullsized":"2024-04-23T19:13:27.400396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.509098+0000","last_change":"2024-04-23T19:14:01.509140+0000","last_active":"2024-04-23T19:14:01.509098+0000","last_peered":"2024-04-23T19:14:01.509098+0000","last_clean":"2024-04-23T19:14:01.509098+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:14:01.509098+0000","last_undegraded":"2024-04-23T19:14:01.509098+0000","last_fullsized":"2024-04-23T19:14:01.509098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.508998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:10.244783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.590245+0000","last_change":"2024-04-23T19:14:08.590245+0000","last_active":"2024-04-23T19:14:08.590245+0000","last_peered":"2024-04-23T19:14:08.590245+0000","last_clean":"2024-04-23T19:14:08.590245+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:14:08.590245+0000","last_undegraded":"2024-04-23T19:14:08.590245+0000","last_fullsized":"2024-04-23T19:14:08.590245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:08.590099+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:08.590099+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.590099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:56.456633+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.554231+0000","last_change":"2024-04-23T19:13:54.554231+0000","last_active":"2024-04-23T19:13:54.554231+0000","last_peered":"2024-04-23T19:13:54.554231+0000","last_clean":"2024-04-23T19:13:54.554231+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:13:54.554231+0000","last_undegraded":"2024-04-23T19:13:54.554231+0000","last_fullsized":"2024-04-23T19:13:54.554231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.554115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:38.772240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.518578+0000","last_change":"2024-04-23T19:14:00.518605+0000","last_active":"2024-04-23T19:14:00.518578+0000","last_peered":"2024-04-23T19:14:00.518578+0000","last_clean":"2024-04-23T19:14:00.518578+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:14:00.518578+0000","last_undegraded":"2024-04-23T19:14:00.518578+0000","last_fullsized":"2024-04-23T19:14:00.518578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.518487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:31.027600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.766254+0000","last_change":"2024-04-23T19:13:45.766254+0000","last_active":"2024-04-23T19:13:45.766254+0000","last_peered":"2024-04-23T19:13:45.766254+0000","last_clean":"2024-04-23T19:13:45.766254+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:45.766254+0000","last_undegraded":"2024-04-23T19:13:45.766254+0000","last_fullsized":"2024-04-23T19:13:45.766254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.766137+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:33.256585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.886328+0000","last_change":"2024-04-23T19:14:13.886328+0000","last_active":"2024-04-23T19:14:13.886328+0000","last_peered":"2024-04-23T19:14:13.886328+0000","last_clean":"2024-04-23T19:14:13.886328+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:14:13.886328+0000","last_undegraded":"2024-04-23T19:14:13.886328+0000","last_fullsized":"2024-04-23T19:14:13.886328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:13.886177+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:13.886177+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.886177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:34.657691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.558742+0000","last_change":"2024-04-23T19:14:07.558742+0000","last_active":"2024-04-23T19:14:07.558742+0000","last_peered":"2024-04-23T19:14:07.558742+0000","last_clean":"2024-04-23T19:14:07.558742+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:14:07.558742+0000","last_undegraded":"2024-04-23T19:14:07.558742+0000","last_fullsized":"2024-04-23T19:14:07.558742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:07.558617+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:07.558617+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.558617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:08.555554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.079231+0000","last_change":"2024-04-23T19:13:58.079291+0000","last_active":"2024-04-23T19:13:58.079231+0000","last_peered":"2024-04-23T19:13:58.079231+0000","last_clean":"2024-04-23T19:13:58.079231+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:58.079231+0000","last_undegraded":"2024-04-23T19:13:58.079231+0000","last_fullsized":"2024-04-23T19:13:58.079231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.079130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:35.830705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.883280+0000","last_change":"2024-04-23T19:14:08.883280+0000","last_active":"2024-04-23T19:14:08.883280+0000","last_peered":"2024-04-23T19:14:08.883280+0000","last_clean":"2024-04-23T19:14:08.883280+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:14:08.883280+0000","last_undegraded":"2024-04-23T19:14:08.883280+0000","last_fullsized":"2024-04-23T19:14:08.883280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:08.883162+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:08.883162+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.883162+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:27.108487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.554467+0000","last_change":"2024-04-23T19:14:02.554519+0000","last_active":"2024-04-23T19:14:02.554467+0000","last_peered":"2024-04-23T19:14:02.554467+0000","last_clean":"2024-04-23T19:14:02.554467+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:14:02.554467+0000","last_undegraded":"2024-04-23T19:14:02.554467+0000","last_fullsized":"2024-04-23T19:14:02.554467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.554376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:52.658050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.844100+0000","last_change":"2024-04-23T19:14:04.844128+0000","last_active":"2024-04-23T19:14:04.844100+0000","last_peered":"2024-04-23T19:14:04.844100+0000","last_clean":"2024-04-23T19:14:04.844100+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:14:04.844100+0000","last_undegraded":"2024-04-23T19:14:04.844100+0000","last_fullsized":"2024-04-23T19:14:04.844100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:04.844020+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:04.844020+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.844020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:21:48.661589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.400413+0000","last_change":"2024-04-23T19:10:07.507068+0000","last_active":"2024-04-23T19:13:27.400413+0000","last_peered":"2024-04-23T19:13:27.400413+0000","last_clean":"2024-04-23T19:13:27.400413+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:13:27.400413+0000","last_undegraded":"2024-04-23T19:13:27.400413+0000","last_fullsized":"2024-04-23T19:13:27.400413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.936668+0000","last_change":"2024-04-23T19:14:00.936694+0000","last_active":"2024-04-23T19:14:00.936668+0000","last_peered":"2024-04-23T19:14:00.936668+0000","last_clean":"2024-04-23T19:14:00.936668+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:14:00.936668+0000","last_undegraded":"2024-04-23T19:14:00.936668+0000","last_fullsized":"2024-04-23T19:14:00.936668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.936576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:19.351629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.563662+0000","last_change":"2024-04-23T19:14:12.563709+0000","last_active":"2024-04-23T19:14:12.563662+0000","last_peered":"2024-04-23T19:14:12.563662+0000","last_clean":"2024-04-23T19:14:12.563662+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:14:12.563662+0000","last_undegraded":"2024-04-23T19:14:12.563662+0000","last_fullsized":"2024-04-23T19:14:12.563662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:12.563575+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:12.563575+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.563575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:45.234515+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'6","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.538431+0000","last_change":"2024-04-23T19:13:52.538459+0000","last_active":"2024-04-23T19:13:52.538431+0000","last_peered":"2024-04-23T19:13:52.538431+0000","last_clean":"2024-04-23T19:13:52.538431+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:13:52.538431+0000","last_undegraded":"2024-04-23T19:13:52.538431+0000","last_fullsized":"2024-04-23T19:13:52.538431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.538343+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:34.602802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.859465+0000","last_change":"2024-04-23T19:14:14.859491+0000","last_active":"2024-04-23T19:14:14.859465+0000","last_peered":"2024-04-23T19:14:14.859465+0000","last_clean":"2024-04-23T19:14:14.859465+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:14:14.859465+0000","last_undegraded":"2024-04-23T19:14:14.859465+0000","last_fullsized":"2024-04-23T19:14:14.859465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.859367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.859367+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.859367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:04.076971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.865209+0000","last_change":"2024-04-23T19:13:59.865265+0000","last_active":"2024-04-23T19:13:59.865209+0000","last_peered":"2024-04-23T19:13:59.865209+0000","last_clean":"2024-04-23T19:13:59.865209+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:13:59.865209+0000","last_undegraded":"2024-04-23T19:13:59.865209+0000","last_fullsized":"2024-04-23T19:13:59.865209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.865099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:18:21.512041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.889480+0000","last_change":"2024-04-23T19:14:03.889509+0000","last_active":"2024-04-23T19:14:03.889480+0000","last_peered":"2024-04-23T19:14:03.889480+0000","last_clean":"2024-04-23T19:14:03.889480+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:14:03.889480+0000","last_undegraded":"2024-04-23T19:14:03.889480+0000","last_fullsized":"2024-04-23T19:14:03.889480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:03.889376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:03.889376+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.889376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.241474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.731878+0000","last_change":"2024-04-23T19:13:52.731942+0000","last_active":"2024-04-23T19:13:52.731878+0000","last_peered":"2024-04-23T19:13:52.731878+0000","last_clean":"2024-04-23T19:13:52.731878+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:13:52.731878+0000","last_undegraded":"2024-04-23T19:13:52.731878+0000","last_fullsized":"2024-04-23T19:13:52.731878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.731777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:37.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:21.719175+0000","last_change":"2024-04-23T19:10:07.490658+0000","last_active":"2024-04-23T19:12:21.719175+0000","last_peered":"2024-04-23T19:12:21.719175+0000","last_clean":"2024-04-23T19:12:21.719175+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:12:21.719175+0000","last_undegraded":"2024-04-23T19:12:21.719175+0000","last_fullsized":"2024-04-23T19:12:21.719175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.863139+0000","last_change":"2024-04-23T19:14:02.863186+0000","last_active":"2024-04-23T19:14:02.863139+0000","last_peered":"2024-04-23T19:14:02.863139+0000","last_clean":"2024-04-23T19:14:02.863139+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:14:02.863139+0000","last_undegraded":"2024-04-23T19:14:02.863139+0000","last_fullsized":"2024-04-23T19:14:02.863139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:02.863017+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:02.863017+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.863017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:33.372130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.950982+0000","last_change":"2024-04-23T19:14:22.951020+0000","last_active":"2024-04-23T19:14:22.950982+0000","last_peered":"2024-04-23T19:14:22.950982+0000","last_clean":"2024-04-23T19:14:22.950982+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:14:22.950982+0000","last_undegraded":"2024-04-23T19:14:22.950982+0000","last_fullsized":"2024-04-23T19:14:22.950982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:22.950894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:22.950894+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.950894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:42.201782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'6","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.983604+0000","last_change":"2024-04-23T19:14:06.983645+0000","last_active":"2024-04-23T19:14:06.983604+0000","last_peered":"2024-04-23T19:14:06.983604+0000","last_clean":"2024-04-23T19:14:06.983604+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:14:06.983604+0000","last_undegraded":"2024-04-23T19:14:06.983604+0000","last_fullsized":"2024-04-23T19:14:06.983604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:06.983514+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:06.983514+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.983514+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:29.820347+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.057731+0000","last_change":"2024-04-23T19:14:00.057782+0000","last_active":"2024-04-23T19:14:00.057731+0000","last_peered":"2024-04-23T19:14:00.057731+0000","last_clean":"2024-04-23T19:14:00.057731+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:14:00.057731+0000","last_undegraded":"2024-04-23T19:14:00.057731+0000","last_fullsized":"2024-04-23T19:14:00.057731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.057618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:34.694433+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.779318+0000","last_change":"2024-04-23T19:14:20.779346+0000","last_active":"2024-04-23T19:14:20.779318+0000","last_peered":"2024-04-23T19:14:20.779318+0000","last_clean":"2024-04-23T19:14:20.779318+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:14:20.779318+0000","last_undegraded":"2024-04-23T19:14:20.779318+0000","last_fullsized":"2024-04-23T19:14:20.779318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:20.779226+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:20.779226+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.779226+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:00.435680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:26.720175+0000","last_change":"2024-04-23T19:10:07.507910+0000","last_active":"2024-04-23T19:12:26.720175+0000","last_peered":"2024-04-23T19:12:26.720175+0000","last_clean":"2024-04-23T19:12:26.720175+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:12:26.720175+0000","last_undegraded":"2024-04-23T19:12:26.720175+0000","last_fullsized":"2024-04-23T19:12:26.720175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:06.455223+0000","last_clean_scrub_stamp":"2024-04-23T19:10:06.455223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.879608+0000","last_change":"2024-04-23T19:14:24.879645+0000","last_active":"2024-04-23T19:14:24.879608+0000","last_peered":"2024-04-23T19:14:24.879608+0000","last_clean":"2024-04-23T19:14:24.879608+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:14:24.879608+0000","last_undegraded":"2024-04-23T19:14:24.879608+0000","last_fullsized":"2024-04-23T19:14:24.879608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:24.879513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:24.879513+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.879513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:13:57.197889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.585443+0000","last_change":"2024-04-23T19:14:11.585491+0000","last_active":"2024-04-23T19:14:11.585443+0000","last_peered":"2024-04-23T19:14:11.585443+0000","last_clean":"2024-04-23T19:14:11.585443+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:14:11.585443+0000","last_undegraded":"2024-04-23T19:14:11.585443+0000","last_fullsized":"2024-04-23T19:14:11.585443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:11.585343+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:11.585343+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.585343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:00.802947+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.561399+0000","last_change":"2024-04-23T19:14:17.561430+0000","last_active":"2024-04-23T19:14:17.561399+0000","last_peered":"2024-04-23T19:14:17.561399+0000","last_clean":"2024-04-23T19:14:17.561399+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:14:17.561399+0000","last_undegraded":"2024-04-23T19:14:17.561399+0000","last_fullsized":"2024-04-23T19:14:17.561399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:17.561302+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:17.561302+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.561302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:47.402903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.753730+0000","last_change":"2024-04-23T19:13:53.753760+0000","last_active":"2024-04-23T19:13:53.753730+0000","last_peered":"2024-04-23T19:13:53.753730+0000","last_clean":"2024-04-23T19:13:53.753730+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:53.753730+0000","last_undegraded":"2024-04-23T19:13:53.753730+0000","last_fullsized":"2024-04-23T19:13:53.753730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.753615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:13.659941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1524,"ondisk_log_size":1524,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":481,"ondisk_log_size":481,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673020,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39948,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331892,"statfs":{"total":96636764160,"available":96595857408,"internally_reserved":0,"allocated":462848,"data_stored":144617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673019,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1024000,"data_stored":734097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40468,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331372,"statfs":{"total":96636764160,"available":96595324928,"internally_reserved":0,"allocated":995328,"data_stored":732924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705723,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":425984,"data_stored":144396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:14:28.935 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.936 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.936 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.936 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.936 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.937 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.937 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.938 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.938 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.938 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.938 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.939 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.939 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.939 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.939 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T19:09:58.435679+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.939 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.940 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-23T19:10:08.463033+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=8, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.940 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.940 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.940 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.941 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.941 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.941 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.941 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.941 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.942 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.942 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.942 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.943 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-23T19:10:06.455223+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=6, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:28.943 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T19:14:48.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:14:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.056+0000 7fb2bf9f4640 1 Processor -- start 2024-04-23T19:14:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.057+0000 7fb2bf9f4640 1 -- start start 2024-04-23T19:14:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.057+0000 7fb2bf9f4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2b81288d0 0x7fb2b8128cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.057+0000 7fb2bf9f4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b8128410 con 0x7fb2b8121e40 2024-04-23T19:14:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.057+0000 7fb2bf9f4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b81286b0 con 0x7fb2b81288d0 2024-04-23T19:14:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.057+0000 7fb2bcf8f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb2b8121e40 0x7fb2b8122280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37366/0 (socket says 172.21.15.16:37366) 2024-04-23T19:14:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.058+0000 7fb2bcf8f640 1 -- 172.21.15.16:0/523847727 learned_addr learned my addr 172.21.15.16:0/523847727 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:14:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.057+0000 7fb2bd790640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2b81288d0 0x7fb2b8128cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.058+0000 7fb2bd790640 1 -- 172.21.15.16:0/523847727 >> v1:172.21.15.16:6789/0 conn(0x7fb2b8121e40 legacy=0x7fb2b8122280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.058+0000 7fb2bd790640 1 -- 172.21.15.16:0/523847727 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b806de40 con 0x7fb2b81288d0 2024-04-23T19:14:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bd790640 1 --2- 172.21.15.16:0/523847727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b81288d0 0x7fb2b8128cb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009720 tx=0x7fb2b402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535bdd531ad0cf18 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2affff640 1 -- 172.21.15.16:0/523847727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b40316d0 con 0x7fb2b81288d0 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2affff640 1 -- 172.21.15.16:0/523847727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b4037070 con 0x7fb2b81288d0 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2affff640 1 -- 172.21.15.16:0/523847727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b40319c0 con 0x7fb2b81288d0 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/523847727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b81288d0 msgr2=0x7fb2b8128cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bf9f4640 1 --2- 172.21.15.16:0/523847727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b81288d0 0x7fb2b8128cb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009720 tx=0x7fb2b402b840 comp rx=0 tx=0).stop 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/523847727 shutdown_connections 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bf9f4640 1 --2- 172.21.15.16:0/523847727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b81288d0 0x7fb2b8128cb0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/523847727 >> 172.21.15.16:0/523847727 conn(0x7fb2b809a200 msgr2=0x7fb2b8097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/523847727 shutdown_connections 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.059+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/523847727 wait complete. 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.060+0000 7fb2bf9f4640 1 Processor -- start 2024-04-23T19:14:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.060+0000 7fb2bf9f4640 1 -- start start 2024-04-23T19:14:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2bf9f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b8121e40 0x7fb2b808f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2bf9f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80945f0 con 0x7fb2b8121e40 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2bd790640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b8121e40 0x7fb2b808f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2bd790640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b8121e40 0x7fb2b808f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34430/0 (socket says 172.21.15.16:34430) 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2bd790640 1 -- 172.21.15.16:0/1730156659 learned_addr learned my addr 172.21.15.16:0/1730156659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2bd790640 1 -- 172.21.15.16:0/1730156659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b8072170 con 0x7fb2b8121e40 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2bd790640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b8121e40 0x7fb2b808f3f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009720 tx=0x7fb2b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2adffb640 1 -- 172.21.15.16:0/1730156659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4032770 con 0x7fb2b8121e40 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.061+0000 7fb2adffb640 1 -- 172.21.15.16:0/1730156659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b4032910 con 0x7fb2b8121e40 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.062+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b8091350 con 0x7fb2b8121e40 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.062+0000 7fb2adffb640 1 -- 172.21.15.16:0/1730156659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4032bf0 con 0x7fb2b8121e40 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.062+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b808f930 con 0x7fb2b8121e40 2024-04-23T19:14:49.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.063+0000 7fb2adffb640 1 -- 172.21.15.16:0/1730156659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7fb2b404c060 con 0x7fb2b8121e40 2024-04-23T19:14:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.063+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b809c8f0 con 0x7fb2b8121e40 2024-04-23T19:14:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.063+0000 7fb2adffb640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb29c0385c0 0x7fb29c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:14:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.063+0000 7fb2adffb640 1 -- 172.21.15.16:0/1730156659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7fb2b402f050 con 0x7fb2b8121e40 2024-04-23T19:14:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.063+0000 7fb2bcf8f640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb29c0385c0 0x7fb29c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:14:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.064+0000 7fb2bcf8f640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb29c0385c0 0x7fb29c03aa80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007fd0 tx=0x7fb2a8007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:14:49.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.068+0000 7fb2adffb640 1 -- 172.21.15.16:0/1730156659 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2b809c8f0 con 0x7fb2b8121e40 2024-04-23T19:14:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.248+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2b8090d50 con 0x7fb29c0385c0 2024-04-23T19:14:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.265+0000 7fb2adffb640 1 -- 172.21.15.16:0/1730156659 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665832 (secure 0 0 0) 0x7fb2a80058b0 con 0x7fb29c0385c0 2024-04-23T19:14:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:14:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:14:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb29c0385c0 msgr2=0x7fb29c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb29c0385c0 0x7fb29c03aa80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007fd0 tx=0x7fb2a8007ac0 comp rx=0 tx=0).stop 2024-04-23T19:14:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b8121e40 msgr2=0x7fb2b808f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b8121e40 0x7fb2b808f3f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009720 tx=0x7fb2b40046c0 comp rx=0 tx=0).stop 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 shutdown_connections 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7fb29c0385c0 0x7fb29c03aa80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 --2- 172.21.15.16:0/1730156659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b8121e40 0x7fb2b808f3f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 >> 172.21.15.16:0/1730156659 conn(0x7fb2b809a200 msgr2=0x7fb2b8127040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 shutdown_connections 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:14:49.269+0000 7fb2bf9f4640 1 -- 172.21.15.16:0/1730156659 wait complete. 2024-04-23T19:14:49.283 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-04-23T19:14:48.729492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5092,"num_read_kb":4673,"num_write":2177,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4783,"ondisk_log_size":4783,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":152504,"kb_used_data":2840,"kb_used_omap":6,"kb_used_meta":149625,"kb_avail":377334856,"statfs":{"total":386547056640,"available":386390892544,"internally_reserved":0,"allocated":2908160,"data_stored":1756034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":153216791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003666"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.042420+0000","last_change":"2024-04-23T19:14:14.042420+0000","last_active":"2024-04-23T19:14:14.042420+0000","last_peered":"2024-04-23T19:14:14.042420+0000","last_clean":"2024-04-23T19:14:14.042420+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:14:14.042420+0000","last_undegraded":"2024-04-23T19:14:14.042420+0000","last_fullsized":"2024-04-23T19:14:14.042420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.042287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.042287+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.042287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:13.352752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.897072+0000","last_change":"2024-04-23T19:14:10.897102+0000","last_active":"2024-04-23T19:14:10.897072+0000","last_peered":"2024-04-23T19:14:10.897072+0000","last_clean":"2024-04-23T19:14:10.897072+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:14:10.897072+0000","last_undegraded":"2024-04-23T19:14:10.897072+0000","last_fullsized":"2024-04-23T19:14:10.897072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:10.896986+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:10.896986+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.896986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:27.637541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.564794+0000","last_change":"2024-04-23T19:14:04.564794+0000","last_active":"2024-04-23T19:14:04.564794+0000","last_peered":"2024-04-23T19:14:04.564794+0000","last_clean":"2024-04-23T19:14:04.564794+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:14:04.564794+0000","last_undegraded":"2024-04-23T19:14:04.564794+0000","last_fullsized":"2024-04-23T19:14:04.564794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.564668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:53.911600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.937584+0000","last_change":"2024-04-23T19:13:59.937609+0000","last_active":"2024-04-23T19:13:59.937584+0000","last_peered":"2024-04-23T19:13:59.937584+0000","last_clean":"2024-04-23T19:13:59.937584+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:59.937584+0000","last_undegraded":"2024-04-23T19:13:59.937584+0000","last_fullsized":"2024-04-23T19:13:59.937584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.937477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:01.611995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.987105+0000","last_change":"2024-04-23T19:14:18.987130+0000","last_active":"2024-04-23T19:14:18.987105+0000","last_peered":"2024-04-23T19:14:18.987105+0000","last_clean":"2024-04-23T19:14:18.987105+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:14:18.987105+0000","last_undegraded":"2024-04-23T19:14:18.987105+0000","last_fullsized":"2024-04-23T19:14:18.987105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.986988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.986988+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.986988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:58.819070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.946559+0000","last_change":"2024-04-23T19:13:46.946583+0000","last_active":"2024-04-23T19:13:46.946559+0000","last_peered":"2024-04-23T19:13:46.946559+0000","last_clean":"2024-04-23T19:13:46.946559+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:13:46.946559+0000","last_undegraded":"2024-04-23T19:13:46.946559+0000","last_fullsized":"2024-04-23T19:13:46.946559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.946470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:19.468921+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.055999+0000","last_change":"2024-04-23T19:14:11.056028+0000","last_active":"2024-04-23T19:14:11.055999+0000","last_peered":"2024-04-23T19:14:11.055999+0000","last_clean":"2024-04-23T19:14:11.055999+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:14:11.055999+0000","last_undegraded":"2024-04-23T19:14:11.055999+0000","last_fullsized":"2024-04-23T19:14:11.055999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:11.055907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:11.055907+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.055907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:44:01.159581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.985264+0000","last_change":"2024-04-23T19:14:11.985294+0000","last_active":"2024-04-23T19:14:11.985264+0000","last_peered":"2024-04-23T19:14:11.985264+0000","last_clean":"2024-04-23T19:14:11.985264+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:14:11.985264+0000","last_undegraded":"2024-04-23T19:14:11.985264+0000","last_fullsized":"2024-04-23T19:14:11.985264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:11.985174+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:11.985174+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.985174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:26.660339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.878169+0000","last_change":"2024-04-23T19:14:25.878199+0000","last_active":"2024-04-23T19:14:25.878169+0000","last_peered":"2024-04-23T19:14:25.878169+0000","last_clean":"2024-04-23T19:14:25.878169+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:14:25.878169+0000","last_undegraded":"2024-04-23T19:14:25.878169+0000","last_fullsized":"2024-04-23T19:14:25.878169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:25.878068+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:25.878068+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.878068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:20.382452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.789955+0000","last_change":"2024-04-23T19:13:57.789955+0000","last_active":"2024-04-23T19:13:57.789955+0000","last_peered":"2024-04-23T19:13:57.789955+0000","last_clean":"2024-04-23T19:13:57.789955+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:13:57.789955+0000","last_undegraded":"2024-04-23T19:13:57.789955+0000","last_fullsized":"2024-04-23T19:13:57.789955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.789839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:44.659415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.007123+0000","last_change":"2024-04-23T19:14:12.007150+0000","last_active":"2024-04-23T19:14:12.007123+0000","last_peered":"2024-04-23T19:14:12.007123+0000","last_clean":"2024-04-23T19:14:12.007123+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:14:12.007123+0000","last_undegraded":"2024-04-23T19:14:12.007123+0000","last_fullsized":"2024-04-23T19:14:12.007123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:12.007017+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:12.007017+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.007017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:38.418068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":86,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.905886+0000","last_change":"2024-04-23T19:13:44.905886+0000","last_active":"2024-04-23T19:13:44.905886+0000","last_peered":"2024-04-23T19:13:44.905886+0000","last_clean":"2024-04-23T19:13:44.905886+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:44.905886+0000","last_undegraded":"2024-04-23T19:13:44.905886+0000","last_fullsized":"2024-04-23T19:13:44.905886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.905747+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.252680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:40.840136+0000","last_change":"2024-04-23T19:14:40.840161+0000","last_active":"2024-04-23T19:14:40.840136+0000","last_peered":"2024-04-23T19:14:40.840136+0000","last_clean":"2024-04-23T19:14:40.840136+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:14:40.840136+0000","last_undegraded":"2024-04-23T19:14:40.840136+0000","last_fullsized":"2024-04-23T19:14:40.840136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:40.840047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:40.840047+0000","last_clean_scrub_stamp":"2024-04-23T19:14:40.840047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:35:20.185487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'24","reported_seq":100,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.035838+0000","last_change":"2024-04-23T19:13:57.035838+0000","last_active":"2024-04-23T19:13:57.035838+0000","last_peered":"2024-04-23T19:13:57.035838+0000","last_clean":"2024-04-23T19:13:57.035838+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:57.035838+0000","last_undegraded":"2024-04-23T19:13:57.035838+0000","last_fullsized":"2024-04-23T19:13:57.035838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'24","last_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_deep_scrub":"32'24","last_deep_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.035696+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:34.133123+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.567190+0000","last_change":"2024-04-23T19:14:19.567220+0000","last_active":"2024-04-23T19:14:19.567190+0000","last_peered":"2024-04-23T19:14:19.567190+0000","last_clean":"2024-04-23T19:14:19.567190+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:14:19.567190+0000","last_undegraded":"2024-04-23T19:14:19.567190+0000","last_fullsized":"2024-04-23T19:14:19.567190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.567093+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.567093+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.567093+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.943962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'18","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.074121+0000","last_change":"2024-04-23T19:14:15.074121+0000","last_active":"2024-04-23T19:14:15.074121+0000","last_peered":"2024-04-23T19:14:15.074121+0000","last_clean":"2024-04-23T19:14:15.074121+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:14:15.074121+0000","last_undegraded":"2024-04-23T19:14:15.074121+0000","last_fullsized":"2024-04-23T19:14:15.074121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'18","last_scrub_stamp":"2024-04-23T19:14:15.073956+0000","last_deep_scrub":"32'18","last_deep_scrub_stamp":"2024-04-23T19:14:15.073956+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.073956+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:47.596407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.943596+0000","last_change":"2024-04-23T19:13:43.943623+0000","last_active":"2024-04-23T19:13:43.943596+0000","last_peered":"2024-04-23T19:13:43.943596+0000","last_clean":"2024-04-23T19:13:43.943596+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:13:43.943596+0000","last_undegraded":"2024-04-23T19:13:43.943596+0000","last_fullsized":"2024-04-23T19:13:43.943596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.943496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:10.354547+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.824684+0000","last_change":"2024-04-23T19:14:15.824733+0000","last_active":"2024-04-23T19:14:15.824684+0000","last_peered":"2024-04-23T19:14:15.824684+0000","last_clean":"2024-04-23T19:14:15.824684+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:14:15.824684+0000","last_undegraded":"2024-04-23T19:14:15.824684+0000","last_fullsized":"2024-04-23T19:14:15.824684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:15.824569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:15.824569+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.824569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:40.296843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.911745+0000","last_change":"2024-04-23T19:14:00.911780+0000","last_active":"2024-04-23T19:14:00.911745+0000","last_peered":"2024-04-23T19:14:00.911745+0000","last_clean":"2024-04-23T19:14:00.911745+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:14:00.911745+0000","last_undegraded":"2024-04-23T19:14:00.911745+0000","last_fullsized":"2024-04-23T19:14:00.911745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.911657+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.206563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.028413+0000","last_change":"2024-04-23T19:14:10.028439+0000","last_active":"2024-04-23T19:14:10.028413+0000","last_peered":"2024-04-23T19:14:10.028413+0000","last_clean":"2024-04-23T19:14:10.028413+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:14:10.028413+0000","last_undegraded":"2024-04-23T19:14:10.028413+0000","last_fullsized":"2024-04-23T19:14:10.028413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:10.028304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:10.028304+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.028304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:18.764089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.839412+0000","last_change":"2024-04-23T19:14:06.839443+0000","last_active":"2024-04-23T19:14:06.839412+0000","last_peered":"2024-04-23T19:14:06.839412+0000","last_clean":"2024-04-23T19:14:06.839412+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:14:06.839412+0000","last_undegraded":"2024-04-23T19:14:06.839412+0000","last_fullsized":"2024-04-23T19:14:06.839412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:06.839307+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:06.839307+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.839307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:24.085334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.796508+0000","last_change":"2024-04-23T19:14:16.796535+0000","last_active":"2024-04-23T19:14:16.796508+0000","last_peered":"2024-04-23T19:14:16.796508+0000","last_clean":"2024-04-23T19:14:16.796508+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:14:16.796508+0000","last_undegraded":"2024-04-23T19:14:16.796508+0000","last_fullsized":"2024-04-23T19:14:16.796508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:16.796417+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:16.796417+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.796417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:01.073302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.526561+0000","last_change":"2024-04-23T19:13:56.526589+0000","last_active":"2024-04-23T19:13:56.526561+0000","last_peered":"2024-04-23T19:13:56.526561+0000","last_clean":"2024-04-23T19:13:56.526561+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:56.526561+0000","last_undegraded":"2024-04-23T19:13:56.526561+0000","last_fullsized":"2024-04-23T19:13:56.526561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.526450+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:02.826857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.855594+0000","last_change":"2024-04-23T19:14:07.855627+0000","last_active":"2024-04-23T19:14:07.855594+0000","last_peered":"2024-04-23T19:14:07.855594+0000","last_clean":"2024-04-23T19:14:07.855594+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:14:07.855594+0000","last_undegraded":"2024-04-23T19:14:07.855594+0000","last_fullsized":"2024-04-23T19:14:07.855594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.855478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.855478+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.855478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:53:54.172855+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.044859+0000","last_change":"2024-04-23T19:14:03.044887+0000","last_active":"2024-04-23T19:14:03.044859+0000","last_peered":"2024-04-23T19:14:03.044859+0000","last_clean":"2024-04-23T19:14:03.044859+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:14:03.044859+0000","last_undegraded":"2024-04-23T19:14:03.044859+0000","last_fullsized":"2024-04-23T19:14:03.044859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.044766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:47.262891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.578908+0000","last_change":"2024-04-23T19:14:06.578937+0000","last_active":"2024-04-23T19:14:06.578908+0000","last_peered":"2024-04-23T19:14:06.578908+0000","last_clean":"2024-04-23T19:14:06.578908+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:14:06.578908+0000","last_undegraded":"2024-04-23T19:14:06.578908+0000","last_fullsized":"2024-04-23T19:14:06.578908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:06.578801+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:06.578801+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.578801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:39.101252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.816135+0000","last_change":"2024-04-23T19:13:56.816160+0000","last_active":"2024-04-23T19:13:56.816135+0000","last_peered":"2024-04-23T19:13:56.816135+0000","last_clean":"2024-04-23T19:13:56.816135+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:56.816135+0000","last_undegraded":"2024-04-23T19:13:56.816135+0000","last_fullsized":"2024-04-23T19:13:56.816135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.816040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:41.034165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:32.834694+0000","last_change":"2024-04-23T19:14:32.834694+0000","last_active":"2024-04-23T19:14:32.834694+0000","last_peered":"2024-04-23T19:14:32.834694+0000","last_clean":"2024-04-23T19:14:32.834694+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:14:32.834694+0000","last_undegraded":"2024-04-23T19:14:32.834694+0000","last_fullsized":"2024-04-23T19:14:32.834694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:32.834539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:32.834539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:32.834539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:30.099684+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'36","reported_seq":109,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.799937+0000","last_change":"2024-04-23T19:13:54.799967+0000","last_active":"2024-04-23T19:13:54.799937+0000","last_peered":"2024-04-23T19:13:54.799937+0000","last_clean":"2024-04-23T19:13:54.799937+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:54.799937+0000","last_undegraded":"2024-04-23T19:13:54.799937+0000","last_fullsized":"2024-04-23T19:13:54.799937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'36","last_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_deep_scrub":"32'36","last_deep_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.799848+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:43.632765+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.014609+0000","last_change":"2024-04-23T19:14:15.014637+0000","last_active":"2024-04-23T19:14:15.014609+0000","last_peered":"2024-04-23T19:14:15.014609+0000","last_clean":"2024-04-23T19:14:15.014609+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:14:15.014609+0000","last_undegraded":"2024-04-23T19:14:15.014609+0000","last_fullsized":"2024-04-23T19:14:15.014609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:15.014507+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:15.014507+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.014507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:49.779222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:31.876322+0000","last_change":"2024-04-23T19:14:31.876322+0000","last_active":"2024-04-23T19:14:31.876322+0000","last_peered":"2024-04-23T19:14:31.876322+0000","last_clean":"2024-04-23T19:14:31.876322+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:14:31.876322+0000","last_undegraded":"2024-04-23T19:14:31.876322+0000","last_fullsized":"2024-04-23T19:14:31.876322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:31.876162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:31.876162+0000","last_clean_scrub_stamp":"2024-04-23T19:14:31.876162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:46:12.190852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.986548+0000","last_change":"2024-04-23T19:13:32.986571+0000","last_active":"2024-04-23T19:13:32.986548+0000","last_peered":"2024-04-23T19:13:32.986548+0000","last_clean":"2024-04-23T19:13:32.986548+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:13:32.986548+0000","last_undegraded":"2024-04-23T19:13:32.986548+0000","last_fullsized":"2024-04-23T19:13:32.986548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.986453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:15.050911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.889110+0000","last_change":"2024-04-23T19:13:41.889150+0000","last_active":"2024-04-23T19:13:41.889110+0000","last_peered":"2024-04-23T19:13:41.889110+0000","last_clean":"2024-04-23T19:13:41.889110+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:13:41.889110+0000","last_undegraded":"2024-04-23T19:13:41.889110+0000","last_fullsized":"2024-04-23T19:13:41.889110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.889007+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:50.624792+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.929910+0000","last_change":"2024-04-23T19:13:38.929939+0000","last_active":"2024-04-23T19:13:38.929910+0000","last_peered":"2024-04-23T19:13:38.929910+0000","last_clean":"2024-04-23T19:13:38.929910+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:13:38.929910+0000","last_undegraded":"2024-04-23T19:13:38.929910+0000","last_fullsized":"2024-04-23T19:13:38.929910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.929807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:00.218571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'10","reported_seq":37,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.758575+0000","last_change":"2024-04-23T19:13:50.758606+0000","last_active":"2024-04-23T19:13:50.758575+0000","last_peered":"2024-04-23T19:13:50.758575+0000","last_clean":"2024-04-23T19:13:50.758575+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:50.758575+0000","last_undegraded":"2024-04-23T19:13:50.758575+0000","last_fullsized":"2024-04-23T19:13:50.758575+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.758473+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:28.510048+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":82,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.892739+0000","last_change":"2024-04-23T19:14:04.892767+0000","last_active":"2024-04-23T19:14:04.892739+0000","last_peered":"2024-04-23T19:14:04.892739+0000","last_clean":"2024-04-23T19:14:04.892739+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:14:04.892739+0000","last_undegraded":"2024-04-23T19:14:04.892739+0000","last_fullsized":"2024-04-23T19:14:04.892739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:14:04.892651+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:14:04.892651+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.892651+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:00.034408+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.010646+0000","last_change":"2024-04-23T19:14:04.010672+0000","last_active":"2024-04-23T19:14:04.010646+0000","last_peered":"2024-04-23T19:14:04.010646+0000","last_clean":"2024-04-23T19:14:04.010646+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:14:04.010646+0000","last_undegraded":"2024-04-23T19:14:04.010646+0000","last_fullsized":"2024-04-23T19:14:04.010646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.010558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:50.880134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":230,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.491387+0000","last_change":"2024-04-23T19:13:50.491419+0000","last_active":"2024-04-23T19:13:50.491387+0000","last_peered":"2024-04-23T19:13:50.491387+0000","last_clean":"2024-04-23T19:13:50.491387+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:13:50.491387+0000","last_undegraded":"2024-04-23T19:13:50.491387+0000","last_fullsized":"2024-04-23T19:13:50.491387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.491297+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:21.383865+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"32'9","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.851896+0000","last_change":"2024-04-23T19:13:32.851932+0000","last_active":"2024-04-23T19:13:32.851896+0000","last_peered":"2024-04-23T19:13:32.851896+0000","last_clean":"2024-04-23T19:13:32.851896+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:13:32.851896+0000","last_undegraded":"2024-04-23T19:13:32.851896+0000","last_fullsized":"2024-04-23T19:13:32.851896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'9","last_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_deep_scrub":"32'9","last_deep_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.851806+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:35.714512+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.911789+0000","last_change":"2024-04-23T19:13:34.911815+0000","last_active":"2024-04-23T19:13:34.911789+0000","last_peered":"2024-04-23T19:13:34.911789+0000","last_clean":"2024-04-23T19:13:34.911789+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:13:34.911789+0000","last_undegraded":"2024-04-23T19:13:34.911789+0000","last_fullsized":"2024-04-23T19:13:34.911789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.911697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:51.912852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.426732+0000","last_change":"2024-04-23T19:14:25.426761+0000","last_active":"2024-04-23T19:14:25.426732+0000","last_peered":"2024-04-23T19:14:25.426732+0000","last_clean":"2024-04-23T19:14:25.426732+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:14:25.426732+0000","last_undegraded":"2024-04-23T19:14:25.426732+0000","last_fullsized":"2024-04-23T19:14:25.426732+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:25.426641+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:25.426641+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.426641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:20.568398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.976956+0000","last_change":"2024-04-23T19:14:10.976979+0000","last_active":"2024-04-23T19:14:10.976956+0000","last_peered":"2024-04-23T19:14:10.976956+0000","last_clean":"2024-04-23T19:14:10.976956+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:14:10.976956+0000","last_undegraded":"2024-04-23T19:14:10.976956+0000","last_fullsized":"2024-04-23T19:14:10.976956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:10.976867+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:10.976867+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.976867+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:05.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.919693+0000","last_change":"2024-04-23T19:14:23.919693+0000","last_active":"2024-04-23T19:14:23.919693+0000","last_peered":"2024-04-23T19:14:23.919693+0000","last_clean":"2024-04-23T19:14:23.919693+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:14:23.919693+0000","last_undegraded":"2024-04-23T19:14:23.919693+0000","last_fullsized":"2024-04-23T19:14:23.919693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:23.919566+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:23.919566+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.919566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:55.156177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.612267+0000","last_change":"2024-04-23T19:13:44.612314+0000","last_active":"2024-04-23T19:13:44.612267+0000","last_peered":"2024-04-23T19:13:44.612267+0000","last_clean":"2024-04-23T19:13:44.612267+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:13:44.612267+0000","last_undegraded":"2024-04-23T19:13:44.612267+0000","last_fullsized":"2024-04-23T19:13:44.612267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.612159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:59.958714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'16","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.181820+0000","last_change":"2024-04-23T19:13:36.181858+0000","last_active":"2024-04-23T19:13:36.181820+0000","last_peered":"2024-04-23T19:13:36.181820+0000","last_clean":"2024-04-23T19:13:36.181820+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:13:36.181820+0000","last_undegraded":"2024-04-23T19:13:36.181820+0000","last_fullsized":"2024-04-23T19:13:36.181820+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.181735+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:21.509377+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'286","reported_seq":398,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.939272+0000","last_change":"2024-04-23T19:14:05.939305+0000","last_active":"2024-04-23T19:14:05.939272+0000","last_peered":"2024-04-23T19:14:05.939272+0000","last_clean":"2024-04-23T19:14:05.939272+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:14:05.939272+0000","last_undegraded":"2024-04-23T19:14:05.939272+0000","last_fullsized":"2024-04-23T19:14:05.939272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'286","last_scrub_stamp":"2024-04-23T19:14:05.939186+0000","last_deep_scrub":"32'286","last_deep_scrub_stamp":"2024-04-23T19:14:05.939186+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.939186+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:33.131024+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.127003+0000","last_change":"2024-04-23T19:14:17.127033+0000","last_active":"2024-04-23T19:14:17.127003+0000","last_peered":"2024-04-23T19:14:17.127003+0000","last_clean":"2024-04-23T19:14:17.127003+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:14:17.127003+0000","last_undegraded":"2024-04-23T19:14:17.127003+0000","last_fullsized":"2024-04-23T19:14:17.127003+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:17.126915+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:17.126915+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.126915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:46:04.993448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.491090+0000","last_change":"2024-04-23T19:14:21.491116+0000","last_active":"2024-04-23T19:14:21.491090+0000","last_peered":"2024-04-23T19:14:21.491090+0000","last_clean":"2024-04-23T19:14:21.491090+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:14:21.491090+0000","last_undegraded":"2024-04-23T19:14:21.491090+0000","last_fullsized":"2024-04-23T19:14:21.491090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:21.490995+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:21.490995+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.490995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:39.906047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.907441+0000","last_change":"2024-04-23T19:14:01.907441+0000","last_active":"2024-04-23T19:14:01.907441+0000","last_peered":"2024-04-23T19:14:01.907441+0000","last_clean":"2024-04-23T19:14:01.907441+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:14:01.907441+0000","last_undegraded":"2024-04-23T19:14:01.907441+0000","last_fullsized":"2024-04-23T19:14:01.907441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:01.907314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:01.907314+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.907314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:33:49.391209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.940488+0000","last_change":"2024-04-23T19:13:34.940545+0000","last_active":"2024-04-23T19:13:34.940488+0000","last_peered":"2024-04-23T19:13:34.940488+0000","last_clean":"2024-04-23T19:13:34.940488+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:13:34.940488+0000","last_undegraded":"2024-04-23T19:13:34.940488+0000","last_fullsized":"2024-04-23T19:13:34.940488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.940371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:19.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.236705+0000","last_change":"2024-04-23T19:13:34.236743+0000","last_active":"2024-04-23T19:13:34.236705+0000","last_peered":"2024-04-23T19:13:34.236705+0000","last_clean":"2024-04-23T19:13:34.236705+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:13:34.236705+0000","last_undegraded":"2024-04-23T19:13:34.236705+0000","last_fullsized":"2024-04-23T19:13:34.236705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.236613+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:39.299958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'285","reported_seq":413,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.684726+0000","last_change":"2024-04-23T19:13:32.684754+0000","last_active":"2024-04-23T19:13:32.684726+0000","last_peered":"2024-04-23T19:13:32.684726+0000","last_clean":"2024-04-23T19:13:32.684726+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:32.684726+0000","last_undegraded":"2024-04-23T19:13:32.684726+0000","last_fullsized":"2024-04-23T19:13:32.684726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.684622+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:16.152805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.163019+0000","last_change":"2024-04-23T19:14:20.163047+0000","last_active":"2024-04-23T19:14:20.163019+0000","last_peered":"2024-04-23T19:14:20.163019+0000","last_clean":"2024-04-23T19:14:20.163019+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:14:20.163019+0000","last_undegraded":"2024-04-23T19:14:20.163019+0000","last_fullsized":"2024-04-23T19:14:20.163019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:20.162930+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:20.162930+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.162930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:09.380533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.053095+0000","last_change":"2024-04-23T19:14:13.053095+0000","last_active":"2024-04-23T19:14:13.053095+0000","last_peered":"2024-04-23T19:14:13.053095+0000","last_clean":"2024-04-23T19:14:13.053095+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:14:13.053095+0000","last_undegraded":"2024-04-23T19:14:13.053095+0000","last_fullsized":"2024-04-23T19:14:13.053095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:13.052973+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:13.052973+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.052973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:02.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.919152+0000","last_change":"2024-04-23T19:13:40.919152+0000","last_active":"2024-04-23T19:13:40.919152+0000","last_peered":"2024-04-23T19:13:40.919152+0000","last_clean":"2024-04-23T19:13:40.919152+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:13:40.919152+0000","last_undegraded":"2024-04-23T19:13:40.919152+0000","last_fullsized":"2024-04-23T19:13:40.919152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.919038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:35.160636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'51","reported_seq":112,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.206079+0000","last_change":"2024-04-23T19:13:39.206079+0000","last_active":"2024-04-23T19:13:39.206079+0000","last_peered":"2024-04-23T19:13:39.206079+0000","last_clean":"2024-04-23T19:13:39.206079+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:13:39.206079+0000","last_undegraded":"2024-04-23T19:13:39.206079+0000","last_fullsized":"2024-04-23T19:13:39.206079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'51","last_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_deep_scrub":"32'51","last_deep_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.205895+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:03.299394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.646133+0000","last_change":"2024-04-23T19:13:43.646133+0000","last_active":"2024-04-23T19:13:43.646133+0000","last_peered":"2024-04-23T19:13:43.646133+0000","last_clean":"2024-04-23T19:13:43.646133+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:13:43.646133+0000","last_undegraded":"2024-04-23T19:13:43.646133+0000","last_fullsized":"2024-04-23T19:13:43.646133+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.646006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:44.344504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.099221+0000","last_change":"2024-04-23T19:14:16.099221+0000","last_active":"2024-04-23T19:14:16.099221+0000","last_peered":"2024-04-23T19:14:16.099221+0000","last_clean":"2024-04-23T19:14:16.099221+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:14:16.099221+0000","last_undegraded":"2024-04-23T19:14:16.099221+0000","last_fullsized":"2024-04-23T19:14:16.099221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:16.099098+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:16.099098+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.099098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:32.839653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.835852+0000","last_change":"2024-04-23T19:13:58.835852+0000","last_active":"2024-04-23T19:13:58.835852+0000","last_peered":"2024-04-23T19:13:58.835852+0000","last_clean":"2024-04-23T19:13:58.835852+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:58.835852+0000","last_undegraded":"2024-04-23T19:13:58.835852+0000","last_fullsized":"2024-04-23T19:13:58.835852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.835665+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:05.136073+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.024266+0000","last_change":"2024-04-23T19:13:54.024266+0000","last_active":"2024-04-23T19:13:54.024266+0000","last_peered":"2024-04-23T19:13:54.024266+0000","last_clean":"2024-04-23T19:13:54.024266+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:13:54.024266+0000","last_undegraded":"2024-04-23T19:13:54.024266+0000","last_fullsized":"2024-04-23T19:13:54.024266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.024146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:24.982777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.680645+0000","last_change":"2024-04-23T19:13:42.680645+0000","last_active":"2024-04-23T19:13:42.680645+0000","last_peered":"2024-04-23T19:13:42.680645+0000","last_clean":"2024-04-23T19:13:42.680645+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:13:42.680645+0000","last_undegraded":"2024-04-23T19:13:42.680645+0000","last_fullsized":"2024-04-23T19:13:42.680645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.680534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:03.703466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'38","reported_seq":94,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.936900+0000","last_change":"2024-04-23T19:13:58.936900+0000","last_active":"2024-04-23T19:13:58.936900+0000","last_peered":"2024-04-23T19:13:58.936900+0000","last_clean":"2024-04-23T19:13:58.936900+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:13:58.936900+0000","last_undegraded":"2024-04-23T19:13:58.936900+0000","last_fullsized":"2024-04-23T19:13:58.936900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.936780+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:49.313649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.585132+0000","last_change":"2024-04-23T19:13:28.585132+0000","last_active":"2024-04-23T19:13:28.585132+0000","last_peered":"2024-04-23T19:13:28.585132+0000","last_clean":"2024-04-23T19:13:28.585132+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:13:28.585132+0000","last_undegraded":"2024-04-23T19:13:28.585132+0000","last_fullsized":"2024-04-23T19:13:28.585132+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.585004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:05.071466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'6","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.768856+0000","last_change":"2024-04-23T19:14:24.768856+0000","last_active":"2024-04-23T19:14:24.768856+0000","last_peered":"2024-04-23T19:14:24.768856+0000","last_clean":"2024-04-23T19:14:24.768856+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:14:24.768856+0000","last_undegraded":"2024-04-23T19:14:24.768856+0000","last_fullsized":"2024-04-23T19:14:24.768856+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:24.768667+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:24.768667+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.768667+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:28.438866+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.098652+0000","last_change":"2024-04-23T19:14:01.098652+0000","last_active":"2024-04-23T19:14:01.098652+0000","last_peered":"2024-04-23T19:14:01.098652+0000","last_clean":"2024-04-23T19:14:01.098652+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:14:01.098652+0000","last_undegraded":"2024-04-23T19:14:01.098652+0000","last_fullsized":"2024-04-23T19:14:01.098652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.098506+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:43.162965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:42.890364+0000","last_change":"2024-04-23T19:14:42.890402+0000","last_active":"2024-04-23T19:14:42.890364+0000","last_peered":"2024-04-23T19:14:42.890364+0000","last_clean":"2024-04-23T19:14:42.890364+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:14:42.890364+0000","last_undegraded":"2024-04-23T19:14:42.890364+0000","last_fullsized":"2024-04-23T19:14:42.890364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:42.890265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:42.890265+0000","last_clean_scrub_stamp":"2024-04-23T19:14:42.890265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:54:21.109740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.573946+0000","last_change":"2024-04-23T19:13:45.573946+0000","last_active":"2024-04-23T19:13:45.573946+0000","last_peered":"2024-04-23T19:13:45.573946+0000","last_clean":"2024-04-23T19:13:45.573946+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:13:45.573946+0000","last_undegraded":"2024-04-23T19:13:45.573946+0000","last_fullsized":"2024-04-23T19:13:45.573946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.573803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:16.532433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'45","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.889437+0000","last_change":"2024-04-23T19:13:50.889437+0000","last_active":"2024-04-23T19:13:50.889437+0000","last_peered":"2024-04-23T19:13:50.889437+0000","last_clean":"2024-04-23T19:13:50.889437+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:13:50.889437+0000","last_undegraded":"2024-04-23T19:13:50.889437+0000","last_fullsized":"2024-04-23T19:13:50.889437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.889290+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:52.811988+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'285","reported_seq":361,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.512239+0000","last_change":"2024-04-23T19:13:51.512239+0000","last_active":"2024-04-23T19:13:51.512239+0000","last_peered":"2024-04-23T19:13:51.512239+0000","last_clean":"2024-04-23T19:13:51.512239+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:51.512239+0000","last_undegraded":"2024-04-23T19:13:51.512239+0000","last_fullsized":"2024-04-23T19:13:51.512239+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.512072+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:26.148887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:26.407672+0000","last_change":"2024-04-23T19:14:26.407672+0000","last_active":"2024-04-23T19:14:26.407672+0000","last_peered":"2024-04-23T19:14:26.407672+0000","last_clean":"2024-04-23T19:14:26.407672+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:14:26.407672+0000","last_undegraded":"2024-04-23T19:14:26.407672+0000","last_fullsized":"2024-04-23T19:14:26.407672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:26.407554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:26.407554+0000","last_clean_scrub_stamp":"2024-04-23T19:14:26.407554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:53.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.092424+0000","last_change":"2024-04-23T19:13:51.092424+0000","last_active":"2024-04-23T19:13:51.092424+0000","last_peered":"2024-04-23T19:13:51.092424+0000","last_clean":"2024-04-23T19:13:51.092424+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:51.092424+0000","last_undegraded":"2024-04-23T19:13:51.092424+0000","last_fullsized":"2024-04-23T19:13:51.092424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.092261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:12.115193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:26.879545+0000","last_change":"2024-04-23T19:14:26.879545+0000","last_active":"2024-04-23T19:14:26.879545+0000","last_peered":"2024-04-23T19:14:26.879545+0000","last_clean":"2024-04-23T19:14:26.879545+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:14:26.879545+0000","last_undegraded":"2024-04-23T19:14:26.879545+0000","last_fullsized":"2024-04-23T19:14:26.879545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:26.879429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:26.879429+0000","last_clean_scrub_stamp":"2024-04-23T19:14:26.879429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:43.879098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.667080+0000","last_change":"2024-04-23T19:13:41.667080+0000","last_active":"2024-04-23T19:13:41.667080+0000","last_peered":"2024-04-23T19:13:41.667080+0000","last_clean":"2024-04-23T19:13:41.667080+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:13:41.667080+0000","last_undegraded":"2024-04-23T19:13:41.667080+0000","last_fullsized":"2024-04-23T19:13:41.667080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.666920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:52.673059+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'63","reported_seq":137,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.975191+0000","last_change":"2024-04-23T19:13:33.975191+0000","last_active":"2024-04-23T19:13:33.975191+0000","last_peered":"2024-04-23T19:13:33.975191+0000","last_clean":"2024-04-23T19:13:33.975191+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:13:33.975191+0000","last_undegraded":"2024-04-23T19:13:33.975191+0000","last_fullsized":"2024-04-23T19:13:33.975191+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.975049+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:07.187015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.695444+0000","last_change":"2024-04-23T19:13:35.695444+0000","last_active":"2024-04-23T19:13:35.695444+0000","last_peered":"2024-04-23T19:13:35.695444+0000","last_clean":"2024-04-23T19:13:35.695444+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:13:35.695444+0000","last_undegraded":"2024-04-23T19:13:35.695444+0000","last_fullsized":"2024-04-23T19:13:35.695444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.695326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:43.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'21","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.534034+0000","last_change":"2024-04-23T19:14:22.534034+0000","last_active":"2024-04-23T19:14:22.534034+0000","last_peered":"2024-04-23T19:14:22.534034+0000","last_clean":"2024-04-23T19:14:22.534034+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:14:22.534034+0000","last_undegraded":"2024-04-23T19:14:22.534034+0000","last_fullsized":"2024-04-23T19:14:22.534034+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:14:22.533903+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:14:22.533903+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.533903+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:26.204102+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.796803+0000","last_change":"2024-04-23T19:13:55.796803+0000","last_active":"2024-04-23T19:13:55.796803+0000","last_peered":"2024-04-23T19:13:55.796803+0000","last_clean":"2024-04-23T19:13:55.796803+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:55.796803+0000","last_undegraded":"2024-04-23T19:13:55.796803+0000","last_fullsized":"2024-04-23T19:13:55.796803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.796686+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:51:24.793617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.565751+0000","last_change":"2024-04-23T19:14:10.565751+0000","last_active":"2024-04-23T19:14:10.565751+0000","last_peered":"2024-04-23T19:14:10.565751+0000","last_clean":"2024-04-23T19:14:10.565751+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:14:10.565751+0000","last_undegraded":"2024-04-23T19:14:10.565751+0000","last_fullsized":"2024-04-23T19:14:10.565751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:10.565616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:10.565616+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.565616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:31.337130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.190374+0000","last_change":"2024-04-23T19:13:44.190374+0000","last_active":"2024-04-23T19:13:44.190374+0000","last_peered":"2024-04-23T19:13:44.190374+0000","last_clean":"2024-04-23T19:13:44.190374+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:13:44.190374+0000","last_undegraded":"2024-04-23T19:13:44.190374+0000","last_fullsized":"2024-04-23T19:13:44.190374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.190242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:32.140502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'39","reported_seq":96,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.877469+0000","last_change":"2024-04-23T19:13:33.877469+0000","last_active":"2024-04-23T19:13:33.877469+0000","last_peered":"2024-04-23T19:13:33.877469+0000","last_clean":"2024-04-23T19:13:33.877469+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:13:33.877469+0000","last_undegraded":"2024-04-23T19:13:33.877469+0000","last_fullsized":"2024-04-23T19:13:33.877469+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.877330+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:17.345513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.951037+0000","last_change":"2024-04-23T19:13:48.951037+0000","last_active":"2024-04-23T19:13:48.951037+0000","last_peered":"2024-04-23T19:13:48.951037+0000","last_clean":"2024-04-23T19:13:48.951037+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:13:48.951037+0000","last_undegraded":"2024-04-23T19:13:48.951037+0000","last_fullsized":"2024-04-23T19:13:48.951037+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.950905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:41.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'4","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.014390+0000","last_change":"2024-04-23T19:14:10.014390+0000","last_active":"2024-04-23T19:14:10.014390+0000","last_peered":"2024-04-23T19:14:10.014390+0000","last_clean":"2024-04-23T19:14:10.014390+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:14:10.014390+0000","last_undegraded":"2024-04-23T19:14:10.014390+0000","last_fullsized":"2024-04-23T19:14:10.014390+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:10.014222+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:10.014222+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.014222+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:00:32.382010+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.920017+0000","last_change":"2024-04-23T19:13:42.920017+0000","last_active":"2024-04-23T19:13:42.920017+0000","last_peered":"2024-04-23T19:13:42.920017+0000","last_clean":"2024-04-23T19:13:42.920017+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:42.920017+0000","last_undegraded":"2024-04-23T19:13:42.920017+0000","last_fullsized":"2024-04-23T19:13:42.920017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.919888+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:33.023562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:30.907972+0000","last_change":"2024-04-23T19:14:30.907995+0000","last_active":"2024-04-23T19:14:30.907972+0000","last_peered":"2024-04-23T19:14:30.907972+0000","last_clean":"2024-04-23T19:14:30.907972+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:14:30.907972+0000","last_undegraded":"2024-04-23T19:14:30.907972+0000","last_fullsized":"2024-04-23T19:14:30.907972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:30.907880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:30.907880+0000","last_clean_scrub_stamp":"2024-04-23T19:14:30.907880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:16.078190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.565530+0000","last_change":"2024-04-23T19:13:46.565530+0000","last_active":"2024-04-23T19:13:46.565530+0000","last_peered":"2024-04-23T19:13:46.565530+0000","last_clean":"2024-04-23T19:13:46.565530+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:13:46.565530+0000","last_undegraded":"2024-04-23T19:13:46.565530+0000","last_fullsized":"2024-04-23T19:13:46.565530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.565380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:54.525905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'45","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.188736+0000","last_change":"2024-04-23T19:13:40.188736+0000","last_active":"2024-04-23T19:13:40.188736+0000","last_peered":"2024-04-23T19:13:40.188736+0000","last_clean":"2024-04-23T19:13:40.188736+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:13:40.188736+0000","last_undegraded":"2024-04-23T19:13:40.188736+0000","last_fullsized":"2024-04-23T19:13:40.188736+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.188598+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:54.836675+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'285","reported_seq":416,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.128972+0000","last_change":"2024-04-23T19:13:28.128972+0000","last_active":"2024-04-23T19:13:28.128972+0000","last_peered":"2024-04-23T19:13:28.128972+0000","last_clean":"2024-04-23T19:13:28.128972+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:28.128972+0000","last_undegraded":"2024-04-23T19:13:28.128972+0000","last_fullsized":"2024-04-23T19:13:28.128972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.128856+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:09.022827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'13","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.464629+0000","last_change":"2024-04-23T19:14:24.464629+0000","last_active":"2024-04-23T19:14:24.464629+0000","last_peered":"2024-04-23T19:14:24.464629+0000","last_clean":"2024-04-23T19:14:24.464629+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:14:24.464629+0000","last_undegraded":"2024-04-23T19:14:24.464629+0000","last_fullsized":"2024-04-23T19:14:24.464629+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'13","last_scrub_stamp":"2024-04-23T19:14:24.464510+0000","last_deep_scrub":"32'13","last_deep_scrub_stamp":"2024-04-23T19:14:24.464510+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.464510+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:40:14.783844+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.008630+0000","last_change":"2024-04-23T19:14:05.008630+0000","last_active":"2024-04-23T19:14:05.008630+0000","last_peered":"2024-04-23T19:14:05.008630+0000","last_clean":"2024-04-23T19:14:05.008630+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:14:05.008630+0000","last_undegraded":"2024-04-23T19:14:05.008630+0000","last_fullsized":"2024-04-23T19:14:05.008630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'20","last_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_deep_scrub":"32'20","last_deep_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.008496+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:11.308903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.997873+0000","last_change":"2024-04-23T19:14:16.997873+0000","last_active":"2024-04-23T19:14:16.997873+0000","last_peered":"2024-04-23T19:14:16.997873+0000","last_clean":"2024-04-23T19:14:16.997873+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:14:16.997873+0000","last_undegraded":"2024-04-23T19:14:16.997873+0000","last_fullsized":"2024-04-23T19:14:16.997873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:16.997750+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:16.997750+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.997750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:57:56.974216+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'5","reported_seq":429,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.912027+0000","last_change":"2024-04-23T19:13:37.912027+0000","last_active":"2024-04-23T19:13:37.912027+0000","last_peered":"2024-04-23T19:13:37.912027+0000","last_clean":"2024-04-23T19:13:37.912027+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:13:37.912027+0000","last_undegraded":"2024-04-23T19:13:37.912027+0000","last_fullsized":"2024-04-23T19:13:37.912027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.911892+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:23.295978+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"32'15","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.923039+0000","last_change":"2024-04-23T19:13:45.923039+0000","last_active":"2024-04-23T19:13:45.923039+0000","last_peered":"2024-04-23T19:13:45.923039+0000","last_clean":"2024-04-23T19:13:45.923039+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:13:45.923039+0000","last_undegraded":"2024-04-23T19:13:45.923039+0000","last_fullsized":"2024-04-23T19:13:45.923039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.922918+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:04.148244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'285","reported_seq":478,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.784184+0000","last_change":"2024-04-23T19:13:26.784184+0000","last_active":"2024-04-23T19:13:26.784184+0000","last_peered":"2024-04-23T19:13:26.784184+0000","last_clean":"2024-04-23T19:13:26.784184+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:26.784184+0000","last_undegraded":"2024-04-23T19:13:26.784184+0000","last_fullsized":"2024-04-23T19:13:26.784184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.784064+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:31.847408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.211854+0000","last_change":"2024-04-23T19:13:41.211854+0000","last_active":"2024-04-23T19:13:41.211854+0000","last_peered":"2024-04-23T19:13:41.211854+0000","last_clean":"2024-04-23T19:13:41.211854+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:13:41.211854+0000","last_undegraded":"2024-04-23T19:13:41.211854+0000","last_fullsized":"2024-04-23T19:13:41.211854+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.211740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.450923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.738341+0000","last_change":"2024-04-23T19:13:48.738341+0000","last_active":"2024-04-23T19:13:48.738341+0000","last_peered":"2024-04-23T19:13:48.738341+0000","last_clean":"2024-04-23T19:13:48.738341+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:13:48.738341+0000","last_undegraded":"2024-04-23T19:13:48.738341+0000","last_fullsized":"2024-04-23T19:13:48.738341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.738148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:12.860218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:33.837797+0000","last_change":"2024-04-23T19:14:33.837822+0000","last_active":"2024-04-23T19:14:33.837797+0000","last_peered":"2024-04-23T19:14:33.837797+0000","last_clean":"2024-04-23T19:14:33.837797+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:14:33.837797+0000","last_undegraded":"2024-04-23T19:14:33.837797+0000","last_fullsized":"2024-04-23T19:14:33.837797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:33.837696+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:33.837696+0000","last_clean_scrub_stamp":"2024-04-23T19:14:33.837696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:50.063335+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.111184+0000","last_change":"2024-04-23T19:13:28.111184+0000","last_active":"2024-04-23T19:13:28.111184+0000","last_peered":"2024-04-23T19:13:28.111184+0000","last_clean":"2024-04-23T19:13:28.111184+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:13:28.111184+0000","last_undegraded":"2024-04-23T19:13:28.111184+0000","last_fullsized":"2024-04-23T19:13:28.111184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.111056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:18.387489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'48","reported_seq":108,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.948796+0000","last_change":"2024-04-23T19:13:35.948796+0000","last_active":"2024-04-23T19:13:35.948796+0000","last_peered":"2024-04-23T19:13:35.948796+0000","last_clean":"2024-04-23T19:13:35.948796+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:13:35.948796+0000","last_undegraded":"2024-04-23T19:13:35.948796+0000","last_fullsized":"2024-04-23T19:13:35.948796+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'48","last_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_deep_scrub":"32'48","last_deep_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.948676+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:52.110221+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.116258+0000","last_change":"2024-04-23T19:13:30.116258+0000","last_active":"2024-04-23T19:13:30.116258+0000","last_peered":"2024-04-23T19:13:30.116258+0000","last_clean":"2024-04-23T19:13:30.116258+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:13:30.116258+0000","last_undegraded":"2024-04-23T19:13:30.116258+0000","last_fullsized":"2024-04-23T19:13:30.116258+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.116108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:58.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'7","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.725037+0000","last_change":"2024-04-23T19:13:46.725037+0000","last_active":"2024-04-23T19:13:46.725037+0000","last_peered":"2024-04-23T19:13:46.725037+0000","last_clean":"2024-04-23T19:13:46.725037+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:46.725037+0000","last_undegraded":"2024-04-23T19:13:46.725037+0000","last_fullsized":"2024-04-23T19:13:46.725037+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'7","last_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_deep_scrub":"32'7","last_deep_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.724881+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:17.683511+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.894509+0000","last_change":"2024-04-23T19:13:40.894509+0000","last_active":"2024-04-23T19:13:40.894509+0000","last_peered":"2024-04-23T19:13:40.894509+0000","last_clean":"2024-04-23T19:13:40.894509+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:13:40.894509+0000","last_undegraded":"2024-04-23T19:13:40.894509+0000","last_fullsized":"2024-04-23T19:13:40.894509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.894372+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:11.403485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"32'119","reported_seq":213,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.926283+0000","last_change":"2024-04-23T19:14:12.926325+0000","last_active":"2024-04-23T19:14:12.926283+0000","last_peered":"2024-04-23T19:14:12.926283+0000","last_clean":"2024-04-23T19:14:12.926283+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:14:12.926283+0000","last_undegraded":"2024-04-23T19:14:12.926283+0000","last_fullsized":"2024-04-23T19:14:12.926283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'119","last_scrub_stamp":"2024-04-23T19:14:12.926188+0000","last_deep_scrub":"32'119","last_deep_scrub_stamp":"2024-04-23T19:14:12.926188+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.926188+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:18.589862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:35.913325+0000","last_change":"2024-04-23T19:14:35.913349+0000","last_active":"2024-04-23T19:14:35.913325+0000","last_peered":"2024-04-23T19:14:35.913325+0000","last_clean":"2024-04-23T19:14:35.913325+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:14:35.913325+0000","last_undegraded":"2024-04-23T19:14:35.913325+0000","last_fullsized":"2024-04-23T19:14:35.913325+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:35.913237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:35.913237+0000","last_clean_scrub_stamp":"2024-04-23T19:14:35.913237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:07:30.809325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.987464+0000","last_change":"2024-04-23T19:14:15.987464+0000","last_active":"2024-04-23T19:14:15.987464+0000","last_peered":"2024-04-23T19:14:15.987464+0000","last_clean":"2024-04-23T19:14:15.987464+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:14:15.987464+0000","last_undegraded":"2024-04-23T19:14:15.987464+0000","last_fullsized":"2024-04-23T19:14:15.987464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:15.987336+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:15.987336+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.987336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:54.753507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.842231+0000","last_change":"2024-04-23T19:13:43.842231+0000","last_active":"2024-04-23T19:13:43.842231+0000","last_peered":"2024-04-23T19:13:43.842231+0000","last_clean":"2024-04-23T19:13:43.842231+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:13:43.842231+0000","last_undegraded":"2024-04-23T19:13:43.842231+0000","last_fullsized":"2024-04-23T19:13:43.842231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.842100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:04.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'88","reported_seq":179,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.876563+0000","last_change":"2024-04-23T19:13:37.876563+0000","last_active":"2024-04-23T19:13:37.876563+0000","last_peered":"2024-04-23T19:13:37.876563+0000","last_clean":"2024-04-23T19:13:37.876563+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:13:37.876563+0000","last_undegraded":"2024-04-23T19:13:37.876563+0000","last_fullsized":"2024-04-23T19:13:37.876563+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'88","last_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_deep_scrub":"32'88","last_deep_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.876439+0000","objects_scrubbed":5,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:33:54.212735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.621413+0000","last_change":"2024-04-23T19:13:29.621413+0000","last_active":"2024-04-23T19:13:29.621413+0000","last_peered":"2024-04-23T19:13:29.621413+0000","last_clean":"2024-04-23T19:13:29.621413+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:13:29.621413+0000","last_undegraded":"2024-04-23T19:13:29.621413+0000","last_fullsized":"2024-04-23T19:13:29.621413+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.621264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:03.242868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.101471+0000","last_change":"2024-04-23T19:13:49.101471+0000","last_active":"2024-04-23T19:13:49.101471+0000","last_peered":"2024-04-23T19:13:49.101471+0000","last_clean":"2024-04-23T19:13:49.101471+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:49.101471+0000","last_undegraded":"2024-04-23T19:13:49.101471+0000","last_fullsized":"2024-04-23T19:13:49.101471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'81","last_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_deep_scrub":"32'81","last_deep_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.101341+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:49.219878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.030892+0000","last_change":"2024-04-23T19:14:13.030892+0000","last_active":"2024-04-23T19:14:13.030892+0000","last_peered":"2024-04-23T19:14:13.030892+0000","last_clean":"2024-04-23T19:14:13.030892+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:14:13.030892+0000","last_undegraded":"2024-04-23T19:14:13.030892+0000","last_fullsized":"2024-04-23T19:14:13.030892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:13.030773+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:13.030773+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.030773+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:24.199482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:37.891862+0000","last_change":"2024-04-23T19:14:37.891887+0000","last_active":"2024-04-23T19:14:37.891862+0000","last_peered":"2024-04-23T19:14:37.891862+0000","last_clean":"2024-04-23T19:14:37.891862+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:14:37.891862+0000","last_undegraded":"2024-04-23T19:14:37.891862+0000","last_fullsized":"2024-04-23T19:14:37.891862+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:37.891758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:37.891758+0000","last_clean_scrub_stamp":"2024-04-23T19:14:37.891758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:05.733163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.550545+0000","last_change":"2024-04-23T19:14:16.550545+0000","last_active":"2024-04-23T19:14:16.550545+0000","last_peered":"2024-04-23T19:14:16.550545+0000","last_clean":"2024-04-23T19:14:16.550545+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:14:16.550545+0000","last_undegraded":"2024-04-23T19:14:16.550545+0000","last_fullsized":"2024-04-23T19:14:16.550545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:16.550416+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:16.550416+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.550416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:59.355007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.891613+0000","last_change":"2024-04-23T19:13:36.891613+0000","last_active":"2024-04-23T19:13:36.891613+0000","last_peered":"2024-04-23T19:13:36.891613+0000","last_clean":"2024-04-23T19:13:36.891613+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:13:36.891613+0000","last_undegraded":"2024-04-23T19:13:36.891613+0000","last_fullsized":"2024-04-23T19:13:36.891613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.891497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:43.191904+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'290","reported_seq":516,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.753118+0000","last_change":"2024-04-23T19:13:28.753118+0000","last_active":"2024-04-23T19:13:28.753118+0000","last_peered":"2024-04-23T19:13:28.753118+0000","last_clean":"2024-04-23T19:13:28.753118+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:28.753118+0000","last_undegraded":"2024-04-23T19:13:28.753118+0000","last_fullsized":"2024-04-23T19:13:28.753118+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'290","last_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_deep_scrub":"32'290","last_deep_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.752962+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:14.080605+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.883869+0000","last_change":"2024-04-23T19:13:53.883869+0000","last_active":"2024-04-23T19:13:53.883869+0000","last_peered":"2024-04-23T19:13:53.883869+0000","last_clean":"2024-04-23T19:13:53.883869+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:13:53.883869+0000","last_undegraded":"2024-04-23T19:13:53.883869+0000","last_fullsized":"2024-04-23T19:13:53.883869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.883735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:38.160620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.725710+0000","last_change":"2024-04-23T19:14:25.725710+0000","last_active":"2024-04-23T19:14:25.725710+0000","last_peered":"2024-04-23T19:14:25.725710+0000","last_clean":"2024-04-23T19:14:25.725710+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:14:25.725710+0000","last_undegraded":"2024-04-23T19:14:25.725710+0000","last_fullsized":"2024-04-23T19:14:25.725710+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:25.725582+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:25.725582+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.725582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:55.273423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.586425+0000","last_change":"2024-04-23T19:14:09.586425+0000","last_active":"2024-04-23T19:14:09.586425+0000","last_peered":"2024-04-23T19:14:09.586425+0000","last_clean":"2024-04-23T19:14:09.586425+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:14:09.586425+0000","last_undegraded":"2024-04-23T19:14:09.586425+0000","last_fullsized":"2024-04-23T19:14:09.586425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:09.586258+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:09.586258+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.586258+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:08.710830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.006366+0000","last_change":"2024-04-23T19:14:07.006366+0000","last_active":"2024-04-23T19:14:07.006366+0000","last_peered":"2024-04-23T19:14:07.006366+0000","last_clean":"2024-04-23T19:14:07.006366+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:14:07.006366+0000","last_undegraded":"2024-04-23T19:14:07.006366+0000","last_fullsized":"2024-04-23T19:14:07.006366+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.006243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.006243+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.006243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:46.529613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.504995+0000","last_change":"2024-04-23T19:13:58.504995+0000","last_active":"2024-04-23T19:13:58.504995+0000","last_peered":"2024-04-23T19:13:58.504995+0000","last_clean":"2024-04-23T19:13:58.504995+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:13:58.504995+0000","last_undegraded":"2024-04-23T19:13:58.504995+0000","last_fullsized":"2024-04-23T19:13:58.504995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.504872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:38.028243+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.171419+0000","last_change":"2024-04-23T19:14:22.171419+0000","last_active":"2024-04-23T19:14:22.171419+0000","last_peered":"2024-04-23T19:14:22.171419+0000","last_clean":"2024-04-23T19:14:22.171419+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:14:22.171419+0000","last_undegraded":"2024-04-23T19:14:22.171419+0000","last_fullsized":"2024-04-23T19:14:22.171419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:22.171266+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:22.171266+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.171266+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:06.448151+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'23","reported_seq":101,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.228773+0000","last_change":"2024-04-23T19:13:38.228773+0000","last_active":"2024-04-23T19:13:38.228773+0000","last_peered":"2024-04-23T19:13:38.228773+0000","last_clean":"2024-04-23T19:13:38.228773+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:38.228773+0000","last_undegraded":"2024-04-23T19:13:38.228773+0000","last_fullsized":"2024-04-23T19:13:38.228773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'23","last_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_deep_scrub":"32'23","last_deep_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.228582+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:14.715044+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.555830+0000","last_change":"2024-04-23T19:13:26.555830+0000","last_active":"2024-04-23T19:13:26.555830+0000","last_peered":"2024-04-23T19:13:26.555830+0000","last_clean":"2024-04-23T19:13:26.555830+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:13:26.555830+0000","last_undegraded":"2024-04-23T19:13:26.555830+0000","last_fullsized":"2024-04-23T19:13:26.555830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.555714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:11.883356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:44.940695+0000","last_change":"2024-04-23T19:14:44.940695+0000","last_active":"2024-04-23T19:14:44.940695+0000","last_peered":"2024-04-23T19:14:44.940695+0000","last_clean":"2024-04-23T19:14:44.940695+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:14:44.940695+0000","last_undegraded":"2024-04-23T19:14:44.940695+0000","last_fullsized":"2024-04-23T19:14:44.940695+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:44.940571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:44.940571+0000","last_clean_scrub_stamp":"2024-04-23T19:14:44.940571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:44:51.759106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.526311+0000","last_change":"2024-04-23T19:13:59.526311+0000","last_active":"2024-04-23T19:13:59.526311+0000","last_peered":"2024-04-23T19:13:59.526311+0000","last_clean":"2024-04-23T19:13:59.526311+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:59.526311+0000","last_undegraded":"2024-04-23T19:13:59.526311+0000","last_fullsized":"2024-04-23T19:13:59.526311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.526195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:39.758894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:36.875396+0000","last_change":"2024-04-23T19:14:36.875431+0000","last_active":"2024-04-23T19:14:36.875396+0000","last_peered":"2024-04-23T19:14:36.875396+0000","last_clean":"2024-04-23T19:14:36.875396+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:14:36.875396+0000","last_undegraded":"2024-04-23T19:14:36.875396+0000","last_fullsized":"2024-04-23T19:14:36.875396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:36.875297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:36.875297+0000","last_clean_scrub_stamp":"2024-04-23T19:14:36.875297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:40:55.108479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.981539+0000","last_change":"2024-04-23T19:14:07.981539+0000","last_active":"2024-04-23T19:14:07.981539+0000","last_peered":"2024-04-23T19:14:07.981539+0000","last_clean":"2024-04-23T19:14:07.981539+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:14:07.981539+0000","last_undegraded":"2024-04-23T19:14:07.981539+0000","last_fullsized":"2024-04-23T19:14:07.981539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.981428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.981428+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.981428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:48.214128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.501453+0000","last_change":"2024-04-23T19:13:49.501453+0000","last_active":"2024-04-23T19:13:49.501453+0000","last_peered":"2024-04-23T19:13:49.501453+0000","last_clean":"2024-04-23T19:13:49.501453+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:13:49.501453+0000","last_undegraded":"2024-04-23T19:13:49.501453+0000","last_fullsized":"2024-04-23T19:13:49.501453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.501336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:21.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'21","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.866575+0000","last_change":"2024-04-23T19:13:39.866575+0000","last_active":"2024-04-23T19:13:39.866575+0000","last_peered":"2024-04-23T19:13:39.866575+0000","last_clean":"2024-04-23T19:13:39.866575+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:39.866575+0000","last_undegraded":"2024-04-23T19:13:39.866575+0000","last_fullsized":"2024-04-23T19:13:39.866575+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.866421+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:20.099121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.919527+0000","last_change":"2024-04-23T19:13:47.919527+0000","last_active":"2024-04-23T19:13:47.919527+0000","last_peered":"2024-04-23T19:13:47.919527+0000","last_clean":"2024-04-23T19:13:47.919527+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:13:47.919527+0000","last_undegraded":"2024-04-23T19:13:47.919527+0000","last_fullsized":"2024-04-23T19:13:47.919527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.919412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:48.916350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.731452+0000","last_change":"2024-04-23T19:14:21.731452+0000","last_active":"2024-04-23T19:14:21.731452+0000","last_peered":"2024-04-23T19:14:21.731452+0000","last_clean":"2024-04-23T19:14:21.731452+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:14:21.731452+0000","last_undegraded":"2024-04-23T19:14:21.731452+0000","last_fullsized":"2024-04-23T19:14:21.731452+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:14:21.731320+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:14:21.731320+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.731320+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:12.108189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.694263+0000","last_change":"2024-04-23T19:13:47.694263+0000","last_active":"2024-04-23T19:13:47.694263+0000","last_peered":"2024-04-23T19:13:47.694263+0000","last_clean":"2024-04-23T19:13:47.694263+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:47.694263+0000","last_undegraded":"2024-04-23T19:13:47.694263+0000","last_fullsized":"2024-04-23T19:13:47.694263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.694146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:55.654671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.997935+0000","last_change":"2024-04-23T19:14:08.997935+0000","last_active":"2024-04-23T19:14:08.997935+0000","last_peered":"2024-04-23T19:14:08.997935+0000","last_clean":"2024-04-23T19:14:08.997935+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:14:08.997935+0000","last_undegraded":"2024-04-23T19:14:08.997935+0000","last_fullsized":"2024-04-23T19:14:08.997935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:08.997767+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:08.997767+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.997767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:39.506879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.514961+0000","last_change":"2024-04-23T19:14:13.514987+0000","last_active":"2024-04-23T19:14:13.514961+0000","last_peered":"2024-04-23T19:14:13.514961+0000","last_clean":"2024-04-23T19:14:13.514961+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:14:13.514961+0000","last_undegraded":"2024-04-23T19:14:13.514961+0000","last_fullsized":"2024-04-23T19:14:13.514961+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:13.514849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:13.514849+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.514849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:45.028155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'2","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.147860+0000","last_change":"2024-04-23T19:14:19.147888+0000","last_active":"2024-04-23T19:14:19.147860+0000","last_peered":"2024-04-23T19:14:19.147860+0000","last_clean":"2024-04-23T19:14:19.147860+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:14:19.147860+0000","last_undegraded":"2024-04-23T19:14:19.147860+0000","last_fullsized":"2024-04-23T19:14:19.147860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.147776+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.147776+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.147776+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:39.919289+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'75","reported_seq":152,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.702548+0000","last_change":"2024-04-23T19:13:34.702548+0000","last_active":"2024-04-23T19:13:34.702548+0000","last_peered":"2024-04-23T19:13:34.702548+0000","last_clean":"2024-04-23T19:13:34.702548+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:13:34.702548+0000","last_undegraded":"2024-04-23T19:13:34.702548+0000","last_fullsized":"2024-04-23T19:13:34.702548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'75","last_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_deep_scrub":"32'75","last_deep_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.702393+0000","objects_scrubbed":9,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:50.863938+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.162563+0000","last_change":"2024-04-23T19:13:31.162563+0000","last_active":"2024-04-23T19:13:31.162563+0000","last_peered":"2024-04-23T19:13:31.162563+0000","last_clean":"2024-04-23T19:13:31.162563+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:13:31.162563+0000","last_undegraded":"2024-04-23T19:13:31.162563+0000","last_fullsized":"2024-04-23T19:13:31.162563+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.162449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:47.422034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.538697+0000","last_change":"2024-04-23T19:13:55.538734+0000","last_active":"2024-04-23T19:13:55.538697+0000","last_peered":"2024-04-23T19:13:55.538697+0000","last_clean":"2024-04-23T19:13:55.538697+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:55.538697+0000","last_undegraded":"2024-04-23T19:13:55.538697+0000","last_fullsized":"2024-04-23T19:13:55.538697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.538607+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:42.408183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.742366+0000","last_change":"2024-04-23T19:13:51.742366+0000","last_active":"2024-04-23T19:13:51.742366+0000","last_peered":"2024-04-23T19:13:51.742366+0000","last_clean":"2024-04-23T19:13:51.742366+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:51.742366+0000","last_undegraded":"2024-04-23T19:13:51.742366+0000","last_fullsized":"2024-04-23T19:13:51.742366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.742223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:22.634791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.561186+0000","last_change":"2024-04-23T19:14:15.561186+0000","last_active":"2024-04-23T19:14:15.561186+0000","last_peered":"2024-04-23T19:14:15.561186+0000","last_clean":"2024-04-23T19:14:15.561186+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:14:15.561186+0000","last_undegraded":"2024-04-23T19:14:15.561186+0000","last_fullsized":"2024-04-23T19:14:15.561186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:15.561056+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:15.561056+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.561056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:42.489103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:38.890816+0000","last_change":"2024-04-23T19:14:38.890843+0000","last_active":"2024-04-23T19:14:38.890816+0000","last_peered":"2024-04-23T19:14:38.890816+0000","last_clean":"2024-04-23T19:14:38.890816+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:14:38.890816+0000","last_undegraded":"2024-04-23T19:14:38.890816+0000","last_fullsized":"2024-04-23T19:14:38.890816+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:38.890720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:38.890720+0000","last_clean_scrub_stamp":"2024-04-23T19:14:38.890720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:28.010171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.046354+0000","last_change":"2024-04-23T19:13:31.046354+0000","last_active":"2024-04-23T19:13:31.046354+0000","last_peered":"2024-04-23T19:13:31.046354+0000","last_clean":"2024-04-23T19:13:31.046354+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:13:31.046354+0000","last_undegraded":"2024-04-23T19:13:31.046354+0000","last_fullsized":"2024-04-23T19:13:31.046354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.046239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:01.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'22","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.762358+0000","last_change":"2024-04-23T19:13:29.762358+0000","last_active":"2024-04-23T19:13:29.762358+0000","last_peered":"2024-04-23T19:13:29.762358+0000","last_clean":"2024-04-23T19:13:29.762358+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:13:29.762358+0000","last_undegraded":"2024-04-23T19:13:29.762358+0000","last_fullsized":"2024-04-23T19:13:29.762358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'22","last_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_deep_scrub":"32'22","last_deep_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.762203+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:37.849358+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'285","reported_seq":368,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.782396+0000","last_change":"2024-04-23T19:13:30.782396+0000","last_active":"2024-04-23T19:13:30.782396+0000","last_peered":"2024-04-23T19:13:30.782396+0000","last_clean":"2024-04-23T19:13:30.782396+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:30.782396+0000","last_undegraded":"2024-04-23T19:13:30.782396+0000","last_fullsized":"2024-04-23T19:13:30.782396+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.782266+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:07.268727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.981194+0000","last_change":"2024-04-23T19:14:17.981194+0000","last_active":"2024-04-23T19:14:17.981194+0000","last_peered":"2024-04-23T19:14:17.981194+0000","last_clean":"2024-04-23T19:14:17.981194+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:14:17.981194+0000","last_undegraded":"2024-04-23T19:14:17.981194+0000","last_fullsized":"2024-04-23T19:14:17.981194+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:17.981076+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:17.981076+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.981076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:04:30.766806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.940021+0000","last_change":"2024-04-23T19:14:03.940021+0000","last_active":"2024-04-23T19:14:03.940021+0000","last_peered":"2024-04-23T19:14:03.940021+0000","last_clean":"2024-04-23T19:14:03.940021+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:14:03.940021+0000","last_undegraded":"2024-04-23T19:14:03.940021+0000","last_fullsized":"2024-04-23T19:14:03.940021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:03.939847+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:03.939847+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.939847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:54.259182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:34.864548+0000","last_change":"2024-04-23T19:14:34.864571+0000","last_active":"2024-04-23T19:14:34.864548+0000","last_peered":"2024-04-23T19:14:34.864548+0000","last_clean":"2024-04-23T19:14:34.864548+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:14:34.864548+0000","last_undegraded":"2024-04-23T19:14:34.864548+0000","last_fullsized":"2024-04-23T19:14:34.864548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:34.864459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:34.864459+0000","last_clean_scrub_stamp":"2024-04-23T19:14:34.864459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:20:21.122071+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.557659+0000","last_change":"2024-04-23T19:14:14.557659+0000","last_active":"2024-04-23T19:14:14.557659+0000","last_peered":"2024-04-23T19:14:14.557659+0000","last_clean":"2024-04-23T19:14:14.557659+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:14:14.557659+0000","last_undegraded":"2024-04-23T19:14:14.557659+0000","last_fullsized":"2024-04-23T19:14:14.557659+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.557539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.557539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.557539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:50:51.381912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.862700+0000","last_change":"2024-04-23T19:13:38.862700+0000","last_active":"2024-04-23T19:13:38.862700+0000","last_peered":"2024-04-23T19:13:38.862700+0000","last_clean":"2024-04-23T19:13:38.862700+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:13:38.862700+0000","last_undegraded":"2024-04-23T19:13:38.862700+0000","last_fullsized":"2024-04-23T19:13:38.862700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.862584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:18.385955+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"32'16","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.889045+0000","last_change":"2024-04-23T19:13:51.889045+0000","last_active":"2024-04-23T19:13:51.889045+0000","last_peered":"2024-04-23T19:13:51.889045+0000","last_clean":"2024-04-23T19:13:51.889045+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:13:51.889045+0000","last_undegraded":"2024-04-23T19:13:51.889045+0000","last_fullsized":"2024-04-23T19:13:51.889045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.888897+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:47:41.106501+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'285","reported_seq":442,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.522645+0000","last_change":"2024-04-23T19:13:47.522645+0000","last_active":"2024-04-23T19:13:47.522645+0000","last_peered":"2024-04-23T19:13:47.522645+0000","last_clean":"2024-04-23T19:13:47.522645+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:47.522645+0000","last_undegraded":"2024-04-23T19:13:47.522645+0000","last_fullsized":"2024-04-23T19:13:47.522645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.522518+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:37.798951+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.764934+0000","last_change":"2024-04-23T19:14:22.764934+0000","last_active":"2024-04-23T19:14:22.764934+0000","last_peered":"2024-04-23T19:14:22.764934+0000","last_clean":"2024-04-23T19:14:22.764934+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:14:22.764934+0000","last_undegraded":"2024-04-23T19:14:22.764934+0000","last_fullsized":"2024-04-23T19:14:22.764934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'3","last_scrub_stamp":"2024-04-23T19:14:22.764803+0000","last_deep_scrub":"30'3","last_deep_scrub_stamp":"2024-04-23T19:14:22.764803+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.764803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:24.439320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.083351+0000","last_change":"2024-04-23T19:13:50.083351+0000","last_active":"2024-04-23T19:13:50.083351+0000","last_peered":"2024-04-23T19:13:50.083351+0000","last_clean":"2024-04-23T19:13:50.083351+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:50.083351+0000","last_undegraded":"2024-04-23T19:13:50.083351+0000","last_fullsized":"2024-04-23T19:13:50.083351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.083230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:01.089369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.940410+0000","last_change":"2024-04-23T19:14:11.940410+0000","last_active":"2024-04-23T19:14:11.940410+0000","last_peered":"2024-04-23T19:14:11.940410+0000","last_clean":"2024-04-23T19:14:11.940410+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:14:11.940410+0000","last_undegraded":"2024-04-23T19:14:11.940410+0000","last_fullsized":"2024-04-23T19:14:11.940410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:11.940272+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:11.940272+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.940272+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:59.806805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:39.842479+0000","last_change":"2024-04-23T19:14:39.842558+0000","last_active":"2024-04-23T19:14:39.842479+0000","last_peered":"2024-04-23T19:14:39.842479+0000","last_clean":"2024-04-23T19:14:39.842479+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:14:39.842479+0000","last_undegraded":"2024-04-23T19:14:39.842479+0000","last_fullsized":"2024-04-23T19:14:39.842479+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:39.842354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:39.842354+0000","last_clean_scrub_stamp":"2024-04-23T19:14:39.842354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:26.500277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.119771+0000","last_change":"2024-04-23T19:13:29.119771+0000","last_active":"2024-04-23T19:13:29.119771+0000","last_peered":"2024-04-23T19:13:29.119771+0000","last_clean":"2024-04-23T19:13:29.119771+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:13:29.119771+0000","last_undegraded":"2024-04-23T19:13:29.119771+0000","last_fullsized":"2024-04-23T19:13:29.119771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.119655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:06.217082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'98","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.195253+0000","last_change":"2024-04-23T19:13:35.195253+0000","last_active":"2024-04-23T19:13:35.195253+0000","last_peered":"2024-04-23T19:13:35.195253+0000","last_clean":"2024-04-23T19:13:35.195253+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:13:35.195253+0000","last_undegraded":"2024-04-23T19:13:35.195253+0000","last_fullsized":"2024-04-23T19:13:35.195253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'98","last_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_deep_scrub":"32'98","last_deep_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.195120+0000","objects_scrubbed":8,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:11.071311+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4316,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":98,"num_read_kb":92,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.664532+0000","last_change":"2024-04-23T19:13:30.664532+0000","last_active":"2024-04-23T19:13:30.664532+0000","last_peered":"2024-04-23T19:13:30.664532+0000","last_clean":"2024-04-23T19:13:30.664532+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:13:30.664532+0000","last_undegraded":"2024-04-23T19:13:30.664532+0000","last_fullsized":"2024-04-23T19:13:30.664532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.664414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:40.569067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.161812+0000","last_change":"2024-04-23T19:13:45.161859+0000","last_active":"2024-04-23T19:13:45.161812+0000","last_peered":"2024-04-23T19:13:45.161812+0000","last_clean":"2024-04-23T19:13:45.161812+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:13:45.161812+0000","last_undegraded":"2024-04-23T19:13:45.161812+0000","last_fullsized":"2024-04-23T19:13:45.161812+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.161710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:30.545797+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.187444+0000","last_change":"2024-04-23T19:13:37.187444+0000","last_active":"2024-04-23T19:13:37.187444+0000","last_peered":"2024-04-23T19:13:37.187444+0000","last_clean":"2024-04-23T19:13:37.187444+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:13:37.187444+0000","last_undegraded":"2024-04-23T19:13:37.187444+0000","last_fullsized":"2024-04-23T19:13:37.187444+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.187313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.274468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.039233+0000","last_change":"2024-04-23T19:13:59.039257+0000","last_active":"2024-04-23T19:13:59.039233+0000","last_peered":"2024-04-23T19:13:59.039233+0000","last_clean":"2024-04-23T19:13:59.039233+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:13:59.039233+0000","last_undegraded":"2024-04-23T19:13:59.039233+0000","last_fullsized":"2024-04-23T19:13:59.039233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.039137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:29.931705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:28.905207+0000","last_change":"2024-04-23T19:14:28.905207+0000","last_active":"2024-04-23T19:14:28.905207+0000","last_peered":"2024-04-23T19:14:28.905207+0000","last_clean":"2024-04-23T19:14:28.905207+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:14:28.905207+0000","last_undegraded":"2024-04-23T19:14:28.905207+0000","last_fullsized":"2024-04-23T19:14:28.905207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:28.905078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:28.905078+0000","last_clean_scrub_stamp":"2024-04-23T19:14:28.905078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:10.007150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:26.688297+0000","last_change":"2024-04-23T19:14:26.688322+0000","last_active":"2024-04-23T19:14:26.688297+0000","last_peered":"2024-04-23T19:14:26.688297+0000","last_clean":"2024-04-23T19:14:26.688297+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:14:26.688297+0000","last_undegraded":"2024-04-23T19:14:26.688297+0000","last_fullsized":"2024-04-23T19:14:26.688297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:26.688210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:26.688210+0000","last_clean_scrub_stamp":"2024-04-23T19:14:26.688210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:32.612364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'39","reported_seq":95,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.033330+0000","last_change":"2024-04-23T19:13:32.033364+0000","last_active":"2024-04-23T19:13:32.033330+0000","last_peered":"2024-04-23T19:13:32.033330+0000","last_clean":"2024-04-23T19:13:32.033330+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:13:32.033330+0000","last_undegraded":"2024-04-23T19:13:32.033330+0000","last_fullsized":"2024-04-23T19:13:32.033330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.033246+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:16.932749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.897278+0000","last_change":"2024-04-23T19:13:57.897299+0000","last_active":"2024-04-23T19:13:57.897278+0000","last_peered":"2024-04-23T19:13:57.897278+0000","last_clean":"2024-04-23T19:13:57.897278+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:13:57.897278+0000","last_undegraded":"2024-04-23T19:13:57.897278+0000","last_fullsized":"2024-04-23T19:13:57.897278+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.897188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:37.553642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.482950+0000","last_change":"2024-04-23T19:14:23.482976+0000","last_active":"2024-04-23T19:14:23.482950+0000","last_peered":"2024-04-23T19:14:23.482950+0000","last_clean":"2024-04-23T19:14:23.482950+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:14:23.482950+0000","last_undegraded":"2024-04-23T19:14:23.482950+0000","last_fullsized":"2024-04-23T19:14:23.482950+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:14:23.482860+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:14:23.482860+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.482860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:32.816753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:27.658333+0000","last_change":"2024-04-23T19:14:27.658360+0000","last_active":"2024-04-23T19:14:27.658333+0000","last_peered":"2024-04-23T19:14:27.658333+0000","last_clean":"2024-04-23T19:14:27.658333+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:14:27.658333+0000","last_undegraded":"2024-04-23T19:14:27.658333+0000","last_fullsized":"2024-04-23T19:14:27.658333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:27.658252+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:27.658252+0000","last_clean_scrub_stamp":"2024-04-23T19:14:27.658252+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:22.800009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.058576+0000","last_change":"2024-04-23T19:14:02.058602+0000","last_active":"2024-04-23T19:14:02.058576+0000","last_peered":"2024-04-23T19:14:02.058576+0000","last_clean":"2024-04-23T19:14:02.058576+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:14:02.058576+0000","last_undegraded":"2024-04-23T19:14:02.058576+0000","last_fullsized":"2024-04-23T19:14:02.058576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.058488+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:35.270454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.891432+0000","last_change":"2024-04-23T19:13:35.891471+0000","last_active":"2024-04-23T19:13:35.891432+0000","last_peered":"2024-04-23T19:13:35.891432+0000","last_clean":"2024-04-23T19:13:35.891432+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:13:35.891432+0000","last_undegraded":"2024-04-23T19:13:35.891432+0000","last_fullsized":"2024-04-23T19:13:35.891432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.891324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:22.760900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'8","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.738794+0000","last_change":"2024-04-23T19:13:37.738834+0000","last_active":"2024-04-23T19:13:37.738794+0000","last_peered":"2024-04-23T19:13:37.738794+0000","last_clean":"2024-04-23T19:13:37.738794+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:37.738794+0000","last_undegraded":"2024-04-23T19:13:37.738794+0000","last_fullsized":"2024-04-23T19:13:37.738794+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.738703+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:59.027468+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.218628+0000","last_change":"2024-04-23T19:13:33.218654+0000","last_active":"2024-04-23T19:13:33.218628+0000","last_peered":"2024-04-23T19:13:33.218628+0000","last_clean":"2024-04-23T19:13:33.218628+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:13:33.218628+0000","last_undegraded":"2024-04-23T19:13:33.218628+0000","last_fullsized":"2024-04-23T19:13:33.218628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.218534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:31.120188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:27.395040+0000","last_change":"2024-04-23T19:14:27.395083+0000","last_active":"2024-04-23T19:14:27.395040+0000","last_peered":"2024-04-23T19:14:27.395040+0000","last_clean":"2024-04-23T19:14:27.395040+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:14:27.395040+0000","last_undegraded":"2024-04-23T19:14:27.395040+0000","last_fullsized":"2024-04-23T19:14:27.395040+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:27.394906+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:27.394906+0000","last_clean_scrub_stamp":"2024-04-23T19:14:27.394906+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:50.231738+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":69,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.581715+0000","last_change":"2024-04-23T19:13:53.581741+0000","last_active":"2024-04-23T19:13:53.581715+0000","last_peered":"2024-04-23T19:13:53.581715+0000","last_clean":"2024-04-23T19:13:53.581715+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:53.581715+0000","last_undegraded":"2024-04-23T19:13:53.581715+0000","last_fullsized":"2024-04-23T19:13:53.581715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.581624+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:26.793590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.067768+0000","last_change":"2024-04-23T19:13:56.067792+0000","last_active":"2024-04-23T19:13:56.067768+0000","last_peered":"2024-04-23T19:13:56.067768+0000","last_clean":"2024-04-23T19:13:56.067768+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:13:56.067768+0000","last_undegraded":"2024-04-23T19:13:56.067768+0000","last_fullsized":"2024-04-23T19:13:56.067768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.067677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:46.344110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.223615+0000","last_change":"2024-04-23T19:13:43.223641+0000","last_active":"2024-04-23T19:13:43.223615+0000","last_peered":"2024-04-23T19:13:43.223615+0000","last_clean":"2024-04-23T19:13:43.223615+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:13:43.223615+0000","last_undegraded":"2024-04-23T19:13:43.223615+0000","last_fullsized":"2024-04-23T19:13:43.223615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.223515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:59.385060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'47","reported_seq":119,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.709428+0000","last_change":"2024-04-23T19:13:40.709462+0000","last_active":"2024-04-23T19:13:40.709428+0000","last_peered":"2024-04-23T19:13:40.709428+0000","last_clean":"2024-04-23T19:13:40.709428+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:40.709428+0000","last_undegraded":"2024-04-23T19:13:40.709428+0000","last_fullsized":"2024-04-23T19:13:40.709428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'47","last_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_deep_scrub":"32'47","last_deep_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.709338+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:40.827875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.737111+0000","last_change":"2024-04-23T19:13:27.737136+0000","last_active":"2024-04-23T19:13:27.737111+0000","last_peered":"2024-04-23T19:13:27.737111+0000","last_clean":"2024-04-23T19:13:27.737111+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:13:27.737111+0000","last_undegraded":"2024-04-23T19:13:27.737111+0000","last_fullsized":"2024-04-23T19:13:27.737111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.737016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:03.613207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:46.723450+0000","last_change":"2024-04-23T19:10:17.521487+0000","last_active":"2024-04-23T19:12:46.723450+0000","last_peered":"2024-04-23T19:12:46.723450+0000","last_clean":"2024-04-23T19:12:46.723450+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:12:46.723450+0000","last_undegraded":"2024-04-23T19:12:46.723450+0000","last_fullsized":"2024-04-23T19:12:46.723450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'10","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.904017+0000","last_change":"2024-04-23T19:14:02.904055+0000","last_active":"2024-04-23T19:14:02.904017+0000","last_peered":"2024-04-23T19:14:02.904017+0000","last_clean":"2024-04-23T19:14:02.904017+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:14:02.904017+0000","last_undegraded":"2024-04-23T19:14:02.904017+0000","last_fullsized":"2024-04-23T19:14:02.904017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:02.903914+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:02.903914+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.903914+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:12.237807+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.755932+0000","last_change":"2024-04-23T19:14:18.755958+0000","last_active":"2024-04-23T19:14:18.755932+0000","last_peered":"2024-04-23T19:14:18.755932+0000","last_clean":"2024-04-23T19:14:18.755932+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:14:18.755932+0000","last_undegraded":"2024-04-23T19:14:18.755932+0000","last_fullsized":"2024-04-23T19:14:18.755932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.755843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.755843+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.755843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:31.351437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.092812+0000","last_change":"2024-04-23T19:13:30.092835+0000","last_active":"2024-04-23T19:13:30.092812+0000","last_peered":"2024-04-23T19:13:30.092812+0000","last_clean":"2024-04-23T19:13:30.092812+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:30.092812+0000","last_undegraded":"2024-04-23T19:13:30.092812+0000","last_fullsized":"2024-04-23T19:13:30.092812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.092707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:02.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'38","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.254489+0000","last_change":"2024-04-23T19:13:42.254523+0000","last_active":"2024-04-23T19:13:42.254489+0000","last_peered":"2024-04-23T19:13:42.254489+0000","last_clean":"2024-04-23T19:13:42.254489+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:13:42.254489+0000","last_undegraded":"2024-04-23T19:13:42.254489+0000","last_fullsized":"2024-04-23T19:13:42.254489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.254400+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:59.255554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":53,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.124596+0000","last_change":"2024-04-23T19:13:27.124622+0000","last_active":"2024-04-23T19:13:27.124596+0000","last_peered":"2024-04-23T19:13:27.124596+0000","last_clean":"2024-04-23T19:13:27.124596+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:13:27.124596+0000","last_undegraded":"2024-04-23T19:13:27.124596+0000","last_fullsized":"2024-04-23T19:13:27.124596+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.124488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:56.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.130082+0000","last_change":"2024-04-23T19:14:18.130109+0000","last_active":"2024-04-23T19:14:18.130082+0000","last_peered":"2024-04-23T19:14:18.130082+0000","last_clean":"2024-04-23T19:14:18.130082+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:14:18.130082+0000","last_undegraded":"2024-04-23T19:14:18.130082+0000","last_fullsized":"2024-04-23T19:14:18.130082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.129993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.129993+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.129993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:23.793668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.996158+0000","last_change":"2024-04-23T19:14:19.996211+0000","last_active":"2024-04-23T19:14:19.996158+0000","last_peered":"2024-04-23T19:14:19.996158+0000","last_clean":"2024-04-23T19:14:19.996158+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:14:19.996158+0000","last_undegraded":"2024-04-23T19:14:19.996158+0000","last_fullsized":"2024-04-23T19:14:19.996158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.996051+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.996051+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.996051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:19.442125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.511211+0000","last_change":"2024-04-23T19:13:48.511211+0000","last_active":"2024-04-23T19:13:48.511211+0000","last_peered":"2024-04-23T19:13:48.511211+0000","last_clean":"2024-04-23T19:13:48.511211+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:13:48.511211+0000","last_undegraded":"2024-04-23T19:13:48.511211+0000","last_fullsized":"2024-04-23T19:13:48.511211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.511103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:25.608530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.697721+0000","last_change":"2024-04-23T19:13:38.697755+0000","last_active":"2024-04-23T19:13:38.697721+0000","last_peered":"2024-04-23T19:13:38.697721+0000","last_clean":"2024-04-23T19:13:38.697721+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:13:38.697721+0000","last_undegraded":"2024-04-23T19:13:38.697721+0000","last_fullsized":"2024-04-23T19:13:38.697721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.697637+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:22.296607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.201793+0000","last_change":"2024-04-23T19:13:32.201821+0000","last_active":"2024-04-23T19:13:32.201793+0000","last_peered":"2024-04-23T19:13:32.201793+0000","last_clean":"2024-04-23T19:13:32.201793+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:13:32.201793+0000","last_undegraded":"2024-04-23T19:13:32.201793+0000","last_fullsized":"2024-04-23T19:13:32.201793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.201694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:43.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:28.376645+0000","last_change":"2024-04-23T19:14:28.376693+0000","last_active":"2024-04-23T19:14:28.376645+0000","last_peered":"2024-04-23T19:14:28.376645+0000","last_clean":"2024-04-23T19:14:28.376645+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:14:28.376645+0000","last_undegraded":"2024-04-23T19:14:28.376645+0000","last_fullsized":"2024-04-23T19:14:28.376645+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:14:28.376541+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:14:28.376541+0000","last_clean_scrub_stamp":"2024-04-23T19:14:28.376541+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:03:24.814925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.612354+0000","last_change":"2024-04-23T19:14:05.612354+0000","last_active":"2024-04-23T19:14:05.612354+0000","last_peered":"2024-04-23T19:14:05.612354+0000","last_clean":"2024-04-23T19:14:05.612354+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:14:05.612354+0000","last_undegraded":"2024-04-23T19:14:05.612354+0000","last_fullsized":"2024-04-23T19:14:05.612354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.612239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:04.922704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.782002+0000","last_change":"2024-04-23T19:14:17.782031+0000","last_active":"2024-04-23T19:14:17.782002+0000","last_peered":"2024-04-23T19:14:17.782002+0000","last_clean":"2024-04-23T19:14:17.782002+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:14:17.782002+0000","last_undegraded":"2024-04-23T19:14:17.782002+0000","last_fullsized":"2024-04-23T19:14:17.782002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:17.781913+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:17.781913+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.781913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:45.876367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.142214+0000","last_change":"2024-04-23T19:13:48.142214+0000","last_active":"2024-04-23T19:13:48.142214+0000","last_peered":"2024-04-23T19:13:48.142214+0000","last_clean":"2024-04-23T19:13:48.142214+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:13:48.142214+0000","last_undegraded":"2024-04-23T19:13:48.142214+0000","last_fullsized":"2024-04-23T19:13:48.142214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.142093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:17.068685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.927235+0000","last_change":"2024-04-23T19:13:54.927318+0000","last_active":"2024-04-23T19:13:54.927235+0000","last_peered":"2024-04-23T19:13:54.927235+0000","last_clean":"2024-04-23T19:13:54.927235+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:13:54.927235+0000","last_undegraded":"2024-04-23T19:13:54.927235+0000","last_fullsized":"2024-04-23T19:13:54.927235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.927131+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:23.021585+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.866434+0000","last_change":"2024-04-23T19:13:42.866499+0000","last_active":"2024-04-23T19:13:42.866434+0000","last_peered":"2024-04-23T19:13:42.866434+0000","last_clean":"2024-04-23T19:13:42.866434+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:13:42.866434+0000","last_undegraded":"2024-04-23T19:13:42.866434+0000","last_fullsized":"2024-04-23T19:13:42.866434+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.866306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.872444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:45.948204+0000","last_change":"2024-04-23T19:14:45.948264+0000","last_active":"2024-04-23T19:14:45.948204+0000","last_peered":"2024-04-23T19:14:45.948204+0000","last_clean":"2024-04-23T19:14:45.948204+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:14:45.948204+0000","last_undegraded":"2024-04-23T19:14:45.948204+0000","last_fullsized":"2024-04-23T19:14:45.948204+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:45.948104+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:45.948104+0000","last_clean_scrub_stamp":"2024-04-23T19:14:45.948104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:10.944728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.193141+0000","last_change":"2024-04-23T19:14:21.193203+0000","last_active":"2024-04-23T19:14:21.193141+0000","last_peered":"2024-04-23T19:14:21.193141+0000","last_clean":"2024-04-23T19:14:21.193141+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:14:21.193141+0000","last_undegraded":"2024-04-23T19:14:21.193141+0000","last_fullsized":"2024-04-23T19:14:21.193141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:21.193026+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:21.193026+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.193026+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:45.665300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.844740+0000","last_change":"2024-04-23T19:14:05.844813+0000","last_active":"2024-04-23T19:14:05.844740+0000","last_peered":"2024-04-23T19:14:05.844740+0000","last_clean":"2024-04-23T19:14:05.844740+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:14:05.844740+0000","last_undegraded":"2024-04-23T19:14:05.844740+0000","last_fullsized":"2024-04-23T19:14:05.844740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:05.844638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:05.844638+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.844638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:52:20.554775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.129709+0000","last_change":"2024-04-23T19:13:47.129736+0000","last_active":"2024-04-23T19:13:47.129709+0000","last_peered":"2024-04-23T19:13:47.129709+0000","last_clean":"2024-04-23T19:13:47.129709+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:13:47.129709+0000","last_undegraded":"2024-04-23T19:13:47.129709+0000","last_fullsized":"2024-04-23T19:13:47.129709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.129623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:16:50.873891+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'39","reported_seq":96,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.723412+0000","last_change":"2024-04-23T19:13:36.723412+0000","last_active":"2024-04-23T19:13:36.723412+0000","last_peered":"2024-04-23T19:13:36.723412+0000","last_clean":"2024-04-23T19:13:36.723412+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:13:36.723412+0000","last_undegraded":"2024-04-23T19:13:36.723412+0000","last_fullsized":"2024-04-23T19:13:36.723412+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.723239+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:22.107325+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.719943+0000","last_change":"2024-04-23T19:13:39.719943+0000","last_active":"2024-04-23T19:13:39.719943+0000","last_peered":"2024-04-23T19:13:39.719943+0000","last_clean":"2024-04-23T19:13:39.719943+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:13:39.719943+0000","last_undegraded":"2024-04-23T19:13:39.719943+0000","last_fullsized":"2024-04-23T19:13:39.719943+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:33.961424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.462496+0000","last_change":"2024-04-23T19:10:17.517918+0000","last_active":"2024-04-23T19:13:16.462496+0000","last_peered":"2024-04-23T19:13:16.462496+0000","last_clean":"2024-04-23T19:13:16.462496+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:13:16.462496+0000","last_undegraded":"2024-04-23T19:13:16.462496+0000","last_fullsized":"2024-04-23T19:13:16.462496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.526283+0000","last_change":"2024-04-23T19:13:57.526310+0000","last_active":"2024-04-23T19:13:57.526283+0000","last_peered":"2024-04-23T19:13:57.526283+0000","last_clean":"2024-04-23T19:13:57.526283+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:57.526283+0000","last_undegraded":"2024-04-23T19:13:57.526283+0000","last_fullsized":"2024-04-23T19:13:57.526283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.526187+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:13.862482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.930668+0000","last_change":"2024-04-23T19:14:21.930715+0000","last_active":"2024-04-23T19:14:21.930668+0000","last_peered":"2024-04-23T19:14:21.930668+0000","last_clean":"2024-04-23T19:14:21.930668+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:14:21.930668+0000","last_undegraded":"2024-04-23T19:14:21.930668+0000","last_fullsized":"2024-04-23T19:14:21.930668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:21.930521+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:21.930521+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.930521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:51.962515+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'5","reported_seq":238,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.794993+0000","last_change":"2024-04-23T19:13:44.795028+0000","last_active":"2024-04-23T19:13:44.794993+0000","last_peered":"2024-04-23T19:13:44.794993+0000","last_clean":"2024-04-23T19:13:44.794993+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:13:44.794993+0000","last_undegraded":"2024-04-23T19:13:44.794993+0000","last_fullsized":"2024-04-23T19:13:44.794993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.794905+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:45.493403+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"32'15","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.880831+0000","last_change":"2024-04-23T19:13:56.880831+0000","last_active":"2024-04-23T19:13:56.880831+0000","last_peered":"2024-04-23T19:13:56.880831+0000","last_clean":"2024-04-23T19:13:56.880831+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:13:56.880831+0000","last_undegraded":"2024-04-23T19:13:56.880831+0000","last_fullsized":"2024-04-23T19:13:56.880831+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.880679+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:26.722280+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'285","reported_seq":430,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.137953+0000","last_change":"2024-04-23T19:13:29.137953+0000","last_active":"2024-04-23T19:13:29.137953+0000","last_peered":"2024-04-23T19:13:29.137953+0000","last_clean":"2024-04-23T19:13:29.137953+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:29.137953+0000","last_undegraded":"2024-04-23T19:13:29.137953+0000","last_fullsized":"2024-04-23T19:13:29.137953+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.137819+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:17.728927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.985379+0000","last_change":"2024-04-23T19:14:13.985410+0000","last_active":"2024-04-23T19:14:13.985379+0000","last_peered":"2024-04-23T19:14:13.985379+0000","last_clean":"2024-04-23T19:14:13.985379+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:14:13.985379+0000","last_undegraded":"2024-04-23T19:14:13.985379+0000","last_fullsized":"2024-04-23T19:14:13.985379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:13.985289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:13.985289+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.985289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:36.480554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.032334+0000","last_change":"2024-04-23T19:13:53.032361+0000","last_active":"2024-04-23T19:13:53.032334+0000","last_peered":"2024-04-23T19:13:53.032334+0000","last_clean":"2024-04-23T19:13:53.032334+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:13:53.032334+0000","last_undegraded":"2024-04-23T19:13:53.032334+0000","last_fullsized":"2024-04-23T19:13:53.032334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.032245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:08.378799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.780980+0000","last_change":"2024-04-23T19:14:19.781007+0000","last_active":"2024-04-23T19:14:19.780980+0000","last_peered":"2024-04-23T19:14:19.780980+0000","last_clean":"2024-04-23T19:14:19.780980+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:14:19.780980+0000","last_undegraded":"2024-04-23T19:14:19.780980+0000","last_fullsized":"2024-04-23T19:14:19.780980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:19.780892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:19.780892+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.780892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:49.622493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.890930+0000","last_change":"2024-04-23T19:13:41.890978+0000","last_active":"2024-04-23T19:13:41.890930+0000","last_peered":"2024-04-23T19:13:41.890930+0000","last_clean":"2024-04-23T19:13:41.890930+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:13:41.890930+0000","last_undegraded":"2024-04-23T19:13:41.890930+0000","last_fullsized":"2024-04-23T19:13:41.890930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.890830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:42.009368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.806873+0000","last_change":"2024-04-23T19:13:31.806949+0000","last_active":"2024-04-23T19:13:31.806873+0000","last_peered":"2024-04-23T19:13:31.806873+0000","last_clean":"2024-04-23T19:13:31.806873+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:13:31.806873+0000","last_undegraded":"2024-04-23T19:13:31.806873+0000","last_fullsized":"2024-04-23T19:13:31.806873+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.806771+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:55.900270+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.655391+0000","last_change":"2024-04-23T19:13:31.655412+0000","last_active":"2024-04-23T19:13:31.655391+0000","last_peered":"2024-04-23T19:13:31.655391+0000","last_clean":"2024-04-23T19:13:31.655391+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:13:31.655391+0000","last_undegraded":"2024-04-23T19:13:31.655391+0000","last_fullsized":"2024-04-23T19:13:31.655391+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.655302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:46.303379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.793862+0000","last_change":"2024-04-23T19:14:23.793885+0000","last_active":"2024-04-23T19:14:23.793862+0000","last_peered":"2024-04-23T19:14:23.793862+0000","last_clean":"2024-04-23T19:14:23.793862+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:14:23.793862+0000","last_undegraded":"2024-04-23T19:14:23.793862+0000","last_fullsized":"2024-04-23T19:14:23.793862+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:23.793752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:23.793752+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.793752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:39.597437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.023793+0000","last_change":"2024-04-23T19:13:55.023820+0000","last_active":"2024-04-23T19:13:55.023793+0000","last_peered":"2024-04-23T19:13:55.023793+0000","last_clean":"2024-04-23T19:13:55.023793+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:55.023793+0000","last_undegraded":"2024-04-23T19:13:55.023793+0000","last_fullsized":"2024-04-23T19:13:55.023793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.023704+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:02.984229+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.970911+0000","last_change":"2024-04-23T19:14:20.970941+0000","last_active":"2024-04-23T19:14:20.970911+0000","last_peered":"2024-04-23T19:14:20.970911+0000","last_clean":"2024-04-23T19:14:20.970911+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:14:20.970911+0000","last_undegraded":"2024-04-23T19:14:20.970911+0000","last_fullsized":"2024-04-23T19:14:20.970911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:20.970802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:20.970802+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.970802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:24:34.128767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.149582+0000","last_change":"2024-04-23T19:13:46.149624+0000","last_active":"2024-04-23T19:13:46.149582+0000","last_peered":"2024-04-23T19:13:46.149582+0000","last_clean":"2024-04-23T19:13:46.149582+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:13:46.149582+0000","last_undegraded":"2024-04-23T19:13:46.149582+0000","last_fullsized":"2024-04-23T19:13:46.149582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.149461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:44.705262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'37","reported_seq":92,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.718061+0000","last_change":"2024-04-23T19:13:33.718094+0000","last_active":"2024-04-23T19:13:33.718061+0000","last_peered":"2024-04-23T19:13:33.718061+0000","last_clean":"2024-04-23T19:13:33.718061+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:13:33.718061+0000","last_undegraded":"2024-04-23T19:13:33.718061+0000","last_fullsized":"2024-04-23T19:13:33.718061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.717974+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:50.719128+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.950064+0000","last_change":"2024-04-23T19:14:01.950089+0000","last_active":"2024-04-23T19:14:01.950064+0000","last_peered":"2024-04-23T19:14:01.950064+0000","last_clean":"2024-04-23T19:14:01.950064+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:14:01.950064+0000","last_undegraded":"2024-04-23T19:14:01.950064+0000","last_fullsized":"2024-04-23T19:14:01.950064+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:01.949971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:01.949971+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.949971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:05.620169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722640+0000","last_change":"2024-04-23T19:10:17.521082+0000","last_active":"2024-04-23T19:12:41.722640+0000","last_peered":"2024-04-23T19:12:41.722640+0000","last_clean":"2024-04-23T19:12:41.722640+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:12:41.722640+0000","last_undegraded":"2024-04-23T19:12:41.722640+0000","last_fullsized":"2024-04-23T19:12:41.722640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'12","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.000633+0000","last_change":"2024-04-23T19:14:09.000667+0000","last_active":"2024-04-23T19:14:09.000633+0000","last_peered":"2024-04-23T19:14:09.000633+0000","last_clean":"2024-04-23T19:14:09.000633+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:14:09.000633+0000","last_undegraded":"2024-04-23T19:14:09.000633+0000","last_fullsized":"2024-04-23T19:14:09.000633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:09.000539+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:09.000539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.000539+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:32.366969+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.545266+0000","last_change":"2024-04-23T19:14:18.545295+0000","last_active":"2024-04-23T19:14:18.545266+0000","last_peered":"2024-04-23T19:14:18.545266+0000","last_clean":"2024-04-23T19:14:18.545266+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:14:18.545266+0000","last_undegraded":"2024-04-23T19:14:18.545266+0000","last_fullsized":"2024-04-23T19:14:18.545266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.545177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.545177+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.545177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:58.201631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.929784+0000","last_change":"2024-04-23T19:13:36.929808+0000","last_active":"2024-04-23T19:13:36.929784+0000","last_peered":"2024-04-23T19:13:36.929784+0000","last_clean":"2024-04-23T19:13:36.929784+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:13:36.929784+0000","last_undegraded":"2024-04-23T19:13:36.929784+0000","last_fullsized":"2024-04-23T19:13:36.929784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.929690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:45.019264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'30","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.925641+0000","last_change":"2024-04-23T19:13:52.925676+0000","last_active":"2024-04-23T19:13:52.925641+0000","last_peered":"2024-04-23T19:13:52.925641+0000","last_clean":"2024-04-23T19:13:52.925641+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:13:52.925641+0000","last_undegraded":"2024-04-23T19:13:52.925641+0000","last_fullsized":"2024-04-23T19:13:52.925641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'30","last_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_deep_scrub":"32'30","last_deep_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.925542+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.397817+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.909679+0000","last_change":"2024-04-23T19:13:49.909700+0000","last_active":"2024-04-23T19:13:49.909679+0000","last_peered":"2024-04-23T19:13:49.909679+0000","last_clean":"2024-04-23T19:13:49.909679+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:13:49.909679+0000","last_undegraded":"2024-04-23T19:13:49.909679+0000","last_fullsized":"2024-04-23T19:13:49.909679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.909590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:01:35.495993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:12:41.722581+0000","last_change":"2024-04-23T19:10:17.521472+0000","last_active":"2024-04-23T19:12:41.722581+0000","last_peered":"2024-04-23T19:12:41.722581+0000","last_clean":"2024-04-23T19:12:41.722581+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:12:41.722581+0000","last_undegraded":"2024-04-23T19:12:41.722581+0000","last_fullsized":"2024-04-23T19:12:41.722581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.048809+0000","last_change":"2024-04-23T19:13:52.048855+0000","last_active":"2024-04-23T19:13:52.048809+0000","last_peered":"2024-04-23T19:13:52.048809+0000","last_clean":"2024-04-23T19:13:52.048809+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:52.048809+0000","last_undegraded":"2024-04-23T19:13:52.048809+0000","last_fullsized":"2024-04-23T19:13:52.048809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.048718+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:52.747216+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.880060+0000","last_change":"2024-04-23T19:14:09.880060+0000","last_active":"2024-04-23T19:14:09.880060+0000","last_peered":"2024-04-23T19:14:09.880060+0000","last_clean":"2024-04-23T19:14:09.880060+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:14:09.880060+0000","last_undegraded":"2024-04-23T19:14:09.880060+0000","last_fullsized":"2024-04-23T19:14:09.880060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:09.879942+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:09.879942+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.879942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.402393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'3","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.954199+0000","last_change":"2024-04-23T19:13:39.954265+0000","last_active":"2024-04-23T19:13:39.954199+0000","last_peered":"2024-04-23T19:13:39.954199+0000","last_clean":"2024-04-23T19:13:39.954199+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:39.954199+0000","last_undegraded":"2024-04-23T19:13:39.954199+0000","last_fullsized":"2024-04-23T19:13:39.954199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.954105+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:23.553074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'40","reported_seq":97,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.877187+0000","last_change":"2024-04-23T19:13:55.877248+0000","last_active":"2024-04-23T19:13:55.877187+0000","last_peered":"2024-04-23T19:13:55.877187+0000","last_clean":"2024-04-23T19:13:55.877187+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:55.877187+0000","last_undegraded":"2024-04-23T19:13:55.877187+0000","last_fullsized":"2024-04-23T19:13:55.877187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'40","last_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_deep_scrub":"32'40","last_deep_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.877078+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:08.472673+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.549277+0000","last_change":"2024-04-23T19:13:27.549295+0000","last_active":"2024-04-23T19:13:27.549277+0000","last_peered":"2024-04-23T19:13:27.549277+0000","last_clean":"2024-04-23T19:13:27.549277+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:13:27.549277+0000","last_undegraded":"2024-04-23T19:13:27.549277+0000","last_fullsized":"2024-04-23T19:13:27.549277+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.549205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:33.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:16.486600+0000","last_change":"2024-04-23T19:10:17.521274+0000","last_active":"2024-04-23T19:13:16.486600+0000","last_peered":"2024-04-23T19:13:16.486600+0000","last_clean":"2024-04-23T19:13:16.486600+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:13:16.486600+0000","last_undegraded":"2024-04-23T19:13:16.486600+0000","last_fullsized":"2024-04-23T19:13:16.486600+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:10:16.488489+0000","last_clean_scrub_stamp":"2024-04-23T19:10:16.488489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.719401+0000","last_change":"2024-04-23T19:13:49.719401+0000","last_active":"2024-04-23T19:13:49.719401+0000","last_peered":"2024-04-23T19:13:49.719401+0000","last_clean":"2024-04-23T19:13:49.719401+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:49.719401+0000","last_undegraded":"2024-04-23T19:13:49.719401+0000","last_fullsized":"2024-04-23T19:13:49.719401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.719206+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:11:27.945878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.054867+0000","last_change":"2024-04-23T19:14:06.054867+0000","last_active":"2024-04-23T19:14:06.054867+0000","last_peered":"2024-04-23T19:14:06.054867+0000","last_clean":"2024-04-23T19:14:06.054867+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:14:06.054867+0000","last_undegraded":"2024-04-23T19:14:06.054867+0000","last_fullsized":"2024-04-23T19:14:06.054867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:06.054734+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:06.054734+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.054734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:22.391030+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.541792+0000","last_change":"2024-04-23T19:14:03.541792+0000","last_active":"2024-04-23T19:14:03.541792+0000","last_peered":"2024-04-23T19:14:03.541792+0000","last_clean":"2024-04-23T19:14:03.541792+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:14:03.541792+0000","last_undegraded":"2024-04-23T19:14:03.541792+0000","last_fullsized":"2024-04-23T19:14:03.541792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.541669+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:29.952719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.533755+0000","last_change":"2024-04-23T19:14:20.533755+0000","last_active":"2024-04-23T19:14:20.533755+0000","last_peered":"2024-04-23T19:14:20.533755+0000","last_clean":"2024-04-23T19:14:20.533755+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:14:20.533755+0000","last_undegraded":"2024-04-23T19:14:20.533755+0000","last_fullsized":"2024-04-23T19:14:20.533755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:20.533629+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:20.533629+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.533629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:58.132753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.033589+0000","last_change":"2024-04-23T19:14:08.033589+0000","last_active":"2024-04-23T19:14:08.033589+0000","last_peered":"2024-04-23T19:14:08.033589+0000","last_clean":"2024-04-23T19:14:08.033589+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:14:08.033589+0000","last_undegraded":"2024-04-23T19:14:08.033589+0000","last_fullsized":"2024-04-23T19:14:08.033589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:08.033479+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:08.033479+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.033479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:30:11.119403+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:27.912078+0000","last_change":"2024-04-23T19:14:27.912102+0000","last_active":"2024-04-23T19:14:27.912078+0000","last_peered":"2024-04-23T19:14:27.912078+0000","last_clean":"2024-04-23T19:14:27.912078+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:14:27.912078+0000","last_undegraded":"2024-04-23T19:14:27.912078+0000","last_fullsized":"2024-04-23T19:14:27.912078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:27.911989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:27.911989+0000","last_clean_scrub_stamp":"2024-04-23T19:14:27.911989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:34:04.938123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.509140+0000","last_change":"2024-04-23T19:14:01.509140+0000","last_active":"2024-04-23T19:14:01.509140+0000","last_peered":"2024-04-23T19:14:01.509140+0000","last_clean":"2024-04-23T19:14:01.509140+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:14:01.509140+0000","last_undegraded":"2024-04-23T19:14:01.509140+0000","last_fullsized":"2024-04-23T19:14:01.509140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.508998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:10.244783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.590245+0000","last_change":"2024-04-23T19:14:08.590245+0000","last_active":"2024-04-23T19:14:08.590245+0000","last_peered":"2024-04-23T19:14:08.590245+0000","last_clean":"2024-04-23T19:14:08.590245+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:14:08.590245+0000","last_undegraded":"2024-04-23T19:14:08.590245+0000","last_fullsized":"2024-04-23T19:14:08.590245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:08.590099+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:08.590099+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.590099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:56.456633+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.554231+0000","last_change":"2024-04-23T19:13:54.554231+0000","last_active":"2024-04-23T19:13:54.554231+0000","last_peered":"2024-04-23T19:13:54.554231+0000","last_clean":"2024-04-23T19:13:54.554231+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:13:54.554231+0000","last_undegraded":"2024-04-23T19:13:54.554231+0000","last_fullsized":"2024-04-23T19:13:54.554231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.554115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:38.772240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.518605+0000","last_change":"2024-04-23T19:14:00.518605+0000","last_active":"2024-04-23T19:14:00.518605+0000","last_peered":"2024-04-23T19:14:00.518605+0000","last_clean":"2024-04-23T19:14:00.518605+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:14:00.518605+0000","last_undegraded":"2024-04-23T19:14:00.518605+0000","last_fullsized":"2024-04-23T19:14:00.518605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.518487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:31.027600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.766254+0000","last_change":"2024-04-23T19:13:45.766254+0000","last_active":"2024-04-23T19:13:45.766254+0000","last_peered":"2024-04-23T19:13:45.766254+0000","last_clean":"2024-04-23T19:13:45.766254+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:45.766254+0000","last_undegraded":"2024-04-23T19:13:45.766254+0000","last_fullsized":"2024-04-23T19:13:45.766254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.766137+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:33.256585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.886328+0000","last_change":"2024-04-23T19:14:13.886328+0000","last_active":"2024-04-23T19:14:13.886328+0000","last_peered":"2024-04-23T19:14:13.886328+0000","last_clean":"2024-04-23T19:14:13.886328+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:14:13.886328+0000","last_undegraded":"2024-04-23T19:14:13.886328+0000","last_fullsized":"2024-04-23T19:14:13.886328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:13.886177+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:13.886177+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.886177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:34.657691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.558742+0000","last_change":"2024-04-23T19:14:07.558742+0000","last_active":"2024-04-23T19:14:07.558742+0000","last_peered":"2024-04-23T19:14:07.558742+0000","last_clean":"2024-04-23T19:14:07.558742+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:14:07.558742+0000","last_undegraded":"2024-04-23T19:14:07.558742+0000","last_fullsized":"2024-04-23T19:14:07.558742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:07.558617+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:07.558617+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.558617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:08.555554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.079291+0000","last_change":"2024-04-23T19:13:58.079291+0000","last_active":"2024-04-23T19:13:58.079291+0000","last_peered":"2024-04-23T19:13:58.079291+0000","last_clean":"2024-04-23T19:13:58.079291+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:58.079291+0000","last_undegraded":"2024-04-23T19:13:58.079291+0000","last_fullsized":"2024-04-23T19:13:58.079291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.079130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:35.830705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.883280+0000","last_change":"2024-04-23T19:14:08.883280+0000","last_active":"2024-04-23T19:14:08.883280+0000","last_peered":"2024-04-23T19:14:08.883280+0000","last_clean":"2024-04-23T19:14:08.883280+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:14:08.883280+0000","last_undegraded":"2024-04-23T19:14:08.883280+0000","last_fullsized":"2024-04-23T19:14:08.883280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:08.883162+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:08.883162+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.883162+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:27.108487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.554519+0000","last_change":"2024-04-23T19:14:02.554519+0000","last_active":"2024-04-23T19:14:02.554519+0000","last_peered":"2024-04-23T19:14:02.554519+0000","last_clean":"2024-04-23T19:14:02.554519+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:14:02.554519+0000","last_undegraded":"2024-04-23T19:14:02.554519+0000","last_fullsized":"2024-04-23T19:14:02.554519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.554376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:52.658050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.844100+0000","last_change":"2024-04-23T19:14:04.844128+0000","last_active":"2024-04-23T19:14:04.844100+0000","last_peered":"2024-04-23T19:14:04.844100+0000","last_clean":"2024-04-23T19:14:04.844100+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:14:04.844100+0000","last_undegraded":"2024-04-23T19:14:04.844100+0000","last_fullsized":"2024-04-23T19:14:04.844100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:04.844020+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:04.844020+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.844020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:21:48.661589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:29.951532+0000","last_change":"2024-04-23T19:14:29.951560+0000","last_active":"2024-04-23T19:14:29.951532+0000","last_peered":"2024-04-23T19:14:29.951532+0000","last_clean":"2024-04-23T19:14:29.951532+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:14:29.951532+0000","last_undegraded":"2024-04-23T19:14:29.951532+0000","last_fullsized":"2024-04-23T19:14:29.951532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:29.951431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:29.951431+0000","last_clean_scrub_stamp":"2024-04-23T19:14:29.951431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:51.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.936668+0000","last_change":"2024-04-23T19:14:00.936694+0000","last_active":"2024-04-23T19:14:00.936668+0000","last_peered":"2024-04-23T19:14:00.936668+0000","last_clean":"2024-04-23T19:14:00.936668+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:14:00.936668+0000","last_undegraded":"2024-04-23T19:14:00.936668+0000","last_fullsized":"2024-04-23T19:14:00.936668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.936576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:19.351629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.563709+0000","last_change":"2024-04-23T19:14:12.563709+0000","last_active":"2024-04-23T19:14:12.563709+0000","last_peered":"2024-04-23T19:14:12.563709+0000","last_clean":"2024-04-23T19:14:12.563709+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:14:12.563709+0000","last_undegraded":"2024-04-23T19:14:12.563709+0000","last_fullsized":"2024-04-23T19:14:12.563709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:12.563575+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:12.563575+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.563575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:45.234515+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'6","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.538459+0000","last_change":"2024-04-23T19:13:52.538459+0000","last_active":"2024-04-23T19:13:52.538459+0000","last_peered":"2024-04-23T19:13:52.538459+0000","last_clean":"2024-04-23T19:13:52.538459+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:13:52.538459+0000","last_undegraded":"2024-04-23T19:13:52.538459+0000","last_fullsized":"2024-04-23T19:13:52.538459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.538343+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:34.602802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.859491+0000","last_change":"2024-04-23T19:14:14.859491+0000","last_active":"2024-04-23T19:14:14.859491+0000","last_peered":"2024-04-23T19:14:14.859491+0000","last_clean":"2024-04-23T19:14:14.859491+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:14:14.859491+0000","last_undegraded":"2024-04-23T19:14:14.859491+0000","last_fullsized":"2024-04-23T19:14:14.859491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.859367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.859367+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.859367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:04.076971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.865265+0000","last_change":"2024-04-23T19:13:59.865265+0000","last_active":"2024-04-23T19:13:59.865265+0000","last_peered":"2024-04-23T19:13:59.865265+0000","last_clean":"2024-04-23T19:13:59.865265+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:13:59.865265+0000","last_undegraded":"2024-04-23T19:13:59.865265+0000","last_fullsized":"2024-04-23T19:13:59.865265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.865099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:18:21.512041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.889509+0000","last_change":"2024-04-23T19:14:03.889509+0000","last_active":"2024-04-23T19:14:03.889509+0000","last_peered":"2024-04-23T19:14:03.889509+0000","last_clean":"2024-04-23T19:14:03.889509+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:14:03.889509+0000","last_undegraded":"2024-04-23T19:14:03.889509+0000","last_fullsized":"2024-04-23T19:14:03.889509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:03.889376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:03.889376+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.889376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.241474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.731942+0000","last_change":"2024-04-23T19:13:52.731942+0000","last_active":"2024-04-23T19:13:52.731942+0000","last_peered":"2024-04-23T19:13:52.731942+0000","last_clean":"2024-04-23T19:13:52.731942+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:13:52.731942+0000","last_undegraded":"2024-04-23T19:13:52.731942+0000","last_fullsized":"2024-04-23T19:13:52.731942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.731777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:37.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:41.850777+0000","last_change":"2024-04-23T19:14:41.850851+0000","last_active":"2024-04-23T19:14:41.850777+0000","last_peered":"2024-04-23T19:14:41.850777+0000","last_clean":"2024-04-23T19:14:41.850777+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:14:41.850777+0000","last_undegraded":"2024-04-23T19:14:41.850777+0000","last_fullsized":"2024-04-23T19:14:41.850777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:41.850671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:41.850671+0000","last_clean_scrub_stamp":"2024-04-23T19:14:41.850671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:10.257673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.863186+0000","last_change":"2024-04-23T19:14:02.863186+0000","last_active":"2024-04-23T19:14:02.863186+0000","last_peered":"2024-04-23T19:14:02.863186+0000","last_clean":"2024-04-23T19:14:02.863186+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:14:02.863186+0000","last_undegraded":"2024-04-23T19:14:02.863186+0000","last_fullsized":"2024-04-23T19:14:02.863186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:02.863017+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:02.863017+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.863017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:33.372130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.951020+0000","last_change":"2024-04-23T19:14:22.951020+0000","last_active":"2024-04-23T19:14:22.951020+0000","last_peered":"2024-04-23T19:14:22.951020+0000","last_clean":"2024-04-23T19:14:22.951020+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:14:22.951020+0000","last_undegraded":"2024-04-23T19:14:22.951020+0000","last_fullsized":"2024-04-23T19:14:22.951020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:22.950894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:22.950894+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.950894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:42.201782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'6","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.983645+0000","last_change":"2024-04-23T19:14:06.983645+0000","last_active":"2024-04-23T19:14:06.983645+0000","last_peered":"2024-04-23T19:14:06.983645+0000","last_clean":"2024-04-23T19:14:06.983645+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:14:06.983645+0000","last_undegraded":"2024-04-23T19:14:06.983645+0000","last_fullsized":"2024-04-23T19:14:06.983645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:06.983514+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:06.983514+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.983514+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:29.820347+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.057731+0000","last_change":"2024-04-23T19:14:00.057782+0000","last_active":"2024-04-23T19:14:00.057731+0000","last_peered":"2024-04-23T19:14:00.057731+0000","last_clean":"2024-04-23T19:14:00.057731+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:14:00.057731+0000","last_undegraded":"2024-04-23T19:14:00.057731+0000","last_fullsized":"2024-04-23T19:14:00.057731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.057618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:34.694433+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.779346+0000","last_change":"2024-04-23T19:14:20.779346+0000","last_active":"2024-04-23T19:14:20.779346+0000","last_peered":"2024-04-23T19:14:20.779346+0000","last_clean":"2024-04-23T19:14:20.779346+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:14:20.779346+0000","last_undegraded":"2024-04-23T19:14:20.779346+0000","last_fullsized":"2024-04-23T19:14:20.779346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:20.779226+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:20.779226+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.779226+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:00.435680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:43.914958+0000","last_change":"2024-04-23T19:14:43.914958+0000","last_active":"2024-04-23T19:14:43.914958+0000","last_peered":"2024-04-23T19:14:43.914958+0000","last_clean":"2024-04-23T19:14:43.914958+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:14:43.914958+0000","last_undegraded":"2024-04-23T19:14:43.914958+0000","last_fullsized":"2024-04-23T19:14:43.914958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:43.914835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:43.914835+0000","last_clean_scrub_stamp":"2024-04-23T19:14:43.914835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:46:43.212848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.879645+0000","last_change":"2024-04-23T19:14:24.879645+0000","last_active":"2024-04-23T19:14:24.879645+0000","last_peered":"2024-04-23T19:14:24.879645+0000","last_clean":"2024-04-23T19:14:24.879645+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:14:24.879645+0000","last_undegraded":"2024-04-23T19:14:24.879645+0000","last_fullsized":"2024-04-23T19:14:24.879645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:24.879513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:24.879513+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.879513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:13:57.197889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.585443+0000","last_change":"2024-04-23T19:14:11.585491+0000","last_active":"2024-04-23T19:14:11.585443+0000","last_peered":"2024-04-23T19:14:11.585443+0000","last_clean":"2024-04-23T19:14:11.585443+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:14:11.585443+0000","last_undegraded":"2024-04-23T19:14:11.585443+0000","last_fullsized":"2024-04-23T19:14:11.585443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:11.585343+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:11.585343+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.585343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:00.802947+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.561399+0000","last_change":"2024-04-23T19:14:17.561430+0000","last_active":"2024-04-23T19:14:17.561399+0000","last_peered":"2024-04-23T19:14:17.561399+0000","last_clean":"2024-04-23T19:14:17.561399+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:14:17.561399+0000","last_undegraded":"2024-04-23T19:14:17.561399+0000","last_fullsized":"2024-04-23T19:14:17.561399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:17.561302+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:17.561302+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.561302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:47.402903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":62,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.753730+0000","last_change":"2024-04-23T19:13:53.753760+0000","last_active":"2024-04-23T19:13:53.753730+0000","last_peered":"2024-04-23T19:13:53.753730+0000","last_clean":"2024-04-23T19:13:53.753730+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:53.753730+0000","last_undegraded":"2024-04-23T19:13:53.753730+0000","last_fullsized":"2024-04-23T19:13:53.753730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.753615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:13.659941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1524,"ondisk_log_size":1524,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":481,"ondisk_log_size":481,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673024,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39948,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331892,"statfs":{"total":96636764160,"available":96595857408,"internally_reserved":0,"allocated":462848,"data_stored":144617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673023,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1024000,"data_stored":734097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40468,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331372,"statfs":{"total":96636764160,"available":96595324928,"internally_reserved":0,"allocated":995328,"data_stored":732924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705727,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":425984,"data_stored":144396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:14:49.296 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:49.297 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:49.297 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:49.297 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:49.297 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-23T19:10:16.488489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=10, tm_sec=16, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=19, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T19:14:49.298 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T19:15:09.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T19:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.411+0000 7f67e814a640 1 Processor -- start 2024-04-23T19:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.411+0000 7f67e814a640 1 -- start start 2024-04-23T19:15:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.411+0000 7f67e814a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e0093420 0x7f67e0093800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:15:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.411+0000 7f67e814a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e0096620 con 0x7f67e012b150 2024-04-23T19:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.411+0000 7f67e814a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f67e006d2b0 con 0x7f67e0093420 2024-04-23T19:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.411+0000 7f67e5ee6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e0093420 0x7f67e0093800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.411+0000 7f67e56e5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f67e012b150 0x7f67e012b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37024/0 (socket says 172.21.15.16:37024) 2024-04-23T19:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.412+0000 7f67e56e5640 1 -- 172.21.15.16:0/2795211096 learned_addr learned my addr 172.21.15.16:0/2795211096 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.412+0000 7f67e5ee6640 1 -- 172.21.15.16:0/2795211096 >> v1:172.21.15.16:6789/0 conn(0x7f67e012b150 legacy=0x7f67e012b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.412+0000 7f67e5ee6640 1 -- 172.21.15.16:0/2795211096 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e0072e50 con 0x7f67e0093420 2024-04-23T19:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.412+0000 7f67e4ee4640 1 -- 172.21.15.16:0/2795211096 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224662628 0 0) 0x7f67e0096620 con 0x7f67e012b150 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e5ee6640 1 --2- 172.21.15.16:0/2795211096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0093800 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009720 tx=0x7f67dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ac5c4a850f2cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e4ee4640 1 -- 172.21.15.16:0/2795211096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67dc0316d0 con 0x7f67e0093420 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e4ee4640 1 -- 172.21.15.16:0/2795211096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67dc037070 con 0x7f67e0093420 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e4ee4640 1 -- 172.21.15.16:0/2795211096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67dc0319c0 con 0x7f67e0093420 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e814a640 1 -- 172.21.15.16:0/2795211096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 msgr2=0x7f67e0093800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e814a640 1 --2- 172.21.15.16:0/2795211096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0093800 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009720 tx=0x7f67dc02b840 comp rx=0 tx=0).stop 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e814a640 1 -- 172.21.15.16:0/2795211096 shutdown_connections 2024-04-23T19:15:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e814a640 1 --2- 172.21.15.16:0/2795211096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0093800 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e814a640 1 -- 172.21.15.16:0/2795211096 >> 172.21.15.16:0/2795211096 conn(0x7f67e009a200 msgr2=0x7f67e0097780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.413+0000 7f67e814a640 1 -- 172.21.15.16:0/2795211096 shutdown_connections 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.414+0000 7f67e814a640 1 -- 172.21.15.16:0/2795211096 wait complete. 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.414+0000 7f67e814a640 1 Processor -- start 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.414+0000 7f67e814a640 1 -- start start 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e814a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0132c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e814a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67e0093d40 con 0x7f67e0093420 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e5ee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0132c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e5ee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0132c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39370/0 (socket says 172.21.15.16:39370) 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e5ee6640 1 -- 172.21.15.16:0/3139904819 learned_addr learned my addr 172.21.15.16:0/3139904819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e5ee6640 1 -- 172.21.15.16:0/3139904819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e006eb10 con 0x7f67e0093420 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e5ee6640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0132c70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f67dc02b880 tx=0x7f67dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:15:09.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67d67fc640 1 -- 172.21.15.16:0/3139904819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67dc004030 con 0x7f67e0093420 2024-04-23T19:15:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67d67fc640 1 -- 172.21.15.16:0/3139904819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67dc0041d0 con 0x7f67e0093420 2024-04-23T19:15:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e006e550 con 0x7f67e0093420 2024-04-23T19:15:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67d67fc640 1 -- 172.21.15.16:0/3139904819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67dc032640 con 0x7f67e0093420 2024-04-23T19:15:09.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.415+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e006dd40 con 0x7f67e0093420 2024-04-23T19:15:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.416+0000 7f67d67fc640 1 -- 172.21.15.16:0/3139904819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f67dc0327e0 con 0x7f67e0093420 2024-04-23T19:15:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.416+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e0063660 con 0x7f67e0093420 2024-04-23T19:15:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.417+0000 7f67d67fc640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f67c40385c0 0x7f67c403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:15:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.417+0000 7f67d67fc640 1 -- 172.21.15.16:0/3139904819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f67dc02f050 con 0x7f67e0093420 2024-04-23T19:15:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.417+0000 7f67e56e5640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f67c40385c0 0x7f67c403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:15:09.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.417+0000 7f67e56e5640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f67c40385c0 0x7f67c403aa80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000c00 tx=0x7f67d0007910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:15:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.421+0000 7f67d67fc640 1 -- 172.21.15.16:0/3139904819 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67dc041070 con 0x7f67e0093420 2024-04-23T19:15:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.605+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 --> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67e0067f20 con 0x7f67c40385c0 2024-04-23T19:15:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.621+0000 7f67d67fc640 1 -- 172.21.15.16:0/3139904819 <== mgr.4102 v2:172.21.15.63:6824/4252129218 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+666027 (secure 0 0 0) 0x7f67e0067f20 con 0x7f67c40385c0 2024-04-23T19:15:09.630 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-23T19:15:09.635 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-23T19:15:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f67c40385c0 msgr2=0x7f67c403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f67c40385c0 0x7f67c403aa80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000c00 tx=0x7f67d0007910 comp rx=0 tx=0).stop 2024-04-23T19:15:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 msgr2=0x7f67e0132c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0132c70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f67dc02b880 tx=0x7f67dc0046c0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 shutdown_connections 2024-04-23T19:15:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f67c40385c0 0x7f67c403aa80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 --2- 172.21.15.16:0/3139904819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e0093420 0x7f67e0132c70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 >> 172.21.15.16:0/3139904819 conn(0x7f67e009a200 msgr2=0x7f67e0094a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:15:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 shutdown_connections 2024-04-23T19:15:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.625+0000 7f67e814a640 1 -- 172.21.15.16:0/3139904819 wait complete. 2024-04-23T19:15:09.639 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-04-23T19:15:08.736092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5092,"num_read_kb":4673,"num_write":2177,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4783,"ondisk_log_size":4783,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":152504,"kb_used_data":2840,"kb_used_omap":6,"kb_used_meta":149625,"kb_avail":377334856,"statfs":{"total":386547056640,"available":386390892544,"internally_reserved":0,"allocated":2908160,"data_stored":1756034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":153216791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004023"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.042420+0000","last_change":"2024-04-23T19:14:14.042420+0000","last_active":"2024-04-23T19:14:14.042420+0000","last_peered":"2024-04-23T19:14:14.042420+0000","last_clean":"2024-04-23T19:14:14.042420+0000","last_became_active":"2024-04-23T19:10:07.492369+0000","last_became_peered":"2024-04-23T19:10:07.492369+0000","last_unstale":"2024-04-23T19:14:14.042420+0000","last_undegraded":"2024-04-23T19:14:14.042420+0000","last_fullsized":"2024-04-23T19:14:14.042420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.042287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.042287+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.042287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:13.352752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"32'4","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.897072+0000","last_change":"2024-04-23T19:14:10.897102+0000","last_active":"2024-04-23T19:14:10.897072+0000","last_peered":"2024-04-23T19:14:10.897072+0000","last_clean":"2024-04-23T19:14:10.897072+0000","last_became_active":"2024-04-23T19:10:09.485043+0000","last_became_peered":"2024-04-23T19:10:09.485043+0000","last_unstale":"2024-04-23T19:14:10.897072+0000","last_undegraded":"2024-04-23T19:14:10.897072+0000","last_fullsized":"2024-04-23T19:14:10.897072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:10.896986+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:10.896986+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.896986+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:27.637541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.564794+0000","last_change":"2024-04-23T19:14:04.564794+0000","last_active":"2024-04-23T19:14:04.564794+0000","last_peered":"2024-04-23T19:14:04.564794+0000","last_clean":"2024-04-23T19:14:04.564794+0000","last_became_active":"2024-04-23T19:10:07.489396+0000","last_became_peered":"2024-04-23T19:10:07.489396+0000","last_unstale":"2024-04-23T19:14:04.564794+0000","last_undegraded":"2024-04-23T19:14:04.564794+0000","last_fullsized":"2024-04-23T19:14:04.564794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.564668+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.564668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:53.911600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"32'8","reported_seq":74,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.937584+0000","last_change":"2024-04-23T19:13:59.937609+0000","last_active":"2024-04-23T19:13:59.937584+0000","last_peered":"2024-04-23T19:13:59.937584+0000","last_clean":"2024-04-23T19:13:59.937584+0000","last_became_active":"2024-04-23T19:10:09.497942+0000","last_became_peered":"2024-04-23T19:10:09.497942+0000","last_unstale":"2024-04-23T19:13:59.937584+0000","last_undegraded":"2024-04-23T19:13:59.937584+0000","last_fullsized":"2024-04-23T19:13:59.937584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.937477+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.937477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:01.611995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.987105+0000","last_change":"2024-04-23T19:14:18.987130+0000","last_active":"2024-04-23T19:14:18.987105+0000","last_peered":"2024-04-23T19:14:18.987105+0000","last_clean":"2024-04-23T19:14:18.987105+0000","last_became_active":"2024-04-23T19:10:07.507964+0000","last_became_peered":"2024-04-23T19:10:07.507964+0000","last_unstale":"2024-04-23T19:14:18.987105+0000","last_undegraded":"2024-04-23T19:14:18.987105+0000","last_fullsized":"2024-04-23T19:14:18.987105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.986988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.986988+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.986988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:58.819070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"32'4","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.946559+0000","last_change":"2024-04-23T19:13:46.946583+0000","last_active":"2024-04-23T19:13:46.946559+0000","last_peered":"2024-04-23T19:13:46.946559+0000","last_clean":"2024-04-23T19:13:46.946559+0000","last_became_active":"2024-04-23T19:10:09.498685+0000","last_became_peered":"2024-04-23T19:10:09.498685+0000","last_unstale":"2024-04-23T19:13:46.946559+0000","last_undegraded":"2024-04-23T19:13:46.946559+0000","last_fullsized":"2024-04-23T19:13:46.946559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:46.946470+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.946470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:19.468921+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.055999+0000","last_change":"2024-04-23T19:14:11.056028+0000","last_active":"2024-04-23T19:14:11.055999+0000","last_peered":"2024-04-23T19:14:11.055999+0000","last_clean":"2024-04-23T19:14:11.055999+0000","last_became_active":"2024-04-23T19:10:07.490474+0000","last_became_peered":"2024-04-23T19:10:07.490474+0000","last_unstale":"2024-04-23T19:14:11.055999+0000","last_undegraded":"2024-04-23T19:14:11.055999+0000","last_fullsized":"2024-04-23T19:14:11.055999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:11.055907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:11.055907+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.055907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:44:01.159581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"32'12","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.985264+0000","last_change":"2024-04-23T19:14:11.985294+0000","last_active":"2024-04-23T19:14:11.985264+0000","last_peered":"2024-04-23T19:14:11.985264+0000","last_clean":"2024-04-23T19:14:11.985264+0000","last_became_active":"2024-04-23T19:10:09.489271+0000","last_became_peered":"2024-04-23T19:10:09.489271+0000","last_unstale":"2024-04-23T19:14:11.985264+0000","last_undegraded":"2024-04-23T19:14:11.985264+0000","last_fullsized":"2024-04-23T19:14:11.985264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:11.985174+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:11.985174+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.985174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:26.660339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":34,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.878199+0000","last_change":"2024-04-23T19:14:25.878199+0000","last_active":"2024-04-23T19:14:25.878199+0000","last_peered":"2024-04-23T19:14:25.878199+0000","last_clean":"2024-04-23T19:14:25.878199+0000","last_became_active":"2024-04-23T19:10:07.501634+0000","last_became_peered":"2024-04-23T19:10:07.501634+0000","last_unstale":"2024-04-23T19:14:25.878199+0000","last_undegraded":"2024-04-23T19:14:25.878199+0000","last_fullsized":"2024-04-23T19:14:25.878199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:25.878068+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:25.878068+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.878068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:36:20.382452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"32'4","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.789955+0000","last_change":"2024-04-23T19:13:57.789955+0000","last_active":"2024-04-23T19:13:57.789955+0000","last_peered":"2024-04-23T19:13:57.789955+0000","last_clean":"2024-04-23T19:13:57.789955+0000","last_became_active":"2024-04-23T19:10:09.482015+0000","last_became_peered":"2024-04-23T19:10:09.482015+0000","last_unstale":"2024-04-23T19:13:57.789955+0000","last_undegraded":"2024-04-23T19:13:57.789955+0000","last_fullsized":"2024-04-23T19:13:57.789955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:57.789839+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.789839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:44.659415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.007123+0000","last_change":"2024-04-23T19:14:12.007150+0000","last_active":"2024-04-23T19:14:12.007123+0000","last_peered":"2024-04-23T19:14:12.007123+0000","last_clean":"2024-04-23T19:14:12.007123+0000","last_became_active":"2024-04-23T19:10:07.490559+0000","last_became_peered":"2024-04-23T19:10:07.490559+0000","last_unstale":"2024-04-23T19:14:12.007123+0000","last_undegraded":"2024-04-23T19:14:12.007123+0000","last_fullsized":"2024-04-23T19:14:12.007123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:12.007017+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:12.007017+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.007017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:38.418068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"32'14","reported_seq":86,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.905886+0000","last_change":"2024-04-23T19:13:44.905886+0000","last_active":"2024-04-23T19:13:44.905886+0000","last_peered":"2024-04-23T19:13:44.905886+0000","last_clean":"2024-04-23T19:13:44.905886+0000","last_became_active":"2024-04-23T19:10:09.498217+0000","last_became_peered":"2024-04-23T19:10:09.498217+0000","last_unstale":"2024-04-23T19:13:44.905886+0000","last_undegraded":"2024-04-23T19:13:44.905886+0000","last_fullsized":"2024-04-23T19:13:44.905886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:44.905747+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.905747+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.252680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":55,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:40.840161+0000","last_change":"2024-04-23T19:14:40.840161+0000","last_active":"2024-04-23T19:14:40.840161+0000","last_peered":"2024-04-23T19:14:40.840161+0000","last_clean":"2024-04-23T19:14:40.840161+0000","last_became_active":"2024-04-23T19:10:07.488983+0000","last_became_peered":"2024-04-23T19:10:07.488983+0000","last_unstale":"2024-04-23T19:14:40.840161+0000","last_undegraded":"2024-04-23T19:14:40.840161+0000","last_fullsized":"2024-04-23T19:14:40.840161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:40.840047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:40.840047+0000","last_clean_scrub_stamp":"2024-04-23T19:14:40.840047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:35:20.185487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"32'24","reported_seq":100,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.035838+0000","last_change":"2024-04-23T19:13:57.035838+0000","last_active":"2024-04-23T19:13:57.035838+0000","last_peered":"2024-04-23T19:13:57.035838+0000","last_clean":"2024-04-23T19:13:57.035838+0000","last_became_active":"2024-04-23T19:10:09.490907+0000","last_became_peered":"2024-04-23T19:10:09.490907+0000","last_unstale":"2024-04-23T19:13:57.035838+0000","last_undegraded":"2024-04-23T19:13:57.035838+0000","last_fullsized":"2024-04-23T19:13:57.035838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'24","last_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_deep_scrub":"32'24","last_deep_scrub_stamp":"2024-04-23T19:13:57.035696+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.035696+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:34.133123+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":25,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"32'2","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.567190+0000","last_change":"2024-04-23T19:14:19.567220+0000","last_active":"2024-04-23T19:14:19.567190+0000","last_peered":"2024-04-23T19:14:19.567190+0000","last_clean":"2024-04-23T19:14:19.567190+0000","last_became_active":"2024-04-23T19:10:07.486713+0000","last_became_peered":"2024-04-23T19:10:07.486713+0000","last_unstale":"2024-04-23T19:14:19.567190+0000","last_undegraded":"2024-04-23T19:14:19.567190+0000","last_fullsized":"2024-04-23T19:14:19.567190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.567093+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.567093+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.567093+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.943962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"32'18","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.074121+0000","last_change":"2024-04-23T19:14:15.074121+0000","last_active":"2024-04-23T19:14:15.074121+0000","last_peered":"2024-04-23T19:14:15.074121+0000","last_clean":"2024-04-23T19:14:15.074121+0000","last_became_active":"2024-04-23T19:10:09.493321+0000","last_became_peered":"2024-04-23T19:10:09.493321+0000","last_unstale":"2024-04-23T19:14:15.074121+0000","last_undegraded":"2024-04-23T19:14:15.074121+0000","last_fullsized":"2024-04-23T19:14:15.074121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'18","last_scrub_stamp":"2024-04-23T19:14:15.073956+0000","last_deep_scrub":"32'18","last_deep_scrub_stamp":"2024-04-23T19:14:15.073956+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.073956+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:47.596407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.943623+0000","last_change":"2024-04-23T19:13:43.943623+0000","last_active":"2024-04-23T19:13:43.943623+0000","last_peered":"2024-04-23T19:13:43.943623+0000","last_clean":"2024-04-23T19:13:43.943623+0000","last_became_active":"2024-04-23T19:10:09.499042+0000","last_became_peered":"2024-04-23T19:10:09.499042+0000","last_unstale":"2024-04-23T19:13:43.943623+0000","last_undegraded":"2024-04-23T19:13:43.943623+0000","last_fullsized":"2024-04-23T19:13:43.943623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.943496+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.943496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:10.354547+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.824733+0000","last_change":"2024-04-23T19:14:15.824733+0000","last_active":"2024-04-23T19:14:15.824733+0000","last_peered":"2024-04-23T19:14:15.824733+0000","last_clean":"2024-04-23T19:14:15.824733+0000","last_became_active":"2024-04-23T19:10:07.491582+0000","last_became_peered":"2024-04-23T19:10:07.491582+0000","last_unstale":"2024-04-23T19:14:15.824733+0000","last_undegraded":"2024-04-23T19:14:15.824733+0000","last_fullsized":"2024-04-23T19:14:15.824733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:15.824569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:15.824569+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.824569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:40.296843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"32'12","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.911780+0000","last_change":"2024-04-23T19:14:00.911780+0000","last_active":"2024-04-23T19:14:00.911780+0000","last_peered":"2024-04-23T19:14:00.911780+0000","last_clean":"2024-04-23T19:14:00.911780+0000","last_became_active":"2024-04-23T19:10:09.482347+0000","last_became_peered":"2024-04-23T19:10:09.482347+0000","last_unstale":"2024-04-23T19:14:00.911780+0000","last_undegraded":"2024-04-23T19:14:00.911780+0000","last_fullsized":"2024-04-23T19:14:00.911780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:00.911657+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.911657+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:48.206563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.028439+0000","last_change":"2024-04-23T19:14:10.028439+0000","last_active":"2024-04-23T19:14:10.028439+0000","last_peered":"2024-04-23T19:14:10.028439+0000","last_clean":"2024-04-23T19:14:10.028439+0000","last_became_active":"2024-04-23T19:10:07.489548+0000","last_became_peered":"2024-04-23T19:10:07.489548+0000","last_unstale":"2024-04-23T19:14:10.028439+0000","last_undegraded":"2024-04-23T19:14:10.028439+0000","last_fullsized":"2024-04-23T19:14:10.028439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:10.028304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:10.028304+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.028304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:18.764089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.839443+0000","last_change":"2024-04-23T19:14:06.839443+0000","last_active":"2024-04-23T19:14:06.839443+0000","last_peered":"2024-04-23T19:14:06.839443+0000","last_clean":"2024-04-23T19:14:06.839443+0000","last_became_active":"2024-04-23T19:10:09.484614+0000","last_became_peered":"2024-04-23T19:10:09.484614+0000","last_unstale":"2024-04-23T19:14:06.839443+0000","last_undegraded":"2024-04-23T19:14:06.839443+0000","last_fullsized":"2024-04-23T19:14:06.839443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:06.839307+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:06.839307+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.839307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:24.085334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.796535+0000","last_change":"2024-04-23T19:14:16.796535+0000","last_active":"2024-04-23T19:14:16.796535+0000","last_peered":"2024-04-23T19:14:16.796535+0000","last_clean":"2024-04-23T19:14:16.796535+0000","last_became_active":"2024-04-23T19:10:07.493226+0000","last_became_peered":"2024-04-23T19:10:07.493226+0000","last_unstale":"2024-04-23T19:14:16.796535+0000","last_undegraded":"2024-04-23T19:14:16.796535+0000","last_fullsized":"2024-04-23T19:14:16.796535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:16.796417+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:16.796417+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.796417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:01.073302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.526589+0000","last_change":"2024-04-23T19:13:56.526589+0000","last_active":"2024-04-23T19:13:56.526589+0000","last_peered":"2024-04-23T19:13:56.526589+0000","last_clean":"2024-04-23T19:13:56.526589+0000","last_became_active":"2024-04-23T19:10:09.490901+0000","last_became_peered":"2024-04-23T19:10:09.490901+0000","last_unstale":"2024-04-23T19:13:56.526589+0000","last_undegraded":"2024-04-23T19:13:56.526589+0000","last_fullsized":"2024-04-23T19:13:56.526589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:56.526450+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.526450+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:02.826857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.855627+0000","last_change":"2024-04-23T19:14:07.855627+0000","last_active":"2024-04-23T19:14:07.855627+0000","last_peered":"2024-04-23T19:14:07.855627+0000","last_clean":"2024-04-23T19:14:07.855627+0000","last_became_active":"2024-04-23T19:10:07.500753+0000","last_became_peered":"2024-04-23T19:10:07.500753+0000","last_unstale":"2024-04-23T19:14:07.855627+0000","last_undegraded":"2024-04-23T19:14:07.855627+0000","last_fullsized":"2024-04-23T19:14:07.855627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.855478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.855478+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.855478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:53:54.172855+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.044887+0000","last_change":"2024-04-23T19:14:03.044887+0000","last_active":"2024-04-23T19:14:03.044887+0000","last_peered":"2024-04-23T19:14:03.044887+0000","last_clean":"2024-04-23T19:14:03.044887+0000","last_became_active":"2024-04-23T19:10:09.492628+0000","last_became_peered":"2024-04-23T19:10:09.492628+0000","last_unstale":"2024-04-23T19:14:03.044887+0000","last_undegraded":"2024-04-23T19:14:03.044887+0000","last_fullsized":"2024-04-23T19:14:03.044887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:03.044766+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.044766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:47.262891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"30'2","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.578937+0000","last_change":"2024-04-23T19:14:06.578937+0000","last_active":"2024-04-23T19:14:06.578937+0000","last_peered":"2024-04-23T19:14:06.578937+0000","last_clean":"2024-04-23T19:14:06.578937+0000","last_became_active":"2024-04-23T19:10:07.490076+0000","last_became_peered":"2024-04-23T19:10:07.490076+0000","last_unstale":"2024-04-23T19:14:06.578937+0000","last_undegraded":"2024-04-23T19:14:06.578937+0000","last_fullsized":"2024-04-23T19:14:06.578937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:06.578801+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:06.578801+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.578801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:39.101252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.816160+0000","last_change":"2024-04-23T19:13:56.816160+0000","last_active":"2024-04-23T19:13:56.816160+0000","last_peered":"2024-04-23T19:13:56.816160+0000","last_clean":"2024-04-23T19:13:56.816160+0000","last_became_active":"2024-04-23T19:10:09.482189+0000","last_became_peered":"2024-04-23T19:10:09.482189+0000","last_unstale":"2024-04-23T19:13:56.816160+0000","last_undegraded":"2024-04-23T19:13:56.816160+0000","last_fullsized":"2024-04-23T19:13:56.816160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:56.816040+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.816040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:41.034165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:32.834694+0000","last_change":"2024-04-23T19:14:32.834694+0000","last_active":"2024-04-23T19:14:32.834694+0000","last_peered":"2024-04-23T19:14:32.834694+0000","last_clean":"2024-04-23T19:14:32.834694+0000","last_became_active":"2024-04-23T19:10:07.493638+0000","last_became_peered":"2024-04-23T19:10:07.493638+0000","last_unstale":"2024-04-23T19:14:32.834694+0000","last_undegraded":"2024-04-23T19:14:32.834694+0000","last_fullsized":"2024-04-23T19:14:32.834694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:32.834539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:32.834539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:32.834539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:30.099684+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"32'36","reported_seq":110,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.799967+0000","last_change":"2024-04-23T19:13:54.799967+0000","last_active":"2024-04-23T19:13:54.799967+0000","last_peered":"2024-04-23T19:13:54.799967+0000","last_clean":"2024-04-23T19:13:54.799967+0000","last_became_active":"2024-04-23T19:10:17.514200+0000","last_became_peered":"2024-04-23T19:10:17.514200+0000","last_unstale":"2024-04-23T19:13:54.799967+0000","last_undegraded":"2024-04-23T19:13:54.799967+0000","last_fullsized":"2024-04-23T19:13:54.799967+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'36","last_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_deep_scrub":"32'36","last_deep_scrub_stamp":"2024-04-23T19:13:54.799848+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.799848+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:43.632765+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.014637+0000","last_change":"2024-04-23T19:14:15.014637+0000","last_active":"2024-04-23T19:14:15.014637+0000","last_peered":"2024-04-23T19:14:15.014637+0000","last_clean":"2024-04-23T19:14:15.014637+0000","last_became_active":"2024-04-23T19:10:09.487161+0000","last_became_peered":"2024-04-23T19:10:09.487161+0000","last_unstale":"2024-04-23T19:14:15.014637+0000","last_undegraded":"2024-04-23T19:14:15.014637+0000","last_fullsized":"2024-04-23T19:14:15.014637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:15.014507+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:15.014507+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.014507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:49.779222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:31.876322+0000","last_change":"2024-04-23T19:14:31.876322+0000","last_active":"2024-04-23T19:14:31.876322+0000","last_peered":"2024-04-23T19:14:31.876322+0000","last_clean":"2024-04-23T19:14:31.876322+0000","last_became_active":"2024-04-23T19:10:07.507015+0000","last_became_peered":"2024-04-23T19:10:07.507015+0000","last_unstale":"2024-04-23T19:14:31.876322+0000","last_undegraded":"2024-04-23T19:14:31.876322+0000","last_fullsized":"2024-04-23T19:14:31.876322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:31.876162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:31.876162+0000","last_clean_scrub_stamp":"2024-04-23T19:14:31.876162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:46:12.190852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.986571+0000","last_change":"2024-04-23T19:13:32.986571+0000","last_active":"2024-04-23T19:13:32.986571+0000","last_peered":"2024-04-23T19:13:32.986571+0000","last_clean":"2024-04-23T19:13:32.986571+0000","last_became_active":"2024-04-23T19:10:11.488033+0000","last_became_peered":"2024-04-23T19:10:11.488033+0000","last_unstale":"2024-04-23T19:13:32.986571+0000","last_undegraded":"2024-04-23T19:13:32.986571+0000","last_fullsized":"2024-04-23T19:13:32.986571+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.986453+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.986453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:15.050911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"32'37","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.889150+0000","last_change":"2024-04-23T19:13:41.889150+0000","last_active":"2024-04-23T19:13:41.889150+0000","last_peered":"2024-04-23T19:13:41.889150+0000","last_clean":"2024-04-23T19:13:41.889150+0000","last_became_active":"2024-04-23T19:10:13.500458+0000","last_became_peered":"2024-04-23T19:10:13.500458+0000","last_unstale":"2024-04-23T19:13:41.889150+0000","last_undegraded":"2024-04-23T19:13:41.889150+0000","last_fullsized":"2024-04-23T19:13:41.889150+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:41.889007+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.889007+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:50.624792+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.929939+0000","last_change":"2024-04-23T19:13:38.929939+0000","last_active":"2024-04-23T19:13:38.929939+0000","last_peered":"2024-04-23T19:13:38.929939+0000","last_clean":"2024-04-23T19:13:38.929939+0000","last_became_active":"2024-04-23T19:10:15.509522+0000","last_became_peered":"2024-04-23T19:10:15.509522+0000","last_unstale":"2024-04-23T19:13:38.929939+0000","last_undegraded":"2024-04-23T19:13:38.929939+0000","last_fullsized":"2024-04-23T19:13:38.929939+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.929807+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.929807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:00.218571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"32'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.758606+0000","last_change":"2024-04-23T19:13:50.758606+0000","last_active":"2024-04-23T19:13:50.758606+0000","last_peered":"2024-04-23T19:13:50.758606+0000","last_clean":"2024-04-23T19:13:50.758606+0000","last_became_active":"2024-04-23T19:10:17.514028+0000","last_became_peered":"2024-04-23T19:10:17.514028+0000","last_unstale":"2024-04-23T19:13:50.758606+0000","last_undegraded":"2024-04-23T19:13:50.758606+0000","last_fullsized":"2024-04-23T19:13:50.758606+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:50.758473+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.758473+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:28.510048+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"32'16","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.892767+0000","last_change":"2024-04-23T19:14:04.892767+0000","last_active":"2024-04-23T19:14:04.892767+0000","last_peered":"2024-04-23T19:14:04.892767+0000","last_clean":"2024-04-23T19:14:04.892767+0000","last_became_active":"2024-04-23T19:10:09.494447+0000","last_became_peered":"2024-04-23T19:10:09.494447+0000","last_unstale":"2024-04-23T19:14:04.892767+0000","last_undegraded":"2024-04-23T19:14:04.892767+0000","last_fullsized":"2024-04-23T19:14:04.892767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:14:04.892651+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:14:04.892651+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.892651+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:00.034408+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.010672+0000","last_change":"2024-04-23T19:14:04.010672+0000","last_active":"2024-04-23T19:14:04.010672+0000","last_peered":"2024-04-23T19:14:04.010672+0000","last_clean":"2024-04-23T19:14:04.010672+0000","last_became_active":"2024-04-23T19:10:07.491921+0000","last_became_peered":"2024-04-23T19:10:07.491921+0000","last_unstale":"2024-04-23T19:14:04.010672+0000","last_undegraded":"2024-04-23T19:14:04.010672+0000","last_fullsized":"2024-04-23T19:14:04.010672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:04.010558+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.010558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:50.880134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":231,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.491419+0000","last_change":"2024-04-23T19:13:50.491419+0000","last_active":"2024-04-23T19:13:50.491419+0000","last_peered":"2024-04-23T19:13:50.491419+0000","last_clean":"2024-04-23T19:13:50.491419+0000","last_became_active":"2024-04-23T19:10:11.488858+0000","last_became_peered":"2024-04-23T19:10:11.488858+0000","last_unstale":"2024-04-23T19:13:50.491419+0000","last_undegraded":"2024-04-23T19:13:50.491419+0000","last_fullsized":"2024-04-23T19:13:50.491419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:50.491297+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.491297+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:21.383865+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"32'9","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.851932+0000","last_change":"2024-04-23T19:13:32.851932+0000","last_active":"2024-04-23T19:13:32.851932+0000","last_peered":"2024-04-23T19:13:32.851932+0000","last_clean":"2024-04-23T19:13:32.851932+0000","last_became_active":"2024-04-23T19:10:13.491209+0000","last_became_peered":"2024-04-23T19:10:13.491209+0000","last_unstale":"2024-04-23T19:13:32.851932+0000","last_undegraded":"2024-04-23T19:13:32.851932+0000","last_fullsized":"2024-04-23T19:13:32.851932+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'9","last_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_deep_scrub":"32'9","last_deep_scrub_stamp":"2024-04-23T19:13:32.851806+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.851806+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:35.714512+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.911815+0000","last_change":"2024-04-23T19:13:34.911815+0000","last_active":"2024-04-23T19:13:34.911815+0000","last_peered":"2024-04-23T19:13:34.911815+0000","last_clean":"2024-04-23T19:13:34.911815+0000","last_became_active":"2024-04-23T19:10:15.506413+0000","last_became_peered":"2024-04-23T19:10:15.506413+0000","last_unstale":"2024-04-23T19:13:34.911815+0000","last_undegraded":"2024-04-23T19:13:34.911815+0000","last_fullsized":"2024-04-23T19:13:34.911815+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.911697+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.911697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:51.912852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"30'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.426732+0000","last_change":"2024-04-23T19:14:25.426761+0000","last_active":"2024-04-23T19:14:25.426732+0000","last_peered":"2024-04-23T19:14:25.426732+0000","last_clean":"2024-04-23T19:14:25.426732+0000","last_became_active":"2024-04-23T19:10:17.517940+0000","last_became_peered":"2024-04-23T19:10:17.517940+0000","last_unstale":"2024-04-23T19:14:25.426732+0000","last_undegraded":"2024-04-23T19:14:25.426732+0000","last_fullsized":"2024-04-23T19:14:25.426732+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:25.426641+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:25.426641+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.426641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:20.568398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.976979+0000","last_change":"2024-04-23T19:14:10.976979+0000","last_active":"2024-04-23T19:14:10.976979+0000","last_peered":"2024-04-23T19:14:10.976979+0000","last_clean":"2024-04-23T19:14:10.976979+0000","last_became_active":"2024-04-23T19:10:09.489914+0000","last_became_peered":"2024-04-23T19:10:09.489914+0000","last_unstale":"2024-04-23T19:14:10.976979+0000","last_undegraded":"2024-04-23T19:14:10.976979+0000","last_fullsized":"2024-04-23T19:14:10.976979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:10.976867+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:10.976867+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.976867+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:05.496159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.919693+0000","last_change":"2024-04-23T19:14:23.919693+0000","last_active":"2024-04-23T19:14:23.919693+0000","last_peered":"2024-04-23T19:14:23.919693+0000","last_clean":"2024-04-23T19:14:23.919693+0000","last_became_active":"2024-04-23T19:10:07.503932+0000","last_became_peered":"2024-04-23T19:10:07.503932+0000","last_unstale":"2024-04-23T19:14:23.919693+0000","last_undegraded":"2024-04-23T19:14:23.919693+0000","last_fullsized":"2024-04-23T19:14:23.919693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:23.919566+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:23.919566+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.919566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:55.156177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.612314+0000","last_change":"2024-04-23T19:13:44.612314+0000","last_active":"2024-04-23T19:13:44.612314+0000","last_peered":"2024-04-23T19:13:44.612314+0000","last_clean":"2024-04-23T19:13:44.612314+0000","last_became_active":"2024-04-23T19:10:11.493247+0000","last_became_peered":"2024-04-23T19:10:11.493247+0000","last_unstale":"2024-04-23T19:13:44.612314+0000","last_undegraded":"2024-04-23T19:13:44.612314+0000","last_fullsized":"2024-04-23T19:13:44.612314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.612159+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.612159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:59.958714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"32'16","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.181858+0000","last_change":"2024-04-23T19:13:36.181858+0000","last_active":"2024-04-23T19:13:36.181858+0000","last_peered":"2024-04-23T19:13:36.181858+0000","last_clean":"2024-04-23T19:13:36.181858+0000","last_became_active":"2024-04-23T19:10:13.502317+0000","last_became_peered":"2024-04-23T19:10:13.502317+0000","last_unstale":"2024-04-23T19:13:36.181858+0000","last_undegraded":"2024-04-23T19:13:36.181858+0000","last_fullsized":"2024-04-23T19:13:36.181858+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:36.181735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.181735+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:21.509377+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"32'286","reported_seq":399,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.939305+0000","last_change":"2024-04-23T19:14:05.939305+0000","last_active":"2024-04-23T19:14:05.939305+0000","last_peered":"2024-04-23T19:14:05.939305+0000","last_clean":"2024-04-23T19:14:05.939305+0000","last_became_active":"2024-04-23T19:10:15.514852+0000","last_became_peered":"2024-04-23T19:10:15.514852+0000","last_unstale":"2024-04-23T19:14:05.939305+0000","last_undegraded":"2024-04-23T19:14:05.939305+0000","last_fullsized":"2024-04-23T19:14:05.939305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'286","last_scrub_stamp":"2024-04-23T19:14:05.939186+0000","last_deep_scrub":"32'286","last_deep_scrub_stamp":"2024-04-23T19:14:05.939186+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.939186+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:33.131024+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"32'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.127003+0000","last_change":"2024-04-23T19:14:17.127033+0000","last_active":"2024-04-23T19:14:17.127003+0000","last_peered":"2024-04-23T19:14:17.127003+0000","last_clean":"2024-04-23T19:14:17.127003+0000","last_became_active":"2024-04-23T19:10:17.521029+0000","last_became_peered":"2024-04-23T19:10:17.521029+0000","last_unstale":"2024-04-23T19:14:17.127003+0000","last_undegraded":"2024-04-23T19:14:17.127003+0000","last_fullsized":"2024-04-23T19:14:17.127003+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:17.126915+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:17.126915+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.126915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:46:04.993448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"32'4","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.491116+0000","last_change":"2024-04-23T19:14:21.491116+0000","last_active":"2024-04-23T19:14:21.491116+0000","last_peered":"2024-04-23T19:14:21.491116+0000","last_clean":"2024-04-23T19:14:21.491116+0000","last_became_active":"2024-04-23T19:10:09.486776+0000","last_became_peered":"2024-04-23T19:10:09.486776+0000","last_unstale":"2024-04-23T19:14:21.491116+0000","last_undegraded":"2024-04-23T19:14:21.491116+0000","last_fullsized":"2024-04-23T19:14:21.491116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:21.490995+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:21.490995+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.490995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:39.906047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.907441+0000","last_change":"2024-04-23T19:14:01.907441+0000","last_active":"2024-04-23T19:14:01.907441+0000","last_peered":"2024-04-23T19:14:01.907441+0000","last_clean":"2024-04-23T19:14:01.907441+0000","last_became_active":"2024-04-23T19:10:07.503286+0000","last_became_peered":"2024-04-23T19:10:07.503286+0000","last_unstale":"2024-04-23T19:14:01.907441+0000","last_undegraded":"2024-04-23T19:14:01.907441+0000","last_fullsized":"2024-04-23T19:14:01.907441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:01.907314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:01.907314+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.907314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:33:49.391209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.940545+0000","last_change":"2024-04-23T19:13:34.940545+0000","last_active":"2024-04-23T19:13:34.940545+0000","last_peered":"2024-04-23T19:13:34.940545+0000","last_clean":"2024-04-23T19:13:34.940545+0000","last_became_active":"2024-04-23T19:10:11.487165+0000","last_became_peered":"2024-04-23T19:10:11.487165+0000","last_unstale":"2024-04-23T19:13:34.940545+0000","last_undegraded":"2024-04-23T19:13:34.940545+0000","last_fullsized":"2024-04-23T19:13:34.940545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:34.940371+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.940371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:19.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"32'37","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.236743+0000","last_change":"2024-04-23T19:13:34.236743+0000","last_active":"2024-04-23T19:13:34.236743+0000","last_peered":"2024-04-23T19:13:34.236743+0000","last_clean":"2024-04-23T19:13:34.236743+0000","last_became_active":"2024-04-23T19:10:13.505499+0000","last_became_peered":"2024-04-23T19:10:13.505499+0000","last_unstale":"2024-04-23T19:13:34.236743+0000","last_undegraded":"2024-04-23T19:13:34.236743+0000","last_fullsized":"2024-04-23T19:13:34.236743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:34.236613+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.236613+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:39.299958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"32'285","reported_seq":414,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.684754+0000","last_change":"2024-04-23T19:13:32.684754+0000","last_active":"2024-04-23T19:13:32.684754+0000","last_peered":"2024-04-23T19:13:32.684754+0000","last_clean":"2024-04-23T19:13:32.684754+0000","last_became_active":"2024-04-23T19:10:15.500201+0000","last_became_peered":"2024-04-23T19:10:15.500201+0000","last_unstale":"2024-04-23T19:13:32.684754+0000","last_undegraded":"2024-04-23T19:13:32.684754+0000","last_fullsized":"2024-04-23T19:13:32.684754+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:32.684622+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.684622+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:16.152805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"32'6","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.163019+0000","last_change":"2024-04-23T19:14:20.163047+0000","last_active":"2024-04-23T19:14:20.163019+0000","last_peered":"2024-04-23T19:14:20.163019+0000","last_clean":"2024-04-23T19:14:20.163019+0000","last_became_active":"2024-04-23T19:10:47.723266+0000","last_became_peered":"2024-04-23T19:10:47.723266+0000","last_unstale":"2024-04-23T19:14:20.163019+0000","last_undegraded":"2024-04-23T19:14:20.163019+0000","last_fullsized":"2024-04-23T19:14:20.163019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:20.162930+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:20.162930+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.162930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:09.380533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.053095+0000","last_change":"2024-04-23T19:14:13.053095+0000","last_active":"2024-04-23T19:14:13.053095+0000","last_peered":"2024-04-23T19:14:13.053095+0000","last_clean":"2024-04-23T19:14:13.053095+0000","last_became_active":"2024-04-23T19:10:07.491682+0000","last_became_peered":"2024-04-23T19:10:07.491682+0000","last_unstale":"2024-04-23T19:14:13.053095+0000","last_undegraded":"2024-04-23T19:14:13.053095+0000","last_fullsized":"2024-04-23T19:14:13.053095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:13.052973+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:13.052973+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.052973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:02.399906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.919152+0000","last_change":"2024-04-23T19:13:40.919152+0000","last_active":"2024-04-23T19:13:40.919152+0000","last_peered":"2024-04-23T19:13:40.919152+0000","last_clean":"2024-04-23T19:13:40.919152+0000","last_became_active":"2024-04-23T19:10:11.489765+0000","last_became_peered":"2024-04-23T19:10:11.489765+0000","last_unstale":"2024-04-23T19:13:40.919152+0000","last_undegraded":"2024-04-23T19:13:40.919152+0000","last_fullsized":"2024-04-23T19:13:40.919152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:40.919038+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.919038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:35.160636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"32'51","reported_seq":112,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.206079+0000","last_change":"2024-04-23T19:13:39.206079+0000","last_active":"2024-04-23T19:13:39.206079+0000","last_peered":"2024-04-23T19:13:39.206079+0000","last_clean":"2024-04-23T19:13:39.206079+0000","last_became_active":"2024-04-23T19:10:13.494409+0000","last_became_peered":"2024-04-23T19:10:13.494409+0000","last_unstale":"2024-04-23T19:13:39.206079+0000","last_undegraded":"2024-04-23T19:13:39.206079+0000","last_fullsized":"2024-04-23T19:13:39.206079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'51","last_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_deep_scrub":"32'51","last_deep_scrub_stamp":"2024-04-23T19:13:39.205895+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.205895+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:03.299394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.646133+0000","last_change":"2024-04-23T19:13:43.646133+0000","last_active":"2024-04-23T19:13:43.646133+0000","last_peered":"2024-04-23T19:13:43.646133+0000","last_clean":"2024-04-23T19:13:43.646133+0000","last_became_active":"2024-04-23T19:10:15.506299+0000","last_became_peered":"2024-04-23T19:10:15.506299+0000","last_unstale":"2024-04-23T19:13:43.646133+0000","last_undegraded":"2024-04-23T19:13:43.646133+0000","last_fullsized":"2024-04-23T19:13:43.646133+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.646006+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.646006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:44.344504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"32'3","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.099221+0000","last_change":"2024-04-23T19:14:16.099221+0000","last_active":"2024-04-23T19:14:16.099221+0000","last_peered":"2024-04-23T19:14:16.099221+0000","last_clean":"2024-04-23T19:14:16.099221+0000","last_became_active":"2024-04-23T19:10:17.519420+0000","last_became_peered":"2024-04-23T19:10:17.519420+0000","last_unstale":"2024-04-23T19:14:16.099221+0000","last_undegraded":"2024-04-23T19:14:16.099221+0000","last_fullsized":"2024-04-23T19:14:16.099221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:16.099098+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:16.099098+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.099098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:32.839653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.835852+0000","last_change":"2024-04-23T19:13:58.835852+0000","last_active":"2024-04-23T19:13:58.835852+0000","last_peered":"2024-04-23T19:13:58.835852+0000","last_clean":"2024-04-23T19:13:58.835852+0000","last_became_active":"2024-04-23T19:10:09.481766+0000","last_became_peered":"2024-04-23T19:10:09.481766+0000","last_unstale":"2024-04-23T19:13:58.835852+0000","last_undegraded":"2024-04-23T19:13:58.835852+0000","last_fullsized":"2024-04-23T19:13:58.835852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:58.835665+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.835665+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:05.136073+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.024266+0000","last_change":"2024-04-23T19:13:54.024266+0000","last_active":"2024-04-23T19:13:54.024266+0000","last_peered":"2024-04-23T19:13:54.024266+0000","last_clean":"2024-04-23T19:13:54.024266+0000","last_became_active":"2024-04-23T19:10:07.500150+0000","last_became_peered":"2024-04-23T19:10:07.500150+0000","last_unstale":"2024-04-23T19:13:54.024266+0000","last_undegraded":"2024-04-23T19:13:54.024266+0000","last_fullsized":"2024-04-23T19:13:54.024266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:54.024146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.024146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:24.982777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.680645+0000","last_change":"2024-04-23T19:13:42.680645+0000","last_active":"2024-04-23T19:13:42.680645+0000","last_peered":"2024-04-23T19:13:42.680645+0000","last_clean":"2024-04-23T19:13:42.680645+0000","last_became_active":"2024-04-23T19:10:11.493280+0000","last_became_peered":"2024-04-23T19:10:11.493280+0000","last_unstale":"2024-04-23T19:13:42.680645+0000","last_undegraded":"2024-04-23T19:13:42.680645+0000","last_fullsized":"2024-04-23T19:13:42.680645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.680534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.680534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:03.703466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"32'38","reported_seq":94,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.936900+0000","last_change":"2024-04-23T19:13:58.936900+0000","last_active":"2024-04-23T19:13:58.936900+0000","last_peered":"2024-04-23T19:13:58.936900+0000","last_clean":"2024-04-23T19:13:58.936900+0000","last_became_active":"2024-04-23T19:10:13.501466+0000","last_became_peered":"2024-04-23T19:10:13.501466+0000","last_unstale":"2024-04-23T19:13:58.936900+0000","last_undegraded":"2024-04-23T19:13:58.936900+0000","last_fullsized":"2024-04-23T19:13:58.936900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:58.936780+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.936780+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:15:49.313649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.585132+0000","last_change":"2024-04-23T19:13:28.585132+0000","last_active":"2024-04-23T19:13:28.585132+0000","last_peered":"2024-04-23T19:13:28.585132+0000","last_clean":"2024-04-23T19:13:28.585132+0000","last_became_active":"2024-04-23T19:10:15.501862+0000","last_became_peered":"2024-04-23T19:10:15.501862+0000","last_unstale":"2024-04-23T19:13:28.585132+0000","last_undegraded":"2024-04-23T19:13:28.585132+0000","last_fullsized":"2024-04-23T19:13:28.585132+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.585004+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.585004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:05.071466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"32'6","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.768856+0000","last_change":"2024-04-23T19:14:24.768856+0000","last_active":"2024-04-23T19:14:24.768856+0000","last_peered":"2024-04-23T19:14:24.768856+0000","last_clean":"2024-04-23T19:14:24.768856+0000","last_became_active":"2024-04-23T19:10:17.518012+0000","last_became_peered":"2024-04-23T19:10:17.518012+0000","last_unstale":"2024-04-23T19:14:24.768856+0000","last_undegraded":"2024-04-23T19:14:24.768856+0000","last_fullsized":"2024-04-23T19:14:24.768856+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:24.768667+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:24.768667+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.768667+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:28.438866+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"32'12","reported_seq":89,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.098652+0000","last_change":"2024-04-23T19:14:01.098652+0000","last_active":"2024-04-23T19:14:01.098652+0000","last_peered":"2024-04-23T19:14:01.098652+0000","last_clean":"2024-04-23T19:14:01.098652+0000","last_became_active":"2024-04-23T19:10:09.491956+0000","last_became_peered":"2024-04-23T19:10:09.491956+0000","last_unstale":"2024-04-23T19:14:01.098652+0000","last_undegraded":"2024-04-23T19:14:01.098652+0000","last_fullsized":"2024-04-23T19:14:01.098652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:01.098506+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.098506+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:43.162965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:42.890364+0000","last_change":"2024-04-23T19:14:42.890402+0000","last_active":"2024-04-23T19:14:42.890364+0000","last_peered":"2024-04-23T19:14:42.890364+0000","last_clean":"2024-04-23T19:14:42.890364+0000","last_became_active":"2024-04-23T19:10:07.492329+0000","last_became_peered":"2024-04-23T19:10:07.492329+0000","last_unstale":"2024-04-23T19:14:42.890364+0000","last_undegraded":"2024-04-23T19:14:42.890364+0000","last_fullsized":"2024-04-23T19:14:42.890364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:42.890265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:42.890265+0000","last_clean_scrub_stamp":"2024-04-23T19:14:42.890265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:54:21.109740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.573946+0000","last_change":"2024-04-23T19:13:45.573946+0000","last_active":"2024-04-23T19:13:45.573946+0000","last_peered":"2024-04-23T19:13:45.573946+0000","last_clean":"2024-04-23T19:13:45.573946+0000","last_became_active":"2024-04-23T19:10:11.491230+0000","last_became_peered":"2024-04-23T19:10:11.491230+0000","last_unstale":"2024-04-23T19:13:45.573946+0000","last_undegraded":"2024-04-23T19:13:45.573946+0000","last_fullsized":"2024-04-23T19:13:45.573946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.573803+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.573803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:16.532433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"32'45","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.889437+0000","last_change":"2024-04-23T19:13:50.889437+0000","last_active":"2024-04-23T19:13:50.889437+0000","last_peered":"2024-04-23T19:13:50.889437+0000","last_clean":"2024-04-23T19:13:50.889437+0000","last_became_active":"2024-04-23T19:10:13.508036+0000","last_became_peered":"2024-04-23T19:10:13.508036+0000","last_unstale":"2024-04-23T19:13:50.889437+0000","last_undegraded":"2024-04-23T19:13:50.889437+0000","last_fullsized":"2024-04-23T19:13:50.889437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:50.889290+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.889290+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:52.811988+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"32'285","reported_seq":361,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.512239+0000","last_change":"2024-04-23T19:13:51.512239+0000","last_active":"2024-04-23T19:13:51.512239+0000","last_peered":"2024-04-23T19:13:51.512239+0000","last_clean":"2024-04-23T19:13:51.512239+0000","last_became_active":"2024-04-23T19:10:15.509440+0000","last_became_peered":"2024-04-23T19:10:15.509440+0000","last_unstale":"2024-04-23T19:13:51.512239+0000","last_undegraded":"2024-04-23T19:13:51.512239+0000","last_fullsized":"2024-04-23T19:13:51.512239+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:51.512072+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.512072+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:26.148887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:26.407672+0000","last_change":"2024-04-23T19:14:26.407672+0000","last_active":"2024-04-23T19:14:26.407672+0000","last_peered":"2024-04-23T19:14:26.407672+0000","last_clean":"2024-04-23T19:14:26.407672+0000","last_became_active":"2024-04-23T19:10:17.520540+0000","last_became_peered":"2024-04-23T19:10:17.520540+0000","last_unstale":"2024-04-23T19:14:26.407672+0000","last_undegraded":"2024-04-23T19:14:26.407672+0000","last_fullsized":"2024-04-23T19:14:26.407672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:26.407554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:26.407554+0000","last_clean_scrub_stamp":"2024-04-23T19:14:26.407554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:53.599392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"32'6","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.092424+0000","last_change":"2024-04-23T19:13:51.092424+0000","last_active":"2024-04-23T19:13:51.092424+0000","last_peered":"2024-04-23T19:13:51.092424+0000","last_clean":"2024-04-23T19:13:51.092424+0000","last_became_active":"2024-04-23T19:10:09.475610+0000","last_became_peered":"2024-04-23T19:10:09.475610+0000","last_unstale":"2024-04-23T19:13:51.092424+0000","last_undegraded":"2024-04-23T19:13:51.092424+0000","last_fullsized":"2024-04-23T19:13:51.092424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:51.092261+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.092261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:12.115193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:26.879545+0000","last_change":"2024-04-23T19:14:26.879545+0000","last_active":"2024-04-23T19:14:26.879545+0000","last_peered":"2024-04-23T19:14:26.879545+0000","last_clean":"2024-04-23T19:14:26.879545+0000","last_became_active":"2024-04-23T19:10:07.500752+0000","last_became_peered":"2024-04-23T19:10:07.500752+0000","last_unstale":"2024-04-23T19:14:26.879545+0000","last_undegraded":"2024-04-23T19:14:26.879545+0000","last_fullsized":"2024-04-23T19:14:26.879545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:26.879429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:26.879429+0000","last_clean_scrub_stamp":"2024-04-23T19:14:26.879429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:43.879098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.667080+0000","last_change":"2024-04-23T19:13:41.667080+0000","last_active":"2024-04-23T19:13:41.667080+0000","last_peered":"2024-04-23T19:13:41.667080+0000","last_clean":"2024-04-23T19:13:41.667080+0000","last_became_active":"2024-04-23T19:10:11.493305+0000","last_became_peered":"2024-04-23T19:10:11.493305+0000","last_unstale":"2024-04-23T19:13:41.667080+0000","last_undegraded":"2024-04-23T19:13:41.667080+0000","last_fullsized":"2024-04-23T19:13:41.667080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.666920+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.666920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:52.673059+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"32'63","reported_seq":137,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.975191+0000","last_change":"2024-04-23T19:13:33.975191+0000","last_active":"2024-04-23T19:13:33.975191+0000","last_peered":"2024-04-23T19:13:33.975191+0000","last_clean":"2024-04-23T19:13:33.975191+0000","last_became_active":"2024-04-23T19:10:13.498684+0000","last_became_peered":"2024-04-23T19:10:13.498684+0000","last_unstale":"2024-04-23T19:13:33.975191+0000","last_undegraded":"2024-04-23T19:13:33.975191+0000","last_fullsized":"2024-04-23T19:13:33.975191+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:33.975049+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.975049+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:07.187015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.695444+0000","last_change":"2024-04-23T19:13:35.695444+0000","last_active":"2024-04-23T19:13:35.695444+0000","last_peered":"2024-04-23T19:13:35.695444+0000","last_clean":"2024-04-23T19:13:35.695444+0000","last_became_active":"2024-04-23T19:10:15.502738+0000","last_became_peered":"2024-04-23T19:10:15.502738+0000","last_unstale":"2024-04-23T19:13:35.695444+0000","last_undegraded":"2024-04-23T19:13:35.695444+0000","last_fullsized":"2024-04-23T19:13:35.695444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.695326+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.695326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:43.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"32'21","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.534034+0000","last_change":"2024-04-23T19:14:22.534034+0000","last_active":"2024-04-23T19:14:22.534034+0000","last_peered":"2024-04-23T19:14:22.534034+0000","last_clean":"2024-04-23T19:14:22.534034+0000","last_became_active":"2024-04-23T19:10:17.513705+0000","last_became_peered":"2024-04-23T19:10:17.513705+0000","last_unstale":"2024-04-23T19:14:22.534034+0000","last_undegraded":"2024-04-23T19:14:22.534034+0000","last_fullsized":"2024-04-23T19:14:22.534034+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:14:22.533903+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:14:22.533903+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.533903+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:26.204102+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"32'8","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.796803+0000","last_change":"2024-04-23T19:13:55.796803+0000","last_active":"2024-04-23T19:13:55.796803+0000","last_peered":"2024-04-23T19:13:55.796803+0000","last_clean":"2024-04-23T19:13:55.796803+0000","last_became_active":"2024-04-23T19:10:09.490021+0000","last_became_peered":"2024-04-23T19:10:09.490021+0000","last_unstale":"2024-04-23T19:13:55.796803+0000","last_undegraded":"2024-04-23T19:13:55.796803+0000","last_fullsized":"2024-04-23T19:13:55.796803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:55.796686+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.796686+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:51:24.793617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.565751+0000","last_change":"2024-04-23T19:14:10.565751+0000","last_active":"2024-04-23T19:14:10.565751+0000","last_peered":"2024-04-23T19:14:10.565751+0000","last_clean":"2024-04-23T19:14:10.565751+0000","last_became_active":"2024-04-23T19:10:07.488610+0000","last_became_peered":"2024-04-23T19:10:07.488610+0000","last_unstale":"2024-04-23T19:14:10.565751+0000","last_undegraded":"2024-04-23T19:14:10.565751+0000","last_fullsized":"2024-04-23T19:14:10.565751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:10.565616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:10.565616+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.565616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:31.337130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.190374+0000","last_change":"2024-04-23T19:13:44.190374+0000","last_active":"2024-04-23T19:13:44.190374+0000","last_peered":"2024-04-23T19:13:44.190374+0000","last_clean":"2024-04-23T19:13:44.190374+0000","last_became_active":"2024-04-23T19:10:11.489753+0000","last_became_peered":"2024-04-23T19:10:11.489753+0000","last_unstale":"2024-04-23T19:13:44.190374+0000","last_undegraded":"2024-04-23T19:13:44.190374+0000","last_fullsized":"2024-04-23T19:13:44.190374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:44.190242+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.190242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:32.140502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"32'39","reported_seq":96,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.877469+0000","last_change":"2024-04-23T19:13:33.877469+0000","last_active":"2024-04-23T19:13:33.877469+0000","last_peered":"2024-04-23T19:13:33.877469+0000","last_clean":"2024-04-23T19:13:33.877469+0000","last_became_active":"2024-04-23T19:10:13.491041+0000","last_became_peered":"2024-04-23T19:10:13.491041+0000","last_unstale":"2024-04-23T19:13:33.877469+0000","last_undegraded":"2024-04-23T19:13:33.877469+0000","last_fullsized":"2024-04-23T19:13:33.877469+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:33.877330+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.877330+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:50:17.345513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.951037+0000","last_change":"2024-04-23T19:13:48.951037+0000","last_active":"2024-04-23T19:13:48.951037+0000","last_peered":"2024-04-23T19:13:48.951037+0000","last_clean":"2024-04-23T19:13:48.951037+0000","last_became_active":"2024-04-23T19:10:15.512080+0000","last_became_peered":"2024-04-23T19:10:15.512080+0000","last_unstale":"2024-04-23T19:13:48.951037+0000","last_undegraded":"2024-04-23T19:13:48.951037+0000","last_fullsized":"2024-04-23T19:13:48.951037+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.950905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.950905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:41.775070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"32'4","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:10.014390+0000","last_change":"2024-04-23T19:14:10.014390+0000","last_active":"2024-04-23T19:14:10.014390+0000","last_peered":"2024-04-23T19:14:10.014390+0000","last_clean":"2024-04-23T19:14:10.014390+0000","last_became_active":"2024-04-23T19:10:17.510042+0000","last_became_peered":"2024-04-23T19:10:17.510042+0000","last_unstale":"2024-04-23T19:14:10.014390+0000","last_undegraded":"2024-04-23T19:14:10.014390+0000","last_fullsized":"2024-04-23T19:14:10.014390+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:10.014222+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:10.014222+0000","last_clean_scrub_stamp":"2024-04-23T19:14:10.014222+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:00:32.382010+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"32'12","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.920017+0000","last_change":"2024-04-23T19:13:42.920017+0000","last_active":"2024-04-23T19:13:42.920017+0000","last_peered":"2024-04-23T19:13:42.920017+0000","last_clean":"2024-04-23T19:13:42.920017+0000","last_became_active":"2024-04-23T19:10:09.498731+0000","last_became_peered":"2024-04-23T19:10:09.498731+0000","last_unstale":"2024-04-23T19:13:42.920017+0000","last_undegraded":"2024-04-23T19:13:42.920017+0000","last_fullsized":"2024-04-23T19:13:42.920017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:13:42.919888+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.919888+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:33.023562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:30.907972+0000","last_change":"2024-04-23T19:14:30.907995+0000","last_active":"2024-04-23T19:14:30.907972+0000","last_peered":"2024-04-23T19:14:30.907972+0000","last_clean":"2024-04-23T19:14:30.907972+0000","last_became_active":"2024-04-23T19:10:07.494611+0000","last_became_peered":"2024-04-23T19:10:07.494611+0000","last_unstale":"2024-04-23T19:14:30.907972+0000","last_undegraded":"2024-04-23T19:14:30.907972+0000","last_fullsized":"2024-04-23T19:14:30.907972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:30.907880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:30.907880+0000","last_clean_scrub_stamp":"2024-04-23T19:14:30.907880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:16.078190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.565530+0000","last_change":"2024-04-23T19:13:46.565530+0000","last_active":"2024-04-23T19:13:46.565530+0000","last_peered":"2024-04-23T19:13:46.565530+0000","last_clean":"2024-04-23T19:13:46.565530+0000","last_became_active":"2024-04-23T19:10:11.491069+0000","last_became_peered":"2024-04-23T19:10:11.491069+0000","last_unstale":"2024-04-23T19:13:46.565530+0000","last_undegraded":"2024-04-23T19:13:46.565530+0000","last_fullsized":"2024-04-23T19:13:46.565530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.565380+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.565380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:54.525905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"32'45","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.188736+0000","last_change":"2024-04-23T19:13:40.188736+0000","last_active":"2024-04-23T19:13:40.188736+0000","last_peered":"2024-04-23T19:13:40.188736+0000","last_clean":"2024-04-23T19:13:40.188736+0000","last_became_active":"2024-04-23T19:10:13.492996+0000","last_became_peered":"2024-04-23T19:10:13.492996+0000","last_unstale":"2024-04-23T19:13:40.188736+0000","last_undegraded":"2024-04-23T19:13:40.188736+0000","last_fullsized":"2024-04-23T19:13:40.188736+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'45","last_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_deep_scrub":"32'45","last_deep_scrub_stamp":"2024-04-23T19:13:40.188598+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.188598+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:54.836675+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"32'285","reported_seq":416,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.128972+0000","last_change":"2024-04-23T19:13:28.128972+0000","last_active":"2024-04-23T19:13:28.128972+0000","last_peered":"2024-04-23T19:13:28.128972+0000","last_clean":"2024-04-23T19:13:28.128972+0000","last_became_active":"2024-04-23T19:10:15.506764+0000","last_became_peered":"2024-04-23T19:10:15.506764+0000","last_unstale":"2024-04-23T19:13:28.128972+0000","last_undegraded":"2024-04-23T19:13:28.128972+0000","last_fullsized":"2024-04-23T19:13:28.128972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:28.128856+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.128856+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:09.022827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"32'13","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.464629+0000","last_change":"2024-04-23T19:14:24.464629+0000","last_active":"2024-04-23T19:14:24.464629+0000","last_peered":"2024-04-23T19:14:24.464629+0000","last_clean":"2024-04-23T19:14:24.464629+0000","last_became_active":"2024-04-23T19:10:17.517865+0000","last_became_peered":"2024-04-23T19:10:17.517865+0000","last_unstale":"2024-04-23T19:14:24.464629+0000","last_undegraded":"2024-04-23T19:14:24.464629+0000","last_fullsized":"2024-04-23T19:14:24.464629+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'13","last_scrub_stamp":"2024-04-23T19:14:24.464510+0000","last_deep_scrub":"32'13","last_deep_scrub_stamp":"2024-04-23T19:14:24.464510+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.464510+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:40:14.783844+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"32'20","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.008630+0000","last_change":"2024-04-23T19:14:05.008630+0000","last_active":"2024-04-23T19:14:05.008630+0000","last_peered":"2024-04-23T19:14:05.008630+0000","last_clean":"2024-04-23T19:14:05.008630+0000","last_became_active":"2024-04-23T19:10:09.492984+0000","last_became_peered":"2024-04-23T19:10:09.492984+0000","last_unstale":"2024-04-23T19:14:05.008630+0000","last_undegraded":"2024-04-23T19:14:05.008630+0000","last_fullsized":"2024-04-23T19:14:05.008630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'20","last_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_deep_scrub":"32'20","last_deep_scrub_stamp":"2024-04-23T19:14:05.008496+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.008496+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:29:11.308903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.997873+0000","last_change":"2024-04-23T19:14:16.997873+0000","last_active":"2024-04-23T19:14:16.997873+0000","last_peered":"2024-04-23T19:14:16.997873+0000","last_clean":"2024-04-23T19:14:16.997873+0000","last_became_active":"2024-04-23T19:10:07.508061+0000","last_became_peered":"2024-04-23T19:10:07.508061+0000","last_unstale":"2024-04-23T19:14:16.997873+0000","last_undegraded":"2024-04-23T19:14:16.997873+0000","last_fullsized":"2024-04-23T19:14:16.997873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:16.997750+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:16.997750+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.997750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:57:56.974216+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'5","reported_seq":429,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.912027+0000","last_change":"2024-04-23T19:13:37.912027+0000","last_active":"2024-04-23T19:13:37.912027+0000","last_peered":"2024-04-23T19:13:37.912027+0000","last_clean":"2024-04-23T19:13:37.912027+0000","last_became_active":"2024-04-23T19:10:11.490935+0000","last_became_peered":"2024-04-23T19:10:11.490935+0000","last_unstale":"2024-04-23T19:13:37.912027+0000","last_undegraded":"2024-04-23T19:13:37.912027+0000","last_fullsized":"2024-04-23T19:13:37.912027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:37.911892+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.911892+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:23.295978+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"32'15","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.923039+0000","last_change":"2024-04-23T19:13:45.923039+0000","last_active":"2024-04-23T19:13:45.923039+0000","last_peered":"2024-04-23T19:13:45.923039+0000","last_clean":"2024-04-23T19:13:45.923039+0000","last_became_active":"2024-04-23T19:10:13.501140+0000","last_became_peered":"2024-04-23T19:10:13.501140+0000","last_unstale":"2024-04-23T19:13:45.923039+0000","last_undegraded":"2024-04-23T19:13:45.923039+0000","last_fullsized":"2024-04-23T19:13:45.923039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:45.922918+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.922918+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:04.148244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"32'285","reported_seq":478,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.784184+0000","last_change":"2024-04-23T19:13:26.784184+0000","last_active":"2024-04-23T19:13:26.784184+0000","last_peered":"2024-04-23T19:13:26.784184+0000","last_clean":"2024-04-23T19:13:26.784184+0000","last_became_active":"2024-04-23T19:10:15.501608+0000","last_became_peered":"2024-04-23T19:10:15.501608+0000","last_unstale":"2024-04-23T19:13:26.784184+0000","last_undegraded":"2024-04-23T19:13:26.784184+0000","last_fullsized":"2024-04-23T19:13:26.784184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:26.784064+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.784064+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:52:31.847408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"32'6","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.211854+0000","last_change":"2024-04-23T19:13:41.211854+0000","last_active":"2024-04-23T19:13:41.211854+0000","last_peered":"2024-04-23T19:13:41.211854+0000","last_clean":"2024-04-23T19:13:41.211854+0000","last_became_active":"2024-04-23T19:10:17.515209+0000","last_became_peered":"2024-04-23T19:10:17.515209+0000","last_unstale":"2024-04-23T19:13:41.211854+0000","last_undegraded":"2024-04-23T19:13:41.211854+0000","last_fullsized":"2024-04-23T19:13:41.211854+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:41.211740+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.211740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.450923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"32'2","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.738341+0000","last_change":"2024-04-23T19:13:48.738341+0000","last_active":"2024-04-23T19:13:48.738341+0000","last_peered":"2024-04-23T19:13:48.738341+0000","last_clean":"2024-04-23T19:13:48.738341+0000","last_became_active":"2024-04-23T19:10:09.494080+0000","last_became_peered":"2024-04-23T19:10:09.494080+0000","last_unstale":"2024-04-23T19:13:48.738341+0000","last_undegraded":"2024-04-23T19:13:48.738341+0000","last_fullsized":"2024-04-23T19:13:48.738341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:48.738148+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.738148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:12.860218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:33.837797+0000","last_change":"2024-04-23T19:14:33.837822+0000","last_active":"2024-04-23T19:14:33.837797+0000","last_peered":"2024-04-23T19:14:33.837797+0000","last_clean":"2024-04-23T19:14:33.837797+0000","last_became_active":"2024-04-23T19:10:07.506968+0000","last_became_peered":"2024-04-23T19:10:07.506968+0000","last_unstale":"2024-04-23T19:14:33.837797+0000","last_undegraded":"2024-04-23T19:14:33.837797+0000","last_fullsized":"2024-04-23T19:14:33.837797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:33.837696+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:33.837696+0000","last_clean_scrub_stamp":"2024-04-23T19:14:33.837696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:50.063335+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.111184+0000","last_change":"2024-04-23T19:13:28.111184+0000","last_active":"2024-04-23T19:13:28.111184+0000","last_peered":"2024-04-23T19:13:28.111184+0000","last_clean":"2024-04-23T19:13:28.111184+0000","last_became_active":"2024-04-23T19:10:11.490277+0000","last_became_peered":"2024-04-23T19:10:11.490277+0000","last_unstale":"2024-04-23T19:13:28.111184+0000","last_undegraded":"2024-04-23T19:13:28.111184+0000","last_fullsized":"2024-04-23T19:13:28.111184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:28.111056+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.111056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:18.387489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"32'48","reported_seq":108,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.948796+0000","last_change":"2024-04-23T19:13:35.948796+0000","last_active":"2024-04-23T19:13:35.948796+0000","last_peered":"2024-04-23T19:13:35.948796+0000","last_clean":"2024-04-23T19:13:35.948796+0000","last_became_active":"2024-04-23T19:10:13.491026+0000","last_became_peered":"2024-04-23T19:10:13.491026+0000","last_unstale":"2024-04-23T19:13:35.948796+0000","last_undegraded":"2024-04-23T19:13:35.948796+0000","last_fullsized":"2024-04-23T19:13:35.948796+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'48","last_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_deep_scrub":"32'48","last_deep_scrub_stamp":"2024-04-23T19:13:35.948676+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.948676+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:52.110221+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.116258+0000","last_change":"2024-04-23T19:13:30.116258+0000","last_active":"2024-04-23T19:13:30.116258+0000","last_peered":"2024-04-23T19:13:30.116258+0000","last_clean":"2024-04-23T19:13:30.116258+0000","last_became_active":"2024-04-23T19:10:15.506273+0000","last_became_peered":"2024-04-23T19:10:15.506273+0000","last_unstale":"2024-04-23T19:13:30.116258+0000","last_undegraded":"2024-04-23T19:13:30.116258+0000","last_fullsized":"2024-04-23T19:13:30.116258+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.116108+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.116108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:01:58.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"32'7","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.725037+0000","last_change":"2024-04-23T19:13:46.725037+0000","last_active":"2024-04-23T19:13:46.725037+0000","last_peered":"2024-04-23T19:13:46.725037+0000","last_clean":"2024-04-23T19:13:46.725037+0000","last_became_active":"2024-04-23T19:10:17.506240+0000","last_became_peered":"2024-04-23T19:10:17.506240+0000","last_unstale":"2024-04-23T19:13:46.725037+0000","last_undegraded":"2024-04-23T19:13:46.725037+0000","last_fullsized":"2024-04-23T19:13:46.725037+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'7","last_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_deep_scrub":"32'7","last_deep_scrub_stamp":"2024-04-23T19:13:46.724881+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.724881+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:17.683511+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"32'5","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.894509+0000","last_change":"2024-04-23T19:13:40.894509+0000","last_active":"2024-04-23T19:13:40.894509+0000","last_peered":"2024-04-23T19:13:40.894509+0000","last_clean":"2024-04-23T19:13:40.894509+0000","last_became_active":"2024-04-23T19:10:09.498595+0000","last_became_peered":"2024-04-23T19:10:09.498595+0000","last_unstale":"2024-04-23T19:13:40.894509+0000","last_undegraded":"2024-04-23T19:13:40.894509+0000","last_fullsized":"2024-04-23T19:13:40.894509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:40.894372+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.894372+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:11.403485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"32'119","reported_seq":213,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.926283+0000","last_change":"2024-04-23T19:14:12.926325+0000","last_active":"2024-04-23T19:14:12.926283+0000","last_peered":"2024-04-23T19:14:12.926283+0000","last_clean":"2024-04-23T19:14:12.926283+0000","last_became_active":"2024-04-23T19:09:52.517048+0000","last_became_peered":"2024-04-23T19:09:52.517048+0000","last_unstale":"2024-04-23T19:14:12.926283+0000","last_undegraded":"2024-04-23T19:14:12.926283+0000","last_fullsized":"2024-04-23T19:14:12.926283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'119","last_scrub_stamp":"2024-04-23T19:14:12.926188+0000","last_deep_scrub":"32'119","last_deep_scrub_stamp":"2024-04-23T19:14:12.926188+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.926188+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:18.589862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:35.913325+0000","last_change":"2024-04-23T19:14:35.913349+0000","last_active":"2024-04-23T19:14:35.913325+0000","last_peered":"2024-04-23T19:14:35.913325+0000","last_clean":"2024-04-23T19:14:35.913325+0000","last_became_active":"2024-04-23T19:09:59.448936+0000","last_became_peered":"2024-04-23T19:09:59.448936+0000","last_unstale":"2024-04-23T19:14:35.913325+0000","last_undegraded":"2024-04-23T19:14:35.913325+0000","last_fullsized":"2024-04-23T19:14:35.913325+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:35.913237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:35.913237+0000","last_clean_scrub_stamp":"2024-04-23T19:14:35.913237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:07:30.809325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.987464+0000","last_change":"2024-04-23T19:14:15.987464+0000","last_active":"2024-04-23T19:14:15.987464+0000","last_peered":"2024-04-23T19:14:15.987464+0000","last_clean":"2024-04-23T19:14:15.987464+0000","last_became_active":"2024-04-23T19:10:07.507531+0000","last_became_peered":"2024-04-23T19:10:07.507531+0000","last_unstale":"2024-04-23T19:14:15.987464+0000","last_undegraded":"2024-04-23T19:14:15.987464+0000","last_fullsized":"2024-04-23T19:14:15.987464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:15.987336+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:15.987336+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.987336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:09:54.753507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.842231+0000","last_change":"2024-04-23T19:13:43.842231+0000","last_active":"2024-04-23T19:13:43.842231+0000","last_peered":"2024-04-23T19:13:43.842231+0000","last_clean":"2024-04-23T19:13:43.842231+0000","last_became_active":"2024-04-23T19:10:11.486280+0000","last_became_peered":"2024-04-23T19:10:11.486280+0000","last_unstale":"2024-04-23T19:13:43.842231+0000","last_undegraded":"2024-04-23T19:13:43.842231+0000","last_fullsized":"2024-04-23T19:13:43.842231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.842100+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.842100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:26:04.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"32'88","reported_seq":179,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.876563+0000","last_change":"2024-04-23T19:13:37.876563+0000","last_active":"2024-04-23T19:13:37.876563+0000","last_peered":"2024-04-23T19:13:37.876563+0000","last_clean":"2024-04-23T19:13:37.876563+0000","last_became_active":"2024-04-23T19:10:13.499708+0000","last_became_peered":"2024-04-23T19:10:13.499708+0000","last_unstale":"2024-04-23T19:13:37.876563+0000","last_undegraded":"2024-04-23T19:13:37.876563+0000","last_fullsized":"2024-04-23T19:13:37.876563+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'88","last_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_deep_scrub":"32'88","last_deep_scrub_stamp":"2024-04-23T19:13:37.876439+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.876439+0000","objects_scrubbed":5,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:33:54.212735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.621413+0000","last_change":"2024-04-23T19:13:29.621413+0000","last_active":"2024-04-23T19:13:29.621413+0000","last_peered":"2024-04-23T19:13:29.621413+0000","last_clean":"2024-04-23T19:13:29.621413+0000","last_became_active":"2024-04-23T19:10:15.501564+0000","last_became_peered":"2024-04-23T19:10:15.501564+0000","last_unstale":"2024-04-23T19:13:29.621413+0000","last_undegraded":"2024-04-23T19:13:29.621413+0000","last_fullsized":"2024-04-23T19:13:29.621413+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.621264+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.621264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:03.242868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"32'81","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.101471+0000","last_change":"2024-04-23T19:13:49.101471+0000","last_active":"2024-04-23T19:13:49.101471+0000","last_peered":"2024-04-23T19:13:49.101471+0000","last_clean":"2024-04-23T19:13:49.101471+0000","last_became_active":"2024-04-23T19:10:17.515632+0000","last_became_peered":"2024-04-23T19:10:17.515632+0000","last_unstale":"2024-04-23T19:13:49.101471+0000","last_undegraded":"2024-04-23T19:13:49.101471+0000","last_fullsized":"2024-04-23T19:13:49.101471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'81","last_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_deep_scrub":"32'81","last_deep_scrub_stamp":"2024-04-23T19:13:49.101341+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.101341+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:49.219878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"32'4","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.030892+0000","last_change":"2024-04-23T19:14:13.030892+0000","last_active":"2024-04-23T19:14:13.030892+0000","last_peered":"2024-04-23T19:14:13.030892+0000","last_clean":"2024-04-23T19:14:13.030892+0000","last_became_active":"2024-04-23T19:10:09.487832+0000","last_became_peered":"2024-04-23T19:10:09.487832+0000","last_unstale":"2024-04-23T19:14:13.030892+0000","last_undegraded":"2024-04-23T19:14:13.030892+0000","last_fullsized":"2024-04-23T19:14:13.030892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:13.030773+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:13.030773+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.030773+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:24.199482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:37.891862+0000","last_change":"2024-04-23T19:14:37.891887+0000","last_active":"2024-04-23T19:14:37.891862+0000","last_peered":"2024-04-23T19:14:37.891862+0000","last_clean":"2024-04-23T19:14:37.891862+0000","last_became_active":"2024-04-23T19:09:59.449389+0000","last_became_peered":"2024-04-23T19:09:59.449389+0000","last_unstale":"2024-04-23T19:14:37.891862+0000","last_undegraded":"2024-04-23T19:14:37.891862+0000","last_fullsized":"2024-04-23T19:14:37.891862+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:37.891758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:37.891758+0000","last_clean_scrub_stamp":"2024-04-23T19:14:37.891758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:22:05.733163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:16.550545+0000","last_change":"2024-04-23T19:14:16.550545+0000","last_active":"2024-04-23T19:14:16.550545+0000","last_peered":"2024-04-23T19:14:16.550545+0000","last_clean":"2024-04-23T19:14:16.550545+0000","last_became_active":"2024-04-23T19:10:07.485919+0000","last_became_peered":"2024-04-23T19:10:07.485919+0000","last_unstale":"2024-04-23T19:14:16.550545+0000","last_undegraded":"2024-04-23T19:14:16.550545+0000","last_fullsized":"2024-04-23T19:14:16.550545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:16.550416+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:16.550416+0000","last_clean_scrub_stamp":"2024-04-23T19:14:16.550416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:59.355007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.891613+0000","last_change":"2024-04-23T19:13:36.891613+0000","last_active":"2024-04-23T19:13:36.891613+0000","last_peered":"2024-04-23T19:13:36.891613+0000","last_clean":"2024-04-23T19:13:36.891613+0000","last_became_active":"2024-04-23T19:10:11.487406+0000","last_became_peered":"2024-04-23T19:10:11.487406+0000","last_unstale":"2024-04-23T19:13:36.891613+0000","last_undegraded":"2024-04-23T19:13:36.891613+0000","last_fullsized":"2024-04-23T19:13:36.891613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.891497+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.891497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:43.191904+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"32'290","reported_seq":516,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:28.753118+0000","last_change":"2024-04-23T19:13:28.753118+0000","last_active":"2024-04-23T19:13:28.753118+0000","last_peered":"2024-04-23T19:13:28.753118+0000","last_clean":"2024-04-23T19:13:28.753118+0000","last_became_active":"2024-04-23T19:10:13.500415+0000","last_became_peered":"2024-04-23T19:10:13.500415+0000","last_unstale":"2024-04-23T19:13:28.753118+0000","last_undegraded":"2024-04-23T19:13:28.753118+0000","last_fullsized":"2024-04-23T19:13:28.753118+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'290","last_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_deep_scrub":"32'290","last_deep_scrub_stamp":"2024-04-23T19:13:28.752962+0000","last_clean_scrub_stamp":"2024-04-23T19:13:28.752962+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:14.080605+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.883869+0000","last_change":"2024-04-23T19:13:53.883869+0000","last_active":"2024-04-23T19:13:53.883869+0000","last_peered":"2024-04-23T19:13:53.883869+0000","last_clean":"2024-04-23T19:13:53.883869+0000","last_became_active":"2024-04-23T19:10:15.512703+0000","last_became_peered":"2024-04-23T19:10:15.512703+0000","last_unstale":"2024-04-23T19:13:53.883869+0000","last_undegraded":"2024-04-23T19:13:53.883869+0000","last_fullsized":"2024-04-23T19:13:53.883869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:53.883735+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.883735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:38.160620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"32'3","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:25.725710+0000","last_change":"2024-04-23T19:14:25.725710+0000","last_active":"2024-04-23T19:14:25.725710+0000","last_peered":"2024-04-23T19:14:25.725710+0000","last_clean":"2024-04-23T19:14:25.725710+0000","last_became_active":"2024-04-23T19:10:17.518431+0000","last_became_peered":"2024-04-23T19:10:17.518431+0000","last_unstale":"2024-04-23T19:14:25.725710+0000","last_undegraded":"2024-04-23T19:14:25.725710+0000","last_fullsized":"2024-04-23T19:14:25.725710+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:25.725582+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:25.725582+0000","last_clean_scrub_stamp":"2024-04-23T19:14:25.725582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:59:55.273423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"32'10","reported_seq":76,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.586425+0000","last_change":"2024-04-23T19:14:09.586425+0000","last_active":"2024-04-23T19:14:09.586425+0000","last_peered":"2024-04-23T19:14:09.586425+0000","last_clean":"2024-04-23T19:14:09.586425+0000","last_became_active":"2024-04-23T19:10:09.486353+0000","last_became_peered":"2024-04-23T19:10:09.486353+0000","last_unstale":"2024-04-23T19:14:09.586425+0000","last_undegraded":"2024-04-23T19:14:09.586425+0000","last_fullsized":"2024-04-23T19:14:09.586425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:09.586258+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:09.586258+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.586258+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:08.710830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.006366+0000","last_change":"2024-04-23T19:14:07.006366+0000","last_active":"2024-04-23T19:14:07.006366+0000","last_peered":"2024-04-23T19:14:07.006366+0000","last_clean":"2024-04-23T19:14:07.006366+0000","last_became_active":"2024-04-23T19:09:59.449360+0000","last_became_peered":"2024-04-23T19:09:59.449360+0000","last_unstale":"2024-04-23T19:14:07.006366+0000","last_undegraded":"2024-04-23T19:14:07.006366+0000","last_fullsized":"2024-04-23T19:14:07.006366+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.006243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.006243+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.006243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:46.529613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.504995+0000","last_change":"2024-04-23T19:13:58.504995+0000","last_active":"2024-04-23T19:13:58.504995+0000","last_peered":"2024-04-23T19:13:58.504995+0000","last_clean":"2024-04-23T19:13:58.504995+0000","last_became_active":"2024-04-23T19:10:07.490357+0000","last_became_peered":"2024-04-23T19:10:07.490357+0000","last_unstale":"2024-04-23T19:13:58.504995+0000","last_undegraded":"2024-04-23T19:13:58.504995+0000","last_fullsized":"2024-04-23T19:13:58.504995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:58.504872+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.504872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:38.028243+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.171419+0000","last_change":"2024-04-23T19:14:22.171419+0000","last_active":"2024-04-23T19:14:22.171419+0000","last_peered":"2024-04-23T19:14:22.171419+0000","last_clean":"2024-04-23T19:14:22.171419+0000","last_became_active":"2024-04-23T19:10:47.730952+0000","last_became_peered":"2024-04-23T19:10:47.730952+0000","last_unstale":"2024-04-23T19:14:22.171419+0000","last_undegraded":"2024-04-23T19:14:22.171419+0000","last_fullsized":"2024-04-23T19:14:22.171419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:22.171266+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:22.171266+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.171266+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:06.448151+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"32'23","reported_seq":101,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.228773+0000","last_change":"2024-04-23T19:13:38.228773+0000","last_active":"2024-04-23T19:13:38.228773+0000","last_peered":"2024-04-23T19:13:38.228773+0000","last_clean":"2024-04-23T19:13:38.228773+0000","last_became_active":"2024-04-23T19:10:13.494927+0000","last_became_peered":"2024-04-23T19:10:13.494927+0000","last_unstale":"2024-04-23T19:13:38.228773+0000","last_undegraded":"2024-04-23T19:13:38.228773+0000","last_fullsized":"2024-04-23T19:13:38.228773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'23","last_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_deep_scrub":"32'23","last_deep_scrub_stamp":"2024-04-23T19:13:38.228582+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.228582+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:14.715044+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:26.555830+0000","last_change":"2024-04-23T19:13:26.555830+0000","last_active":"2024-04-23T19:13:26.555830+0000","last_peered":"2024-04-23T19:13:26.555830+0000","last_clean":"2024-04-23T19:13:26.555830+0000","last_became_active":"2024-04-23T19:10:15.501846+0000","last_became_peered":"2024-04-23T19:10:15.501846+0000","last_unstale":"2024-04-23T19:13:26.555830+0000","last_undegraded":"2024-04-23T19:13:26.555830+0000","last_fullsized":"2024-04-23T19:13:26.555830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:26.555714+0000","last_clean_scrub_stamp":"2024-04-23T19:13:26.555714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:11.883356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:44.940695+0000","last_change":"2024-04-23T19:14:44.940695+0000","last_active":"2024-04-23T19:14:44.940695+0000","last_peered":"2024-04-23T19:14:44.940695+0000","last_clean":"2024-04-23T19:14:44.940695+0000","last_became_active":"2024-04-23T19:10:17.516798+0000","last_became_peered":"2024-04-23T19:10:17.516798+0000","last_unstale":"2024-04-23T19:14:44.940695+0000","last_undegraded":"2024-04-23T19:14:44.940695+0000","last_fullsized":"2024-04-23T19:14:44.940695+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:44.940571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:44.940571+0000","last_clean_scrub_stamp":"2024-04-23T19:14:44.940571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:44:51.759106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"32'6","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.526311+0000","last_change":"2024-04-23T19:13:59.526311+0000","last_active":"2024-04-23T19:13:59.526311+0000","last_peered":"2024-04-23T19:13:59.526311+0000","last_clean":"2024-04-23T19:13:59.526311+0000","last_became_active":"2024-04-23T19:10:09.477419+0000","last_became_peered":"2024-04-23T19:10:09.477419+0000","last_unstale":"2024-04-23T19:13:59.526311+0000","last_undegraded":"2024-04-23T19:13:59.526311+0000","last_fullsized":"2024-04-23T19:13:59.526311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:59.526195+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.526195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:39.758894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:36.875396+0000","last_change":"2024-04-23T19:14:36.875431+0000","last_active":"2024-04-23T19:14:36.875396+0000","last_peered":"2024-04-23T19:14:36.875396+0000","last_clean":"2024-04-23T19:14:36.875396+0000","last_became_active":"2024-04-23T19:09:59.449339+0000","last_became_peered":"2024-04-23T19:09:59.449339+0000","last_unstale":"2024-04-23T19:14:36.875396+0000","last_undegraded":"2024-04-23T19:14:36.875396+0000","last_fullsized":"2024-04-23T19:14:36.875396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:36.875297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:36.875297+0000","last_clean_scrub_stamp":"2024-04-23T19:14:36.875297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:40:55.108479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.981539+0000","last_change":"2024-04-23T19:14:07.981539+0000","last_active":"2024-04-23T19:14:07.981539+0000","last_peered":"2024-04-23T19:14:07.981539+0000","last_clean":"2024-04-23T19:14:07.981539+0000","last_became_active":"2024-04-23T19:10:07.481928+0000","last_became_peered":"2024-04-23T19:10:07.481928+0000","last_unstale":"2024-04-23T19:14:07.981539+0000","last_undegraded":"2024-04-23T19:14:07.981539+0000","last_fullsized":"2024-04-23T19:14:07.981539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:07.981428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:07.981428+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.981428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:48.214128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.501453+0000","last_change":"2024-04-23T19:13:49.501453+0000","last_active":"2024-04-23T19:13:49.501453+0000","last_peered":"2024-04-23T19:13:49.501453+0000","last_clean":"2024-04-23T19:13:49.501453+0000","last_became_active":"2024-04-23T19:10:11.490073+0000","last_became_peered":"2024-04-23T19:10:11.490073+0000","last_unstale":"2024-04-23T19:13:49.501453+0000","last_undegraded":"2024-04-23T19:13:49.501453+0000","last_fullsized":"2024-04-23T19:13:49.501453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.501336+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.501336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:21.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"32'21","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.866575+0000","last_change":"2024-04-23T19:13:39.866575+0000","last_active":"2024-04-23T19:13:39.866575+0000","last_peered":"2024-04-23T19:13:39.866575+0000","last_clean":"2024-04-23T19:13:39.866575+0000","last_became_active":"2024-04-23T19:10:13.500192+0000","last_became_peered":"2024-04-23T19:10:13.500192+0000","last_unstale":"2024-04-23T19:13:39.866575+0000","last_undegraded":"2024-04-23T19:13:39.866575+0000","last_fullsized":"2024-04-23T19:13:39.866575+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'21","last_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_deep_scrub":"32'21","last_deep_scrub_stamp":"2024-04-23T19:13:39.866421+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.866421+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:20.099121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.919527+0000","last_change":"2024-04-23T19:13:47.919527+0000","last_active":"2024-04-23T19:13:47.919527+0000","last_peered":"2024-04-23T19:13:47.919527+0000","last_clean":"2024-04-23T19:13:47.919527+0000","last_became_active":"2024-04-23T19:10:15.512179+0000","last_became_peered":"2024-04-23T19:10:15.512179+0000","last_unstale":"2024-04-23T19:13:47.919527+0000","last_undegraded":"2024-04-23T19:13:47.919527+0000","last_fullsized":"2024-04-23T19:13:47.919527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.919412+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.919412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:48.916350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"32'16","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.731452+0000","last_change":"2024-04-23T19:14:21.731452+0000","last_active":"2024-04-23T19:14:21.731452+0000","last_peered":"2024-04-23T19:14:21.731452+0000","last_clean":"2024-04-23T19:14:21.731452+0000","last_became_active":"2024-04-23T19:10:17.514150+0000","last_became_peered":"2024-04-23T19:10:17.514150+0000","last_unstale":"2024-04-23T19:14:21.731452+0000","last_undegraded":"2024-04-23T19:14:21.731452+0000","last_fullsized":"2024-04-23T19:14:21.731452+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:14:21.731320+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:14:21.731320+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.731320+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:12.108189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":71,"num_write":41,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"32'6","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.694263+0000","last_change":"2024-04-23T19:13:47.694263+0000","last_active":"2024-04-23T19:13:47.694263+0000","last_peered":"2024-04-23T19:13:47.694263+0000","last_clean":"2024-04-23T19:13:47.694263+0000","last_became_active":"2024-04-23T19:10:09.494253+0000","last_became_peered":"2024-04-23T19:10:09.494253+0000","last_unstale":"2024-04-23T19:13:47.694263+0000","last_undegraded":"2024-04-23T19:13:47.694263+0000","last_fullsized":"2024-04-23T19:13:47.694263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:47.694146+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.694146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:55.654671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'4","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.997935+0000","last_change":"2024-04-23T19:14:08.997935+0000","last_active":"2024-04-23T19:14:08.997935+0000","last_peered":"2024-04-23T19:14:08.997935+0000","last_clean":"2024-04-23T19:14:08.997935+0000","last_became_active":"2024-04-23T19:10:07.483527+0000","last_became_peered":"2024-04-23T19:10:07.483527+0000","last_unstale":"2024-04-23T19:14:08.997935+0000","last_undegraded":"2024-04-23T19:14:08.997935+0000","last_fullsized":"2024-04-23T19:14:08.997935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:08.997767+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:08.997767+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.997767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:39.506879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.514961+0000","last_change":"2024-04-23T19:14:13.514987+0000","last_active":"2024-04-23T19:14:13.514961+0000","last_peered":"2024-04-23T19:14:13.514961+0000","last_clean":"2024-04-23T19:14:13.514961+0000","last_became_active":"2024-04-23T19:09:59.450580+0000","last_became_peered":"2024-04-23T19:09:59.450580+0000","last_unstale":"2024-04-23T19:14:13.514961+0000","last_undegraded":"2024-04-23T19:14:13.514961+0000","last_fullsized":"2024-04-23T19:14:13.514961+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:13.514849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:13.514849+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.514849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:45.028155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"32'2","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.147888+0000","last_change":"2024-04-23T19:14:19.147888+0000","last_active":"2024-04-23T19:14:19.147888+0000","last_peered":"2024-04-23T19:14:19.147888+0000","last_clean":"2024-04-23T19:14:19.147888+0000","last_became_active":"2024-04-23T19:10:47.729677+0000","last_became_peered":"2024-04-23T19:10:47.729677+0000","last_unstale":"2024-04-23T19:14:19.147888+0000","last_undegraded":"2024-04-23T19:14:19.147888+0000","last_fullsized":"2024-04-23T19:14:19.147888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.147776+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.147776+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.147776+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:39.919289+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"32'75","reported_seq":152,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:34.702548+0000","last_change":"2024-04-23T19:13:34.702548+0000","last_active":"2024-04-23T19:13:34.702548+0000","last_peered":"2024-04-23T19:13:34.702548+0000","last_clean":"2024-04-23T19:13:34.702548+0000","last_became_active":"2024-04-23T19:10:13.505444+0000","last_became_peered":"2024-04-23T19:10:13.505444+0000","last_unstale":"2024-04-23T19:13:34.702548+0000","last_undegraded":"2024-04-23T19:13:34.702548+0000","last_fullsized":"2024-04-23T19:13:34.702548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'75","last_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_deep_scrub":"32'75","last_deep_scrub_stamp":"2024-04-23T19:13:34.702393+0000","last_clean_scrub_stamp":"2024-04-23T19:13:34.702393+0000","objects_scrubbed":9,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:50.863938+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.162563+0000","last_change":"2024-04-23T19:13:31.162563+0000","last_active":"2024-04-23T19:13:31.162563+0000","last_peered":"2024-04-23T19:13:31.162563+0000","last_clean":"2024-04-23T19:13:31.162563+0000","last_became_active":"2024-04-23T19:10:15.506017+0000","last_became_peered":"2024-04-23T19:10:15.506017+0000","last_unstale":"2024-04-23T19:13:31.162563+0000","last_undegraded":"2024-04-23T19:13:31.162563+0000","last_fullsized":"2024-04-23T19:13:31.162563+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.162449+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.162449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:47.422034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"32'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.538734+0000","last_change":"2024-04-23T19:13:55.538734+0000","last_active":"2024-04-23T19:13:55.538734+0000","last_peered":"2024-04-23T19:13:55.538734+0000","last_clean":"2024-04-23T19:13:55.538734+0000","last_became_active":"2024-04-23T19:10:17.506529+0000","last_became_peered":"2024-04-23T19:10:17.506529+0000","last_unstale":"2024-04-23T19:13:55.538734+0000","last_undegraded":"2024-04-23T19:13:55.538734+0000","last_fullsized":"2024-04-23T19:13:55.538734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:55.538607+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.538607+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:42.408183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"32'4","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.742366+0000","last_change":"2024-04-23T19:13:51.742366+0000","last_active":"2024-04-23T19:13:51.742366+0000","last_peered":"2024-04-23T19:13:51.742366+0000","last_clean":"2024-04-23T19:13:51.742366+0000","last_became_active":"2024-04-23T19:10:09.492526+0000","last_became_peered":"2024-04-23T19:10:09.492526+0000","last_unstale":"2024-04-23T19:13:51.742366+0000","last_undegraded":"2024-04-23T19:13:51.742366+0000","last_fullsized":"2024-04-23T19:13:51.742366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:51.742223+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.742223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:22.634791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:15.561186+0000","last_change":"2024-04-23T19:14:15.561186+0000","last_active":"2024-04-23T19:14:15.561186+0000","last_peered":"2024-04-23T19:14:15.561186+0000","last_clean":"2024-04-23T19:14:15.561186+0000","last_became_active":"2024-04-23T19:10:07.485912+0000","last_became_peered":"2024-04-23T19:10:07.485912+0000","last_unstale":"2024-04-23T19:14:15.561186+0000","last_undegraded":"2024-04-23T19:14:15.561186+0000","last_fullsized":"2024-04-23T19:14:15.561186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:15.561056+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:15.561056+0000","last_clean_scrub_stamp":"2024-04-23T19:14:15.561056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:20:42.489103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:38.890816+0000","last_change":"2024-04-23T19:14:38.890843+0000","last_active":"2024-04-23T19:14:38.890816+0000","last_peered":"2024-04-23T19:14:38.890816+0000","last_clean":"2024-04-23T19:14:38.890816+0000","last_became_active":"2024-04-23T19:09:59.449649+0000","last_became_peered":"2024-04-23T19:09:59.449649+0000","last_unstale":"2024-04-23T19:14:38.890816+0000","last_undegraded":"2024-04-23T19:14:38.890816+0000","last_fullsized":"2024-04-23T19:14:38.890816+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:38.890720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:38.890720+0000","last_clean_scrub_stamp":"2024-04-23T19:14:38.890720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:36:28.010171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.046354+0000","last_change":"2024-04-23T19:13:31.046354+0000","last_active":"2024-04-23T19:13:31.046354+0000","last_peered":"2024-04-23T19:13:31.046354+0000","last_clean":"2024-04-23T19:13:31.046354+0000","last_became_active":"2024-04-23T19:10:11.488623+0000","last_became_peered":"2024-04-23T19:10:11.488623+0000","last_unstale":"2024-04-23T19:13:31.046354+0000","last_undegraded":"2024-04-23T19:13:31.046354+0000","last_fullsized":"2024-04-23T19:13:31.046354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.046239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.046239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:01.938807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"32'22","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.762358+0000","last_change":"2024-04-23T19:13:29.762358+0000","last_active":"2024-04-23T19:13:29.762358+0000","last_peered":"2024-04-23T19:13:29.762358+0000","last_clean":"2024-04-23T19:13:29.762358+0000","last_became_active":"2024-04-23T19:10:13.497543+0000","last_became_peered":"2024-04-23T19:10:13.497543+0000","last_unstale":"2024-04-23T19:13:29.762358+0000","last_undegraded":"2024-04-23T19:13:29.762358+0000","last_fullsized":"2024-04-23T19:13:29.762358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'22","last_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_deep_scrub":"32'22","last_deep_scrub_stamp":"2024-04-23T19:13:29.762203+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.762203+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:37.849358+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"32'285","reported_seq":368,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.782396+0000","last_change":"2024-04-23T19:13:30.782396+0000","last_active":"2024-04-23T19:13:30.782396+0000","last_peered":"2024-04-23T19:13:30.782396+0000","last_clean":"2024-04-23T19:13:30.782396+0000","last_became_active":"2024-04-23T19:10:15.501657+0000","last_became_peered":"2024-04-23T19:10:15.501657+0000","last_unstale":"2024-04-23T19:13:30.782396+0000","last_undegraded":"2024-04-23T19:13:30.782396+0000","last_fullsized":"2024-04-23T19:13:30.782396+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:30.782266+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.782266+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:07.268727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"32'2","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.981194+0000","last_change":"2024-04-23T19:14:17.981194+0000","last_active":"2024-04-23T19:14:17.981194+0000","last_peered":"2024-04-23T19:14:17.981194+0000","last_clean":"2024-04-23T19:14:17.981194+0000","last_became_active":"2024-04-23T19:10:17.513529+0000","last_became_peered":"2024-04-23T19:10:17.513529+0000","last_unstale":"2024-04-23T19:14:17.981194+0000","last_undegraded":"2024-04-23T19:14:17.981194+0000","last_fullsized":"2024-04-23T19:14:17.981194+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:17.981076+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:17.981076+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.981076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:04:30.766806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.940021+0000","last_change":"2024-04-23T19:14:03.940021+0000","last_active":"2024-04-23T19:14:03.940021+0000","last_peered":"2024-04-23T19:14:03.940021+0000","last_clean":"2024-04-23T19:14:03.940021+0000","last_became_active":"2024-04-23T19:10:09.496587+0000","last_became_peered":"2024-04-23T19:10:09.496587+0000","last_unstale":"2024-04-23T19:14:03.940021+0000","last_undegraded":"2024-04-23T19:14:03.940021+0000","last_fullsized":"2024-04-23T19:14:03.940021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:03.939847+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:03.939847+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.939847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:54.259182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:34.864548+0000","last_change":"2024-04-23T19:14:34.864571+0000","last_active":"2024-04-23T19:14:34.864548+0000","last_peered":"2024-04-23T19:14:34.864548+0000","last_clean":"2024-04-23T19:14:34.864548+0000","last_became_active":"2024-04-23T19:10:07.493205+0000","last_became_peered":"2024-04-23T19:10:07.493205+0000","last_unstale":"2024-04-23T19:14:34.864548+0000","last_undegraded":"2024-04-23T19:14:34.864548+0000","last_fullsized":"2024-04-23T19:14:34.864548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:34.864459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:34.864459+0000","last_clean_scrub_stamp":"2024-04-23T19:14:34.864459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:20:21.122071+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.557659+0000","last_change":"2024-04-23T19:14:14.557659+0000","last_active":"2024-04-23T19:14:14.557659+0000","last_peered":"2024-04-23T19:14:14.557659+0000","last_clean":"2024-04-23T19:14:14.557659+0000","last_became_active":"2024-04-23T19:09:59.450785+0000","last_became_peered":"2024-04-23T19:09:59.450785+0000","last_unstale":"2024-04-23T19:14:14.557659+0000","last_undegraded":"2024-04-23T19:14:14.557659+0000","last_fullsized":"2024-04-23T19:14:14.557659+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.557539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.557539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.557539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:50:51.381912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.862700+0000","last_change":"2024-04-23T19:13:38.862700+0000","last_active":"2024-04-23T19:13:38.862700+0000","last_peered":"2024-04-23T19:13:38.862700+0000","last_clean":"2024-04-23T19:13:38.862700+0000","last_became_active":"2024-04-23T19:10:11.487174+0000","last_became_peered":"2024-04-23T19:10:11.487174+0000","last_unstale":"2024-04-23T19:13:38.862700+0000","last_undegraded":"2024-04-23T19:13:38.862700+0000","last_fullsized":"2024-04-23T19:13:38.862700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:38.862584+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.862584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:16:18.385955+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"32'16","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:51.889045+0000","last_change":"2024-04-23T19:13:51.889045+0000","last_active":"2024-04-23T19:13:51.889045+0000","last_peered":"2024-04-23T19:13:51.889045+0000","last_clean":"2024-04-23T19:13:51.889045+0000","last_became_active":"2024-04-23T19:10:13.503496+0000","last_became_peered":"2024-04-23T19:10:13.503496+0000","last_unstale":"2024-04-23T19:13:51.889045+0000","last_undegraded":"2024-04-23T19:13:51.889045+0000","last_fullsized":"2024-04-23T19:13:51.889045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'16","last_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_deep_scrub":"32'16","last_deep_scrub_stamp":"2024-04-23T19:13:51.888897+0000","last_clean_scrub_stamp":"2024-04-23T19:13:51.888897+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:47:41.106501+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"32'285","reported_seq":442,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.522645+0000","last_change":"2024-04-23T19:13:47.522645+0000","last_active":"2024-04-23T19:13:47.522645+0000","last_peered":"2024-04-23T19:13:47.522645+0000","last_clean":"2024-04-23T19:13:47.522645+0000","last_became_active":"2024-04-23T19:10:15.509073+0000","last_became_peered":"2024-04-23T19:10:15.509073+0000","last_unstale":"2024-04-23T19:13:47.522645+0000","last_undegraded":"2024-04-23T19:13:47.522645+0000","last_fullsized":"2024-04-23T19:13:47.522645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:47.522518+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.522518+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:37.798951+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"30'3","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.764934+0000","last_change":"2024-04-23T19:14:22.764934+0000","last_active":"2024-04-23T19:14:22.764934+0000","last_peered":"2024-04-23T19:14:22.764934+0000","last_clean":"2024-04-23T19:14:22.764934+0000","last_became_active":"2024-04-23T19:10:17.516485+0000","last_became_peered":"2024-04-23T19:10:17.516485+0000","last_unstale":"2024-04-23T19:14:22.764934+0000","last_undegraded":"2024-04-23T19:14:22.764934+0000","last_fullsized":"2024-04-23T19:14:22.764934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'3","last_scrub_stamp":"2024-04-23T19:14:22.764803+0000","last_deep_scrub":"30'3","last_deep_scrub_stamp":"2024-04-23T19:14:22.764803+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.764803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:24.439320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:50.083351+0000","last_change":"2024-04-23T19:13:50.083351+0000","last_active":"2024-04-23T19:13:50.083351+0000","last_peered":"2024-04-23T19:13:50.083351+0000","last_clean":"2024-04-23T19:13:50.083351+0000","last_became_active":"2024-04-23T19:10:09.493392+0000","last_became_peered":"2024-04-23T19:10:09.493392+0000","last_unstale":"2024-04-23T19:13:50.083351+0000","last_undegraded":"2024-04-23T19:13:50.083351+0000","last_fullsized":"2024-04-23T19:13:50.083351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:50.083230+0000","last_clean_scrub_stamp":"2024-04-23T19:13:50.083230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:01.089369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.940410+0000","last_change":"2024-04-23T19:14:11.940410+0000","last_active":"2024-04-23T19:14:11.940410+0000","last_peered":"2024-04-23T19:14:11.940410+0000","last_clean":"2024-04-23T19:14:11.940410+0000","last_became_active":"2024-04-23T19:10:07.499700+0000","last_became_peered":"2024-04-23T19:10:07.499700+0000","last_unstale":"2024-04-23T19:14:11.940410+0000","last_undegraded":"2024-04-23T19:14:11.940410+0000","last_fullsized":"2024-04-23T19:14:11.940410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:11.940272+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:11.940272+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.940272+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:59.806805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":48,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:39.842479+0000","last_change":"2024-04-23T19:14:39.842558+0000","last_active":"2024-04-23T19:14:39.842479+0000","last_peered":"2024-04-23T19:14:39.842479+0000","last_clean":"2024-04-23T19:14:39.842479+0000","last_became_active":"2024-04-23T19:09:59.577309+0000","last_became_peered":"2024-04-23T19:09:59.577309+0000","last_unstale":"2024-04-23T19:14:39.842479+0000","last_undegraded":"2024-04-23T19:14:39.842479+0000","last_fullsized":"2024-04-23T19:14:39.842479+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:39.842354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:39.842354+0000","last_clean_scrub_stamp":"2024-04-23T19:14:39.842354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:39:26.500277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.119771+0000","last_change":"2024-04-23T19:13:29.119771+0000","last_active":"2024-04-23T19:13:29.119771+0000","last_peered":"2024-04-23T19:13:29.119771+0000","last_clean":"2024-04-23T19:13:29.119771+0000","last_became_active":"2024-04-23T19:10:11.488807+0000","last_became_peered":"2024-04-23T19:10:11.488807+0000","last_unstale":"2024-04-23T19:13:29.119771+0000","last_undegraded":"2024-04-23T19:13:29.119771+0000","last_fullsized":"2024-04-23T19:13:29.119771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:29.119655+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.119655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:06.217082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"32'98","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.195253+0000","last_change":"2024-04-23T19:13:35.195253+0000","last_active":"2024-04-23T19:13:35.195253+0000","last_peered":"2024-04-23T19:13:35.195253+0000","last_clean":"2024-04-23T19:13:35.195253+0000","last_became_active":"2024-04-23T19:10:13.503056+0000","last_became_peered":"2024-04-23T19:10:13.503056+0000","last_unstale":"2024-04-23T19:13:35.195253+0000","last_undegraded":"2024-04-23T19:13:35.195253+0000","last_fullsized":"2024-04-23T19:13:35.195253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'98","last_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_deep_scrub":"32'98","last_deep_scrub_stamp":"2024-04-23T19:13:35.195120+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.195120+0000","objects_scrubbed":8,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:11.071311+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4316,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":98,"num_read_kb":92,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.664532+0000","last_change":"2024-04-23T19:13:30.664532+0000","last_active":"2024-04-23T19:13:30.664532+0000","last_peered":"2024-04-23T19:13:30.664532+0000","last_clean":"2024-04-23T19:13:30.664532+0000","last_became_active":"2024-04-23T19:10:15.500218+0000","last_became_peered":"2024-04-23T19:10:15.500218+0000","last_unstale":"2024-04-23T19:13:30.664532+0000","last_undegraded":"2024-04-23T19:13:30.664532+0000","last_fullsized":"2024-04-23T19:13:30.664532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.664414+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.664414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:40.569067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.161859+0000","last_change":"2024-04-23T19:13:45.161859+0000","last_active":"2024-04-23T19:13:45.161859+0000","last_peered":"2024-04-23T19:13:45.161859+0000","last_clean":"2024-04-23T19:13:45.161859+0000","last_became_active":"2024-04-23T19:10:17.515292+0000","last_became_peered":"2024-04-23T19:10:17.515292+0000","last_unstale":"2024-04-23T19:13:45.161859+0000","last_undegraded":"2024-04-23T19:13:45.161859+0000","last_fullsized":"2024-04-23T19:13:45.161859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:45.161710+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.161710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:30.545797+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.187444+0000","last_change":"2024-04-23T19:13:37.187444+0000","last_active":"2024-04-23T19:13:37.187444+0000","last_peered":"2024-04-23T19:13:37.187444+0000","last_clean":"2024-04-23T19:13:37.187444+0000","last_became_active":"2024-04-23T19:10:17.514926+0000","last_became_peered":"2024-04-23T19:10:17.514926+0000","last_unstale":"2024-04-23T19:13:37.187444+0000","last_undegraded":"2024-04-23T19:13:37.187444+0000","last_fullsized":"2024-04-23T19:13:37.187444+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:37.187313+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.187313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.274468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.039257+0000","last_change":"2024-04-23T19:13:59.039257+0000","last_active":"2024-04-23T19:13:59.039257+0000","last_peered":"2024-04-23T19:13:59.039257+0000","last_clean":"2024-04-23T19:13:59.039257+0000","last_became_active":"2024-04-23T19:10:09.491889+0000","last_became_peered":"2024-04-23T19:10:09.491889+0000","last_unstale":"2024-04-23T19:13:59.039257+0000","last_undegraded":"2024-04-23T19:13:59.039257+0000","last_fullsized":"2024-04-23T19:13:59.039257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:13:59.039137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.039137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:44:29.931705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:28.905207+0000","last_change":"2024-04-23T19:14:28.905207+0000","last_active":"2024-04-23T19:14:28.905207+0000","last_peered":"2024-04-23T19:14:28.905207+0000","last_clean":"2024-04-23T19:14:28.905207+0000","last_became_active":"2024-04-23T19:10:07.499780+0000","last_became_peered":"2024-04-23T19:10:07.499780+0000","last_unstale":"2024-04-23T19:14:28.905207+0000","last_undegraded":"2024-04-23T19:14:28.905207+0000","last_fullsized":"2024-04-23T19:14:28.905207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:28.905078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:28.905078+0000","last_clean_scrub_stamp":"2024-04-23T19:14:28.905078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:23:10.007150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:26.688322+0000","last_change":"2024-04-23T19:14:26.688322+0000","last_active":"2024-04-23T19:14:26.688322+0000","last_peered":"2024-04-23T19:14:26.688322+0000","last_clean":"2024-04-23T19:14:26.688322+0000","last_became_active":"2024-04-23T19:10:47.724606+0000","last_became_peered":"2024-04-23T19:10:47.724606+0000","last_unstale":"2024-04-23T19:14:26.688322+0000","last_undegraded":"2024-04-23T19:14:26.688322+0000","last_fullsized":"2024-04-23T19:14:26.688322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:26.688210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:26.688210+0000","last_clean_scrub_stamp":"2024-04-23T19:14:26.688210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:11:32.612364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"32'39","reported_seq":96,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.033364+0000","last_change":"2024-04-23T19:13:32.033364+0000","last_active":"2024-04-23T19:13:32.033364+0000","last_peered":"2024-04-23T19:13:32.033364+0000","last_clean":"2024-04-23T19:13:32.033364+0000","last_became_active":"2024-04-23T19:10:13.494779+0000","last_became_peered":"2024-04-23T19:10:13.494779+0000","last_unstale":"2024-04-23T19:13:32.033364+0000","last_undegraded":"2024-04-23T19:13:32.033364+0000","last_fullsized":"2024-04-23T19:13:32.033364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:32.033246+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.033246+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:16.932749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.897299+0000","last_change":"2024-04-23T19:13:57.897299+0000","last_active":"2024-04-23T19:13:57.897299+0000","last_peered":"2024-04-23T19:13:57.897299+0000","last_clean":"2024-04-23T19:13:57.897299+0000","last_became_active":"2024-04-23T19:10:15.513989+0000","last_became_peered":"2024-04-23T19:10:15.513989+0000","last_unstale":"2024-04-23T19:13:57.897299+0000","last_undegraded":"2024-04-23T19:13:57.897299+0000","last_fullsized":"2024-04-23T19:13:57.897299+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:57.897188+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.897188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:37.553642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"32'5","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.482950+0000","last_change":"2024-04-23T19:14:23.482976+0000","last_active":"2024-04-23T19:14:23.482950+0000","last_peered":"2024-04-23T19:14:23.482950+0000","last_clean":"2024-04-23T19:14:23.482950+0000","last_became_active":"2024-04-23T19:10:17.513754+0000","last_became_peered":"2024-04-23T19:10:17.513754+0000","last_unstale":"2024-04-23T19:14:23.482950+0000","last_undegraded":"2024-04-23T19:14:23.482950+0000","last_fullsized":"2024-04-23T19:14:23.482950+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:14:23.482860+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:14:23.482860+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.482860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:32.816753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"30'2","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:27.658360+0000","last_change":"2024-04-23T19:14:27.658360+0000","last_active":"2024-04-23T19:14:27.658360+0000","last_peered":"2024-04-23T19:14:27.658360+0000","last_clean":"2024-04-23T19:14:27.658360+0000","last_became_active":"2024-04-23T19:10:47.729060+0000","last_became_peered":"2024-04-23T19:10:47.729060+0000","last_unstale":"2024-04-23T19:14:27.658360+0000","last_undegraded":"2024-04-23T19:14:27.658360+0000","last_fullsized":"2024-04-23T19:14:27.658360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:27.658252+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:27.658252+0000","last_clean_scrub_stamp":"2024-04-23T19:14:27.658252+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:22.800009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.058602+0000","last_change":"2024-04-23T19:14:02.058602+0000","last_active":"2024-04-23T19:14:02.058602+0000","last_peered":"2024-04-23T19:14:02.058602+0000","last_clean":"2024-04-23T19:14:02.058602+0000","last_became_active":"2024-04-23T19:10:07.493094+0000","last_became_peered":"2024-04-23T19:10:07.493094+0000","last_unstale":"2024-04-23T19:14:02.058602+0000","last_undegraded":"2024-04-23T19:14:02.058602+0000","last_fullsized":"2024-04-23T19:14:02.058602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:02.058488+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.058488+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:35.270454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:35.891471+0000","last_change":"2024-04-23T19:13:35.891471+0000","last_active":"2024-04-23T19:13:35.891471+0000","last_peered":"2024-04-23T19:13:35.891471+0000","last_clean":"2024-04-23T19:13:35.891471+0000","last_became_active":"2024-04-23T19:10:11.487373+0000","last_became_peered":"2024-04-23T19:10:11.487373+0000","last_unstale":"2024-04-23T19:13:35.891471+0000","last_undegraded":"2024-04-23T19:13:35.891471+0000","last_fullsized":"2024-04-23T19:13:35.891471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:35.891324+0000","last_clean_scrub_stamp":"2024-04-23T19:13:35.891324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:22.760900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"32'8","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:37.738834+0000","last_change":"2024-04-23T19:13:37.738834+0000","last_active":"2024-04-23T19:13:37.738834+0000","last_peered":"2024-04-23T19:13:37.738834+0000","last_clean":"2024-04-23T19:13:37.738834+0000","last_became_active":"2024-04-23T19:10:13.494280+0000","last_became_peered":"2024-04-23T19:10:13.494280+0000","last_unstale":"2024-04-23T19:13:37.738834+0000","last_undegraded":"2024-04-23T19:13:37.738834+0000","last_fullsized":"2024-04-23T19:13:37.738834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:37.738703+0000","last_clean_scrub_stamp":"2024-04-23T19:13:37.738703+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:59.027468+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.218654+0000","last_change":"2024-04-23T19:13:33.218654+0000","last_active":"2024-04-23T19:13:33.218654+0000","last_peered":"2024-04-23T19:13:33.218654+0000","last_clean":"2024-04-23T19:13:33.218654+0000","last_became_active":"2024-04-23T19:10:15.513380+0000","last_became_peered":"2024-04-23T19:10:15.513380+0000","last_unstale":"2024-04-23T19:13:33.218654+0000","last_undegraded":"2024-04-23T19:13:33.218654+0000","last_fullsized":"2024-04-23T19:13:33.218654+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:33.218534+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.218534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:31.120188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"30'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:27.395040+0000","last_change":"2024-04-23T19:14:27.395083+0000","last_active":"2024-04-23T19:14:27.395040+0000","last_peered":"2024-04-23T19:14:27.395040+0000","last_clean":"2024-04-23T19:14:27.395040+0000","last_became_active":"2024-04-23T19:10:17.520523+0000","last_became_peered":"2024-04-23T19:10:17.520523+0000","last_unstale":"2024-04-23T19:14:27.395040+0000","last_undegraded":"2024-04-23T19:14:27.395040+0000","last_fullsized":"2024-04-23T19:14:27.395040+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:27.394906+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:27.394906+0000","last_clean_scrub_stamp":"2024-04-23T19:14:27.394906+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:50.231738+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"32'10","reported_seq":70,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.581741+0000","last_change":"2024-04-23T19:13:53.581741+0000","last_active":"2024-04-23T19:13:53.581741+0000","last_peered":"2024-04-23T19:13:53.581741+0000","last_clean":"2024-04-23T19:13:53.581741+0000","last_became_active":"2024-04-23T19:10:09.491201+0000","last_became_peered":"2024-04-23T19:10:09.491201+0000","last_unstale":"2024-04-23T19:13:53.581741+0000","last_undegraded":"2024-04-23T19:13:53.581741+0000","last_fullsized":"2024-04-23T19:13:53.581741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:13:53.581624+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.581624+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:37:26.793590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.067792+0000","last_change":"2024-04-23T19:13:56.067792+0000","last_active":"2024-04-23T19:13:56.067792+0000","last_peered":"2024-04-23T19:13:56.067792+0000","last_clean":"2024-04-23T19:13:56.067792+0000","last_became_active":"2024-04-23T19:10:07.500087+0000","last_became_peered":"2024-04-23T19:10:07.500087+0000","last_unstale":"2024-04-23T19:13:56.067792+0000","last_undegraded":"2024-04-23T19:13:56.067792+0000","last_fullsized":"2024-04-23T19:13:56.067792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:56.067677+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.067677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:27:46.344110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:43.223641+0000","last_change":"2024-04-23T19:13:43.223641+0000","last_active":"2024-04-23T19:13:43.223641+0000","last_peered":"2024-04-23T19:13:43.223641+0000","last_clean":"2024-04-23T19:13:43.223641+0000","last_became_active":"2024-04-23T19:10:11.492831+0000","last_became_peered":"2024-04-23T19:10:11.492831+0000","last_unstale":"2024-04-23T19:13:43.223641+0000","last_undegraded":"2024-04-23T19:13:43.223641+0000","last_fullsized":"2024-04-23T19:13:43.223641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:43.223515+0000","last_clean_scrub_stamp":"2024-04-23T19:13:43.223515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:59.385060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"32'47","reported_seq":120,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:40.709462+0000","last_change":"2024-04-23T19:13:40.709462+0000","last_active":"2024-04-23T19:13:40.709462+0000","last_peered":"2024-04-23T19:13:40.709462+0000","last_clean":"2024-04-23T19:13:40.709462+0000","last_became_active":"2024-04-23T19:10:13.490810+0000","last_became_peered":"2024-04-23T19:10:13.490810+0000","last_unstale":"2024-04-23T19:13:40.709462+0000","last_undegraded":"2024-04-23T19:13:40.709462+0000","last_fullsized":"2024-04-23T19:13:40.709462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'47","last_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_deep_scrub":"32'47","last_deep_scrub_stamp":"2024-04-23T19:13:40.709338+0000","last_clean_scrub_stamp":"2024-04-23T19:13:40.709338+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:40.827875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.737136+0000","last_change":"2024-04-23T19:13:27.737136+0000","last_active":"2024-04-23T19:13:27.737136+0000","last_peered":"2024-04-23T19:13:27.737136+0000","last_clean":"2024-04-23T19:13:27.737136+0000","last_became_active":"2024-04-23T19:10:15.497346+0000","last_became_peered":"2024-04-23T19:10:15.497346+0000","last_unstale":"2024-04-23T19:13:27.737136+0000","last_undegraded":"2024-04-23T19:13:27.737136+0000","last_fullsized":"2024-04-23T19:13:27.737136+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.737016+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.737016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:03.613207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"32'2","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:50.912365+0000","last_change":"2024-04-23T19:14:50.912404+0000","last_active":"2024-04-23T19:14:50.912365+0000","last_peered":"2024-04-23T19:14:50.912365+0000","last_clean":"2024-04-23T19:14:50.912365+0000","last_became_active":"2024-04-23T19:10:17.521164+0000","last_became_peered":"2024-04-23T19:10:17.521164+0000","last_unstale":"2024-04-23T19:14:50.912365+0000","last_undegraded":"2024-04-23T19:14:50.912365+0000","last_fullsized":"2024-04-23T19:14:50.912365+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:50.912271+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:50.912271+0000","last_clean_scrub_stamp":"2024-04-23T19:14:50.912271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:57.063390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'10","reported_seq":80,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.904055+0000","last_change":"2024-04-23T19:14:02.904055+0000","last_active":"2024-04-23T19:14:02.904055+0000","last_peered":"2024-04-23T19:14:02.904055+0000","last_clean":"2024-04-23T19:14:02.904055+0000","last_became_active":"2024-04-23T19:10:09.498034+0000","last_became_peered":"2024-04-23T19:10:09.498034+0000","last_unstale":"2024-04-23T19:14:02.904055+0000","last_undegraded":"2024-04-23T19:14:02.904055+0000","last_fullsized":"2024-04-23T19:14:02.904055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:02.903914+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:02.903914+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.903914+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:12.237807+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.755958+0000","last_change":"2024-04-23T19:14:18.755958+0000","last_active":"2024-04-23T19:14:18.755958+0000","last_peered":"2024-04-23T19:14:18.755958+0000","last_clean":"2024-04-23T19:14:18.755958+0000","last_became_active":"2024-04-23T19:10:07.496135+0000","last_became_peered":"2024-04-23T19:10:07.496135+0000","last_unstale":"2024-04-23T19:14:18.755958+0000","last_undegraded":"2024-04-23T19:14:18.755958+0000","last_fullsized":"2024-04-23T19:14:18.755958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.755843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.755843+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.755843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:31.351437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:30.092835+0000","last_change":"2024-04-23T19:13:30.092835+0000","last_active":"2024-04-23T19:13:30.092835+0000","last_peered":"2024-04-23T19:13:30.092835+0000","last_clean":"2024-04-23T19:13:30.092835+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:30.092835+0000","last_undegraded":"2024-04-23T19:13:30.092835+0000","last_fullsized":"2024-04-23T19:13:30.092835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:30.092707+0000","last_clean_scrub_stamp":"2024-04-23T19:13:30.092707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:20:02.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"32'38","reported_seq":94,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.254523+0000","last_change":"2024-04-23T19:13:42.254523+0000","last_active":"2024-04-23T19:13:42.254523+0000","last_peered":"2024-04-23T19:13:42.254523+0000","last_clean":"2024-04-23T19:13:42.254523+0000","last_became_active":"2024-04-23T19:10:13.492925+0000","last_became_peered":"2024-04-23T19:10:13.492925+0000","last_unstale":"2024-04-23T19:13:42.254523+0000","last_undegraded":"2024-04-23T19:13:42.254523+0000","last_fullsized":"2024-04-23T19:13:42.254523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'38","last_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_deep_scrub":"32'38","last_deep_scrub_stamp":"2024-04-23T19:13:42.254400+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.254400+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:59.255554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":53,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.124622+0000","last_change":"2024-04-23T19:13:27.124622+0000","last_active":"2024-04-23T19:13:27.124622+0000","last_peered":"2024-04-23T19:13:27.124622+0000","last_clean":"2024-04-23T19:13:27.124622+0000","last_became_active":"2024-04-23T19:10:15.508257+0000","last_became_peered":"2024-04-23T19:10:15.508257+0000","last_unstale":"2024-04-23T19:13:27.124622+0000","last_undegraded":"2024-04-23T19:13:27.124622+0000","last_fullsized":"2024-04-23T19:13:27.124622+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.124488+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.124488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:56.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.130109+0000","last_change":"2024-04-23T19:14:18.130109+0000","last_active":"2024-04-23T19:14:18.130109+0000","last_peered":"2024-04-23T19:14:18.130109+0000","last_clean":"2024-04-23T19:14:18.130109+0000","last_became_active":"2024-04-23T19:10:47.728069+0000","last_became_peered":"2024-04-23T19:10:47.728069+0000","last_unstale":"2024-04-23T19:14:18.130109+0000","last_undegraded":"2024-04-23T19:14:18.130109+0000","last_fullsized":"2024-04-23T19:14:18.130109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.129993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.129993+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.129993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:45:23.793668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.996211+0000","last_change":"2024-04-23T19:14:19.996211+0000","last_active":"2024-04-23T19:14:19.996211+0000","last_peered":"2024-04-23T19:14:19.996211+0000","last_clean":"2024-04-23T19:14:19.996211+0000","last_became_active":"2024-04-23T19:10:07.503551+0000","last_became_peered":"2024-04-23T19:10:07.503551+0000","last_unstale":"2024-04-23T19:14:19.996211+0000","last_undegraded":"2024-04-23T19:14:19.996211+0000","last_fullsized":"2024-04-23T19:14:19.996211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:19.996051+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:19.996051+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.996051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:19.442125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.511211+0000","last_change":"2024-04-23T19:13:48.511211+0000","last_active":"2024-04-23T19:13:48.511211+0000","last_peered":"2024-04-23T19:13:48.511211+0000","last_clean":"2024-04-23T19:13:48.511211+0000","last_became_active":"2024-04-23T19:10:11.490245+0000","last_became_peered":"2024-04-23T19:10:11.490245+0000","last_unstale":"2024-04-23T19:13:48.511211+0000","last_undegraded":"2024-04-23T19:13:48.511211+0000","last_fullsized":"2024-04-23T19:13:48.511211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.511103+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.511103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:25.608530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"32'63","reported_seq":136,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:38.697721+0000","last_change":"2024-04-23T19:13:38.697755+0000","last_active":"2024-04-23T19:13:38.697721+0000","last_peered":"2024-04-23T19:13:38.697721+0000","last_clean":"2024-04-23T19:13:38.697721+0000","last_became_active":"2024-04-23T19:10:13.493911+0000","last_became_peered":"2024-04-23T19:10:13.493911+0000","last_unstale":"2024-04-23T19:13:38.697721+0000","last_undegraded":"2024-04-23T19:13:38.697721+0000","last_fullsized":"2024-04-23T19:13:38.697721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'63","last_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_deep_scrub":"32'63","last_deep_scrub_stamp":"2024-04-23T19:13:38.697637+0000","last_clean_scrub_stamp":"2024-04-23T19:13:38.697637+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:22.296607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:32.201793+0000","last_change":"2024-04-23T19:13:32.201821+0000","last_active":"2024-04-23T19:13:32.201793+0000","last_peered":"2024-04-23T19:13:32.201793+0000","last_clean":"2024-04-23T19:13:32.201793+0000","last_became_active":"2024-04-23T19:10:15.505444+0000","last_became_peered":"2024-04-23T19:10:15.505444+0000","last_unstale":"2024-04-23T19:13:32.201793+0000","last_undegraded":"2024-04-23T19:13:32.201793+0000","last_fullsized":"2024-04-23T19:13:32.201793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:32.201694+0000","last_clean_scrub_stamp":"2024-04-23T19:13:32.201694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:43.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"32'14","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:28.376693+0000","last_change":"2024-04-23T19:14:28.376693+0000","last_active":"2024-04-23T19:14:28.376693+0000","last_peered":"2024-04-23T19:14:28.376693+0000","last_clean":"2024-04-23T19:14:28.376693+0000","last_became_active":"2024-04-23T19:10:17.521056+0000","last_became_peered":"2024-04-23T19:10:17.521056+0000","last_unstale":"2024-04-23T19:14:28.376693+0000","last_undegraded":"2024-04-23T19:14:28.376693+0000","last_fullsized":"2024-04-23T19:14:28.376693+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:14:28.376541+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:14:28.376541+0000","last_clean_scrub_stamp":"2024-04-23T19:14:28.376541+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:03:24.814925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"32'8","reported_seq":58,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.612354+0000","last_change":"2024-04-23T19:14:05.612354+0000","last_active":"2024-04-23T19:14:05.612354+0000","last_peered":"2024-04-23T19:14:05.612354+0000","last_clean":"2024-04-23T19:14:05.612354+0000","last_became_active":"2024-04-23T19:10:09.482282+0000","last_became_peered":"2024-04-23T19:10:09.482282+0000","last_unstale":"2024-04-23T19:14:05.612354+0000","last_undegraded":"2024-04-23T19:14:05.612354+0000","last_fullsized":"2024-04-23T19:14:05.612354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:05.612239+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.612239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:04.922704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"32'4","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.782031+0000","last_change":"2024-04-23T19:14:17.782031+0000","last_active":"2024-04-23T19:14:17.782031+0000","last_peered":"2024-04-23T19:14:17.782031+0000","last_clean":"2024-04-23T19:14:17.782031+0000","last_became_active":"2024-04-23T19:10:07.493514+0000","last_became_peered":"2024-04-23T19:10:07.493514+0000","last_unstale":"2024-04-23T19:14:17.782031+0000","last_undegraded":"2024-04-23T19:14:17.782031+0000","last_fullsized":"2024-04-23T19:14:17.782031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:17.781913+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:17.781913+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.781913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:45.876367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:48.142214+0000","last_change":"2024-04-23T19:13:48.142214+0000","last_active":"2024-04-23T19:13:48.142214+0000","last_peered":"2024-04-23T19:13:48.142214+0000","last_clean":"2024-04-23T19:13:48.142214+0000","last_became_active":"2024-04-23T19:10:11.486189+0000","last_became_peered":"2024-04-23T19:10:11.486189+0000","last_unstale":"2024-04-23T19:13:48.142214+0000","last_undegraded":"2024-04-23T19:13:48.142214+0000","last_fullsized":"2024-04-23T19:13:48.142214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:48.142093+0000","last_clean_scrub_stamp":"2024-04-23T19:13:48.142093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:17.068685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"32'46","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.927235+0000","last_change":"2024-04-23T19:13:54.927318+0000","last_active":"2024-04-23T19:13:54.927235+0000","last_peered":"2024-04-23T19:13:54.927235+0000","last_clean":"2024-04-23T19:13:54.927235+0000","last_became_active":"2024-04-23T19:10:13.501986+0000","last_became_peered":"2024-04-23T19:10:13.501986+0000","last_unstale":"2024-04-23T19:13:54.927235+0000","last_undegraded":"2024-04-23T19:13:54.927235+0000","last_fullsized":"2024-04-23T19:13:54.927235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:54.927131+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.927131+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:16:23.021585+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:42.866434+0000","last_change":"2024-04-23T19:13:42.866499+0000","last_active":"2024-04-23T19:13:42.866434+0000","last_peered":"2024-04-23T19:13:42.866434+0000","last_clean":"2024-04-23T19:13:42.866434+0000","last_became_active":"2024-04-23T19:10:15.509784+0000","last_became_peered":"2024-04-23T19:10:15.509784+0000","last_unstale":"2024-04-23T19:13:42.866434+0000","last_undegraded":"2024-04-23T19:13:42.866434+0000","last_fullsized":"2024-04-23T19:13:42.866434+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:42.866306+0000","last_clean_scrub_stamp":"2024-04-23T19:13:42.866306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:53.872444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"32'6","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:45.948264+0000","last_change":"2024-04-23T19:14:45.948264+0000","last_active":"2024-04-23T19:14:45.948264+0000","last_peered":"2024-04-23T19:14:45.948264+0000","last_clean":"2024-04-23T19:14:45.948264+0000","last_became_active":"2024-04-23T19:10:17.516857+0000","last_became_peered":"2024-04-23T19:10:17.516857+0000","last_unstale":"2024-04-23T19:14:45.948264+0000","last_undegraded":"2024-04-23T19:14:45.948264+0000","last_fullsized":"2024-04-23T19:14:45.948264+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:45.948104+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:45.948104+0000","last_clean_scrub_stamp":"2024-04-23T19:14:45.948104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:10.944728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"32'6","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.193141+0000","last_change":"2024-04-23T19:14:21.193203+0000","last_active":"2024-04-23T19:14:21.193141+0000","last_peered":"2024-04-23T19:14:21.193141+0000","last_clean":"2024-04-23T19:14:21.193141+0000","last_became_active":"2024-04-23T19:10:47.724371+0000","last_became_peered":"2024-04-23T19:10:47.724371+0000","last_unstale":"2024-04-23T19:14:21.193141+0000","last_undegraded":"2024-04-23T19:14:21.193141+0000","last_fullsized":"2024-04-23T19:14:21.193141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:21.193026+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:21.193026+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.193026+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:45.665300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:05.844813+0000","last_change":"2024-04-23T19:14:05.844813+0000","last_active":"2024-04-23T19:14:05.844813+0000","last_peered":"2024-04-23T19:14:05.844813+0000","last_clean":"2024-04-23T19:14:05.844813+0000","last_became_active":"2024-04-23T19:10:07.502156+0000","last_became_peered":"2024-04-23T19:10:07.502156+0000","last_unstale":"2024-04-23T19:14:05.844813+0000","last_undegraded":"2024-04-23T19:14:05.844813+0000","last_fullsized":"2024-04-23T19:14:05.844813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:05.844638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:05.844638+0000","last_clean_scrub_stamp":"2024-04-23T19:14:05.844638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:52:20.554775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:47.129709+0000","last_change":"2024-04-23T19:13:47.129736+0000","last_active":"2024-04-23T19:13:47.129709+0000","last_peered":"2024-04-23T19:13:47.129709+0000","last_clean":"2024-04-23T19:13:47.129709+0000","last_became_active":"2024-04-23T19:10:11.487136+0000","last_became_peered":"2024-04-23T19:10:11.487136+0000","last_unstale":"2024-04-23T19:13:47.129709+0000","last_undegraded":"2024-04-23T19:13:47.129709+0000","last_fullsized":"2024-04-23T19:13:47.129709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:47.129623+0000","last_clean_scrub_stamp":"2024-04-23T19:13:47.129623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:16:50.873891+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"32'39","reported_seq":96,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.723412+0000","last_change":"2024-04-23T19:13:36.723412+0000","last_active":"2024-04-23T19:13:36.723412+0000","last_peered":"2024-04-23T19:13:36.723412+0000","last_clean":"2024-04-23T19:13:36.723412+0000","last_became_active":"2024-04-23T19:10:13.497693+0000","last_became_peered":"2024-04-23T19:10:13.497693+0000","last_unstale":"2024-04-23T19:13:36.723412+0000","last_undegraded":"2024-04-23T19:13:36.723412+0000","last_fullsized":"2024-04-23T19:13:36.723412+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'39","last_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_deep_scrub":"32'39","last_deep_scrub_stamp":"2024-04-23T19:13:36.723239+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.723239+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:57:22.107325+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.719943+0000","last_change":"2024-04-23T19:13:39.719943+0000","last_active":"2024-04-23T19:13:39.719943+0000","last_peered":"2024-04-23T19:13:39.719943+0000","last_clean":"2024-04-23T19:13:39.719943+0000","last_became_active":"2024-04-23T19:10:15.503920+0000","last_became_peered":"2024-04-23T19:10:15.503920+0000","last_unstale":"2024-04-23T19:13:39.719943+0000","last_undegraded":"2024-04-23T19:13:39.719943+0000","last_fullsized":"2024-04-23T19:13:39.719943+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:39.719827+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:33.961424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"32'7","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:46.905294+0000","last_change":"2024-04-23T19:14:46.905294+0000","last_active":"2024-04-23T19:14:46.905294+0000","last_peered":"2024-04-23T19:14:46.905294+0000","last_clean":"2024-04-23T19:14:46.905294+0000","last_became_active":"2024-04-23T19:10:17.516836+0000","last_became_peered":"2024-04-23T19:10:17.516836+0000","last_unstale":"2024-04-23T19:14:46.905294+0000","last_undegraded":"2024-04-23T19:14:46.905294+0000","last_fullsized":"2024-04-23T19:14:46.905294+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'7","last_scrub_stamp":"2024-04-23T19:14:46.905160+0000","last_deep_scrub":"32'7","last_deep_scrub_stamp":"2024-04-23T19:14:46.905160+0000","last_clean_scrub_stamp":"2024-04-23T19:14:46.905160+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:39.044240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"32'14","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:57.526283+0000","last_change":"2024-04-23T19:13:57.526310+0000","last_active":"2024-04-23T19:13:57.526283+0000","last_peered":"2024-04-23T19:13:57.526283+0000","last_clean":"2024-04-23T19:13:57.526283+0000","last_became_active":"2024-04-23T19:10:09.477336+0000","last_became_peered":"2024-04-23T19:10:09.477336+0000","last_unstale":"2024-04-23T19:13:57.526283+0000","last_undegraded":"2024-04-23T19:13:57.526283+0000","last_fullsized":"2024-04-23T19:13:57.526283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:57.526187+0000","last_clean_scrub_stamp":"2024-04-23T19:13:57.526187+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:13.862482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":18,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:21.930715+0000","last_change":"2024-04-23T19:14:21.930715+0000","last_active":"2024-04-23T19:14:21.930715+0000","last_peered":"2024-04-23T19:14:21.930715+0000","last_clean":"2024-04-23T19:14:21.930715+0000","last_became_active":"2024-04-23T19:10:07.502138+0000","last_became_peered":"2024-04-23T19:10:07.502138+0000","last_unstale":"2024-04-23T19:14:21.930715+0000","last_undegraded":"2024-04-23T19:14:21.930715+0000","last_fullsized":"2024-04-23T19:14:21.930715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:21.930521+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:21.930521+0000","last_clean_scrub_stamp":"2024-04-23T19:14:21.930521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:51.962515+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"32'5","reported_seq":238,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:44.794993+0000","last_change":"2024-04-23T19:13:44.795028+0000","last_active":"2024-04-23T19:13:44.794993+0000","last_peered":"2024-04-23T19:13:44.794993+0000","last_clean":"2024-04-23T19:13:44.794993+0000","last_became_active":"2024-04-23T19:10:11.485491+0000","last_became_peered":"2024-04-23T19:10:11.485491+0000","last_unstale":"2024-04-23T19:13:44.794993+0000","last_undegraded":"2024-04-23T19:13:44.794993+0000","last_fullsized":"2024-04-23T19:13:44.794993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'5","last_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_deep_scrub":"32'5","last_deep_scrub_stamp":"2024-04-23T19:13:44.794905+0000","last_clean_scrub_stamp":"2024-04-23T19:13:44.794905+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:45.493403+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"32'15","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:56.880831+0000","last_change":"2024-04-23T19:13:56.880831+0000","last_active":"2024-04-23T19:13:56.880831+0000","last_peered":"2024-04-23T19:13:56.880831+0000","last_clean":"2024-04-23T19:13:56.880831+0000","last_became_active":"2024-04-23T19:10:13.501843+0000","last_became_peered":"2024-04-23T19:10:13.501843+0000","last_unstale":"2024-04-23T19:13:56.880831+0000","last_undegraded":"2024-04-23T19:13:56.880831+0000","last_fullsized":"2024-04-23T19:13:56.880831+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'15","last_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_deep_scrub":"32'15","last_deep_scrub_stamp":"2024-04-23T19:13:56.880679+0000","last_clean_scrub_stamp":"2024-04-23T19:13:56.880679+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:26.722280+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"32'285","reported_seq":430,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:29.137953+0000","last_change":"2024-04-23T19:13:29.137953+0000","last_active":"2024-04-23T19:13:29.137953+0000","last_peered":"2024-04-23T19:13:29.137953+0000","last_clean":"2024-04-23T19:13:29.137953+0000","last_became_active":"2024-04-23T19:10:15.510296+0000","last_became_peered":"2024-04-23T19:10:15.510296+0000","last_unstale":"2024-04-23T19:13:29.137953+0000","last_undegraded":"2024-04-23T19:13:29.137953+0000","last_fullsized":"2024-04-23T19:13:29.137953+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'285","last_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_deep_scrub":"32'285","last_deep_scrub_stamp":"2024-04-23T19:13:29.137819+0000","last_clean_scrub_stamp":"2024-04-23T19:13:29.137819+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:17.728927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.985410+0000","last_change":"2024-04-23T19:14:13.985410+0000","last_active":"2024-04-23T19:14:13.985410+0000","last_peered":"2024-04-23T19:14:13.985410+0000","last_clean":"2024-04-23T19:14:13.985410+0000","last_became_active":"2024-04-23T19:10:17.513488+0000","last_became_peered":"2024-04-23T19:10:17.513488+0000","last_unstale":"2024-04-23T19:14:13.985410+0000","last_undegraded":"2024-04-23T19:14:13.985410+0000","last_fullsized":"2024-04-23T19:14:13.985410+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:13.985289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:13.985289+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.985289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:36.480554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"32'2","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.032334+0000","last_change":"2024-04-23T19:13:53.032361+0000","last_active":"2024-04-23T19:13:53.032334+0000","last_peered":"2024-04-23T19:13:53.032334+0000","last_clean":"2024-04-23T19:13:53.032334+0000","last_became_active":"2024-04-23T19:10:09.476114+0000","last_became_peered":"2024-04-23T19:10:09.476114+0000","last_unstale":"2024-04-23T19:13:53.032334+0000","last_undegraded":"2024-04-23T19:13:53.032334+0000","last_fullsized":"2024-04-23T19:13:53.032334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:53.032245+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.032245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:08.378799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:19.781007+0000","last_change":"2024-04-23T19:14:19.781007+0000","last_active":"2024-04-23T19:14:19.781007+0000","last_peered":"2024-04-23T19:14:19.781007+0000","last_clean":"2024-04-23T19:14:19.781007+0000","last_became_active":"2024-04-23T19:10:07.497649+0000","last_became_peered":"2024-04-23T19:10:07.497649+0000","last_unstale":"2024-04-23T19:14:19.781007+0000","last_undegraded":"2024-04-23T19:14:19.781007+0000","last_fullsized":"2024-04-23T19:14:19.781007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:19.780892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:19.780892+0000","last_clean_scrub_stamp":"2024-04-23T19:14:19.780892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:49.622493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:41.890930+0000","last_change":"2024-04-23T19:13:41.890978+0000","last_active":"2024-04-23T19:13:41.890930+0000","last_peered":"2024-04-23T19:13:41.890930+0000","last_clean":"2024-04-23T19:13:41.890930+0000","last_became_active":"2024-04-23T19:10:11.489029+0000","last_became_peered":"2024-04-23T19:10:11.489029+0000","last_unstale":"2024-04-23T19:13:41.890930+0000","last_undegraded":"2024-04-23T19:13:41.890930+0000","last_fullsized":"2024-04-23T19:13:41.890930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:41.890830+0000","last_clean_scrub_stamp":"2024-04-23T19:13:41.890830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:42.009368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"32'46","reported_seq":105,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.806949+0000","last_change":"2024-04-23T19:13:31.806949+0000","last_active":"2024-04-23T19:13:31.806949+0000","last_peered":"2024-04-23T19:13:31.806949+0000","last_clean":"2024-04-23T19:13:31.806949+0000","last_became_active":"2024-04-23T19:10:13.491925+0000","last_became_peered":"2024-04-23T19:10:13.491925+0000","last_unstale":"2024-04-23T19:13:31.806949+0000","last_undegraded":"2024-04-23T19:13:31.806949+0000","last_fullsized":"2024-04-23T19:13:31.806949+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'46","last_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_deep_scrub":"32'46","last_deep_scrub_stamp":"2024-04-23T19:13:31.806771+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.806771+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:55.900270+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:31.655412+0000","last_change":"2024-04-23T19:13:31.655412+0000","last_active":"2024-04-23T19:13:31.655412+0000","last_peered":"2024-04-23T19:13:31.655412+0000","last_clean":"2024-04-23T19:13:31.655412+0000","last_became_active":"2024-04-23T19:10:15.501848+0000","last_became_peered":"2024-04-23T19:10:15.501848+0000","last_unstale":"2024-04-23T19:13:31.655412+0000","last_undegraded":"2024-04-23T19:13:31.655412+0000","last_fullsized":"2024-04-23T19:13:31.655412+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:31.655302+0000","last_clean_scrub_stamp":"2024-04-23T19:13:31.655302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:21:46.303379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:23.793885+0000","last_change":"2024-04-23T19:14:23.793885+0000","last_active":"2024-04-23T19:14:23.793885+0000","last_peered":"2024-04-23T19:14:23.793885+0000","last_clean":"2024-04-23T19:14:23.793885+0000","last_became_active":"2024-04-23T19:10:17.518037+0000","last_became_peered":"2024-04-23T19:10:17.518037+0000","last_unstale":"2024-04-23T19:14:23.793885+0000","last_undegraded":"2024-04-23T19:14:23.793885+0000","last_fullsized":"2024-04-23T19:14:23.793885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:23.793752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:23.793752+0000","last_clean_scrub_stamp":"2024-04-23T19:14:23.793752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:52:39.597437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'4","reported_seq":49,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.023793+0000","last_change":"2024-04-23T19:13:55.023820+0000","last_active":"2024-04-23T19:13:55.023793+0000","last_peered":"2024-04-23T19:13:55.023793+0000","last_clean":"2024-04-23T19:13:55.023793+0000","last_became_active":"2024-04-23T19:10:09.477332+0000","last_became_peered":"2024-04-23T19:10:09.477332+0000","last_unstale":"2024-04-23T19:13:55.023793+0000","last_undegraded":"2024-04-23T19:13:55.023793+0000","last_fullsized":"2024-04-23T19:13:55.023793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:55.023704+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.023704+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:02.984229+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.970941+0000","last_change":"2024-04-23T19:14:20.970941+0000","last_active":"2024-04-23T19:14:20.970941+0000","last_peered":"2024-04-23T19:14:20.970941+0000","last_clean":"2024-04-23T19:14:20.970941+0000","last_became_active":"2024-04-23T19:10:07.502486+0000","last_became_peered":"2024-04-23T19:10:07.502486+0000","last_unstale":"2024-04-23T19:14:20.970941+0000","last_undegraded":"2024-04-23T19:14:20.970941+0000","last_fullsized":"2024-04-23T19:14:20.970941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:20.970802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:20.970802+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.970802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:24:34.128767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:46.149582+0000","last_change":"2024-04-23T19:13:46.149624+0000","last_active":"2024-04-23T19:13:46.149582+0000","last_peered":"2024-04-23T19:13:46.149582+0000","last_clean":"2024-04-23T19:13:46.149582+0000","last_became_active":"2024-04-23T19:10:11.489650+0000","last_became_peered":"2024-04-23T19:10:11.489650+0000","last_unstale":"2024-04-23T19:13:46.149582+0000","last_undegraded":"2024-04-23T19:13:46.149582+0000","last_fullsized":"2024-04-23T19:13:46.149582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:46.149461+0000","last_clean_scrub_stamp":"2024-04-23T19:13:46.149461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:05:44.705262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"32'37","reported_seq":93,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:33.718094+0000","last_change":"2024-04-23T19:13:33.718094+0000","last_active":"2024-04-23T19:13:33.718094+0000","last_peered":"2024-04-23T19:13:33.718094+0000","last_clean":"2024-04-23T19:13:33.718094+0000","last_became_active":"2024-04-23T19:10:13.505472+0000","last_became_peered":"2024-04-23T19:10:13.505472+0000","last_unstale":"2024-04-23T19:13:33.718094+0000","last_undegraded":"2024-04-23T19:13:33.718094+0000","last_fullsized":"2024-04-23T19:13:33.718094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'37","last_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_deep_scrub":"32'37","last_deep_scrub_stamp":"2024-04-23T19:13:33.717974+0000","last_clean_scrub_stamp":"2024-04-23T19:13:33.717974+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:49:50.719128+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.950089+0000","last_change":"2024-04-23T19:14:01.950089+0000","last_active":"2024-04-23T19:14:01.950089+0000","last_peered":"2024-04-23T19:14:01.950089+0000","last_clean":"2024-04-23T19:14:01.950089+0000","last_became_active":"2024-04-23T19:10:15.514372+0000","last_became_peered":"2024-04-23T19:10:15.514372+0000","last_unstale":"2024-04-23T19:14:01.950089+0000","last_undegraded":"2024-04-23T19:14:01.950089+0000","last_fullsized":"2024-04-23T19:14:01.950089+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:01.949971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:01.949971+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.949971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:05.620169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"32'2","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:47.877559+0000","last_change":"2024-04-23T19:14:47.877559+0000","last_active":"2024-04-23T19:14:47.877559+0000","last_peered":"2024-04-23T19:14:47.877559+0000","last_clean":"2024-04-23T19:14:47.877559+0000","last_became_active":"2024-04-23T19:10:17.520762+0000","last_became_peered":"2024-04-23T19:10:17.520762+0000","last_unstale":"2024-04-23T19:14:47.877559+0000","last_undegraded":"2024-04-23T19:14:47.877559+0000","last_fullsized":"2024-04-23T19:14:47.877559+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:47.877428+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:47.877428+0000","last_clean_scrub_stamp":"2024-04-23T19:14:47.877428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:50.439618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"32'12","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.000667+0000","last_change":"2024-04-23T19:14:09.000667+0000","last_active":"2024-04-23T19:14:09.000667+0000","last_peered":"2024-04-23T19:14:09.000667+0000","last_clean":"2024-04-23T19:14:09.000667+0000","last_became_active":"2024-04-23T19:10:09.490218+0000","last_became_peered":"2024-04-23T19:10:09.490218+0000","last_unstale":"2024-04-23T19:14:09.000667+0000","last_undegraded":"2024-04-23T19:14:09.000667+0000","last_fullsized":"2024-04-23T19:14:09.000667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:09.000539+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:09.000539+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.000539+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:32.366969+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":51,"num_read_kb":47,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:18.545266+0000","last_change":"2024-04-23T19:14:18.545295+0000","last_active":"2024-04-23T19:14:18.545266+0000","last_peered":"2024-04-23T19:14:18.545266+0000","last_clean":"2024-04-23T19:14:18.545266+0000","last_became_active":"2024-04-23T19:10:07.486077+0000","last_became_peered":"2024-04-23T19:10:07.486077+0000","last_unstale":"2024-04-23T19:14:18.545266+0000","last_undegraded":"2024-04-23T19:14:18.545266+0000","last_fullsized":"2024-04-23T19:14:18.545266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:18.545177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:18.545177+0000","last_clean_scrub_stamp":"2024-04-23T19:14:18.545177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:58.201631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:36.929808+0000","last_change":"2024-04-23T19:13:36.929808+0000","last_active":"2024-04-23T19:13:36.929808+0000","last_peered":"2024-04-23T19:13:36.929808+0000","last_clean":"2024-04-23T19:13:36.929808+0000","last_became_active":"2024-04-23T19:10:11.490493+0000","last_became_peered":"2024-04-23T19:10:11.490493+0000","last_unstale":"2024-04-23T19:13:36.929808+0000","last_undegraded":"2024-04-23T19:13:36.929808+0000","last_fullsized":"2024-04-23T19:13:36.929808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:36.929690+0000","last_clean_scrub_stamp":"2024-04-23T19:13:36.929690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:45.019264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"32'30","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.925641+0000","last_change":"2024-04-23T19:13:52.925676+0000","last_active":"2024-04-23T19:13:52.925641+0000","last_peered":"2024-04-23T19:13:52.925641+0000","last_clean":"2024-04-23T19:13:52.925641+0000","last_became_active":"2024-04-23T19:10:13.501800+0000","last_became_peered":"2024-04-23T19:10:13.501800+0000","last_unstale":"2024-04-23T19:13:52.925641+0000","last_undegraded":"2024-04-23T19:13:52.925641+0000","last_fullsized":"2024-04-23T19:13:52.925641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'30","last_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_deep_scrub":"32'30","last_deep_scrub_stamp":"2024-04-23T19:13:52.925542+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.925542+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.397817+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.909679+0000","last_change":"2024-04-23T19:13:49.909700+0000","last_active":"2024-04-23T19:13:49.909679+0000","last_peered":"2024-04-23T19:13:49.909679+0000","last_clean":"2024-04-23T19:13:49.909679+0000","last_became_active":"2024-04-23T19:10:15.512557+0000","last_became_peered":"2024-04-23T19:10:15.512557+0000","last_unstale":"2024-04-23T19:13:49.909679+0000","last_undegraded":"2024-04-23T19:13:49.909679+0000","last_fullsized":"2024-04-23T19:13:49.909679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:49.909590+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.909590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:01:35.495993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"32'4","reported_seq":31,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:49.912678+0000","last_change":"2024-04-23T19:14:49.912678+0000","last_active":"2024-04-23T19:14:49.912678+0000","last_peered":"2024-04-23T19:14:49.912678+0000","last_clean":"2024-04-23T19:14:49.912678+0000","last_became_active":"2024-04-23T19:10:17.521149+0000","last_became_peered":"2024-04-23T19:10:17.521149+0000","last_unstale":"2024-04-23T19:14:49.912678+0000","last_undegraded":"2024-04-23T19:14:49.912678+0000","last_fullsized":"2024-04-23T19:14:49.912678+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:49.912496+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:49.912496+0000","last_clean_scrub_stamp":"2024-04-23T19:14:49.912496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:25.211152+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"32'4","reported_seq":56,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.048855+0000","last_change":"2024-04-23T19:13:52.048855+0000","last_active":"2024-04-23T19:13:52.048855+0000","last_peered":"2024-04-23T19:13:52.048855+0000","last_clean":"2024-04-23T19:13:52.048855+0000","last_became_active":"2024-04-23T19:10:09.475692+0000","last_became_peered":"2024-04-23T19:10:09.475692+0000","last_unstale":"2024-04-23T19:13:52.048855+0000","last_undegraded":"2024-04-23T19:13:52.048855+0000","last_fullsized":"2024-04-23T19:13:52.048855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.048718+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.048718+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:54:52.747216+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:09.880060+0000","last_change":"2024-04-23T19:14:09.880060+0000","last_active":"2024-04-23T19:14:09.880060+0000","last_peered":"2024-04-23T19:14:09.880060+0000","last_clean":"2024-04-23T19:14:09.880060+0000","last_became_active":"2024-04-23T19:10:07.500297+0000","last_became_peered":"2024-04-23T19:10:07.500297+0000","last_unstale":"2024-04-23T19:14:09.880060+0000","last_undegraded":"2024-04-23T19:14:09.880060+0000","last_fullsized":"2024-04-23T19:14:09.880060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:09.879942+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:09.879942+0000","last_clean_scrub_stamp":"2024-04-23T19:14:09.879942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:25:42.402393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"32'3","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:39.954265+0000","last_change":"2024-04-23T19:13:39.954265+0000","last_active":"2024-04-23T19:13:39.954265+0000","last_peered":"2024-04-23T19:13:39.954265+0000","last_clean":"2024-04-23T19:13:39.954265+0000","last_became_active":"2024-04-23T19:10:11.490411+0000","last_became_peered":"2024-04-23T19:10:11.490411+0000","last_unstale":"2024-04-23T19:13:39.954265+0000","last_undegraded":"2024-04-23T19:13:39.954265+0000","last_fullsized":"2024-04-23T19:13:39.954265+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:13:39.954105+0000","last_clean_scrub_stamp":"2024-04-23T19:13:39.954105+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:23.553074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"32'40","reported_seq":97,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:55.877187+0000","last_change":"2024-04-23T19:13:55.877248+0000","last_active":"2024-04-23T19:13:55.877187+0000","last_peered":"2024-04-23T19:13:55.877187+0000","last_clean":"2024-04-23T19:13:55.877187+0000","last_became_active":"2024-04-23T19:10:13.502026+0000","last_became_peered":"2024-04-23T19:10:13.502026+0000","last_unstale":"2024-04-23T19:13:55.877187+0000","last_undegraded":"2024-04-23T19:13:55.877187+0000","last_fullsized":"2024-04-23T19:13:55.877187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'40","last_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_deep_scrub":"32'40","last_deep_scrub_stamp":"2024-04-23T19:13:55.877078+0000","last_clean_scrub_stamp":"2024-04-23T19:13:55.877078+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:08.472673+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:27.549277+0000","last_change":"2024-04-23T19:13:27.549295+0000","last_active":"2024-04-23T19:13:27.549277+0000","last_peered":"2024-04-23T19:13:27.549277+0000","last_clean":"2024-04-23T19:13:27.549277+0000","last_became_active":"2024-04-23T19:10:15.501889+0000","last_became_peered":"2024-04-23T19:10:15.501889+0000","last_unstale":"2024-04-23T19:13:27.549277+0000","last_undegraded":"2024-04-23T19:13:27.549277+0000","last_fullsized":"2024-04-23T19:13:27.549277+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:13:27.549205+0000","last_clean_scrub_stamp":"2024-04-23T19:13:27.549205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:29:33.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"32'4","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:48.871105+0000","last_change":"2024-04-23T19:14:48.871105+0000","last_active":"2024-04-23T19:14:48.871105+0000","last_peered":"2024-04-23T19:14:48.871105+0000","last_clean":"2024-04-23T19:14:48.871105+0000","last_became_active":"2024-04-23T19:10:17.520963+0000","last_became_peered":"2024-04-23T19:10:17.520963+0000","last_unstale":"2024-04-23T19:14:48.871105+0000","last_undegraded":"2024-04-23T19:14:48.871105+0000","last_fullsized":"2024-04-23T19:14:48.871105+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:48.870967+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:48.870967+0000","last_clean_scrub_stamp":"2024-04-23T19:14:48.870967+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:19:47.452300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"32'14","reported_seq":78,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:49.719401+0000","last_change":"2024-04-23T19:13:49.719401+0000","last_active":"2024-04-23T19:13:49.719401+0000","last_peered":"2024-04-23T19:13:49.719401+0000","last_clean":"2024-04-23T19:13:49.719401+0000","last_became_active":"2024-04-23T19:10:09.493999+0000","last_became_peered":"2024-04-23T19:10:09.493999+0000","last_unstale":"2024-04-23T19:13:49.719401+0000","last_undegraded":"2024-04-23T19:13:49.719401+0000","last_fullsized":"2024-04-23T19:13:49.719401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:49.719206+0000","last_clean_scrub_stamp":"2024-04-23T19:13:49.719206+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:11:27.945878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"32'4","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.054867+0000","last_change":"2024-04-23T19:14:06.054867+0000","last_active":"2024-04-23T19:14:06.054867+0000","last_peered":"2024-04-23T19:14:06.054867+0000","last_clean":"2024-04-23T19:14:06.054867+0000","last_became_active":"2024-04-23T19:10:07.491440+0000","last_became_peered":"2024-04-23T19:10:07.491440+0000","last_unstale":"2024-04-23T19:14:06.054867+0000","last_undegraded":"2024-04-23T19:14:06.054867+0000","last_fullsized":"2024-04-23T19:14:06.054867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:06.054734+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:06.054734+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.054734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:34:22.391030+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"32'6","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.541792+0000","last_change":"2024-04-23T19:14:03.541792+0000","last_active":"2024-04-23T19:14:03.541792+0000","last_peered":"2024-04-23T19:14:03.541792+0000","last_clean":"2024-04-23T19:14:03.541792+0000","last_became_active":"2024-04-23T19:10:09.478447+0000","last_became_peered":"2024-04-23T19:10:09.478447+0000","last_unstale":"2024-04-23T19:14:03.541792+0000","last_undegraded":"2024-04-23T19:14:03.541792+0000","last_fullsized":"2024-04-23T19:14:03.541792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:03.541669+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.541669+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:10:29.952719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.533755+0000","last_change":"2024-04-23T19:14:20.533755+0000","last_active":"2024-04-23T19:14:20.533755+0000","last_peered":"2024-04-23T19:14:20.533755+0000","last_clean":"2024-04-23T19:14:20.533755+0000","last_became_active":"2024-04-23T19:10:07.488280+0000","last_became_peered":"2024-04-23T19:10:07.488280+0000","last_unstale":"2024-04-23T19:14:20.533755+0000","last_undegraded":"2024-04-23T19:14:20.533755+0000","last_fullsized":"2024-04-23T19:14:20.533755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:20.533629+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:20.533629+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.533629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:58.132753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.033589+0000","last_change":"2024-04-23T19:14:08.033589+0000","last_active":"2024-04-23T19:14:08.033589+0000","last_peered":"2024-04-23T19:14:08.033589+0000","last_clean":"2024-04-23T19:14:08.033589+0000","last_became_active":"2024-04-23T19:10:09.494421+0000","last_became_peered":"2024-04-23T19:10:09.494421+0000","last_unstale":"2024-04-23T19:14:08.033589+0000","last_undegraded":"2024-04-23T19:14:08.033589+0000","last_fullsized":"2024-04-23T19:14:08.033589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:08.033479+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:08.033479+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.033479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:30:11.119403+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:27.912078+0000","last_change":"2024-04-23T19:14:27.912102+0000","last_active":"2024-04-23T19:14:27.912078+0000","last_peered":"2024-04-23T19:14:27.912078+0000","last_clean":"2024-04-23T19:14:27.912078+0000","last_became_active":"2024-04-23T19:10:07.506368+0000","last_became_peered":"2024-04-23T19:10:07.506368+0000","last_unstale":"2024-04-23T19:14:27.912078+0000","last_undegraded":"2024-04-23T19:14:27.912078+0000","last_fullsized":"2024-04-23T19:14:27.912078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:27.911989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:27.911989+0000","last_clean_scrub_stamp":"2024-04-23T19:14:27.911989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:34:04.938123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:01.509140+0000","last_change":"2024-04-23T19:14:01.509140+0000","last_active":"2024-04-23T19:14:01.509140+0000","last_peered":"2024-04-23T19:14:01.509140+0000","last_clean":"2024-04-23T19:14:01.509140+0000","last_became_active":"2024-04-23T19:10:09.478394+0000","last_became_peered":"2024-04-23T19:10:09.478394+0000","last_unstale":"2024-04-23T19:14:01.509140+0000","last_undegraded":"2024-04-23T19:14:01.509140+0000","last_fullsized":"2024-04-23T19:14:01.509140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:01.508998+0000","last_clean_scrub_stamp":"2024-04-23T19:14:01.508998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:10.244783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.590245+0000","last_change":"2024-04-23T19:14:08.590245+0000","last_active":"2024-04-23T19:14:08.590245+0000","last_peered":"2024-04-23T19:14:08.590245+0000","last_clean":"2024-04-23T19:14:08.590245+0000","last_became_active":"2024-04-23T19:10:07.489331+0000","last_became_peered":"2024-04-23T19:10:07.489331+0000","last_unstale":"2024-04-23T19:14:08.590245+0000","last_undegraded":"2024-04-23T19:14:08.590245+0000","last_fullsized":"2024-04-23T19:14:08.590245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:08.590099+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:08.590099+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.590099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:56.456633+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"32'14","reported_seq":60,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:54.554231+0000","last_change":"2024-04-23T19:13:54.554231+0000","last_active":"2024-04-23T19:13:54.554231+0000","last_peered":"2024-04-23T19:13:54.554231+0000","last_clean":"2024-04-23T19:13:54.554231+0000","last_became_active":"2024-04-23T19:10:09.490920+0000","last_became_peered":"2024-04-23T19:10:09.490920+0000","last_unstale":"2024-04-23T19:13:54.554231+0000","last_undegraded":"2024-04-23T19:13:54.554231+0000","last_fullsized":"2024-04-23T19:13:54.554231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'14","last_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_deep_scrub":"32'14","last_deep_scrub_stamp":"2024-04-23T19:13:54.554115+0000","last_clean_scrub_stamp":"2024-04-23T19:13:54.554115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:38.772240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.518605+0000","last_change":"2024-04-23T19:14:00.518605+0000","last_active":"2024-04-23T19:14:00.518605+0000","last_peered":"2024-04-23T19:14:00.518605+0000","last_clean":"2024-04-23T19:14:00.518605+0000","last_became_active":"2024-04-23T19:10:07.489489+0000","last_became_peered":"2024-04-23T19:10:07.489489+0000","last_unstale":"2024-04-23T19:14:00.518605+0000","last_undegraded":"2024-04-23T19:14:00.518605+0000","last_fullsized":"2024-04-23T19:14:00.518605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:00.518487+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.518487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:31.027600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"32'8","reported_seq":65,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:45.766254+0000","last_change":"2024-04-23T19:13:45.766254+0000","last_active":"2024-04-23T19:13:45.766254+0000","last_peered":"2024-04-23T19:13:45.766254+0000","last_clean":"2024-04-23T19:13:45.766254+0000","last_became_active":"2024-04-23T19:10:09.494672+0000","last_became_peered":"2024-04-23T19:10:09.494672+0000","last_unstale":"2024-04-23T19:13:45.766254+0000","last_undegraded":"2024-04-23T19:13:45.766254+0000","last_fullsized":"2024-04-23T19:13:45.766254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:45.766137+0000","last_clean_scrub_stamp":"2024-04-23T19:13:45.766137+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:33.256585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"32'4","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:13.886328+0000","last_change":"2024-04-23T19:14:13.886328+0000","last_active":"2024-04-23T19:14:13.886328+0000","last_peered":"2024-04-23T19:14:13.886328+0000","last_clean":"2024-04-23T19:14:13.886328+0000","last_became_active":"2024-04-23T19:10:07.490411+0000","last_became_peered":"2024-04-23T19:10:07.490411+0000","last_unstale":"2024-04-23T19:14:13.886328+0000","last_undegraded":"2024-04-23T19:14:13.886328+0000","last_fullsized":"2024-04-23T19:14:13.886328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:14:13.886177+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:14:13.886177+0000","last_clean_scrub_stamp":"2024-04-23T19:14:13.886177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:34.657691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"32'8","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:07.558742+0000","last_change":"2024-04-23T19:14:07.558742+0000","last_active":"2024-04-23T19:14:07.558742+0000","last_peered":"2024-04-23T19:14:07.558742+0000","last_clean":"2024-04-23T19:14:07.558742+0000","last_became_active":"2024-04-23T19:10:09.482326+0000","last_became_peered":"2024-04-23T19:10:09.482326+0000","last_unstale":"2024-04-23T19:14:07.558742+0000","last_undegraded":"2024-04-23T19:14:07.558742+0000","last_fullsized":"2024-04-23T19:14:07.558742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:07.558617+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:07.558617+0000","last_clean_scrub_stamp":"2024-04-23T19:14:07.558617+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:08.555554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:58.079291+0000","last_change":"2024-04-23T19:13:58.079291+0000","last_active":"2024-04-23T19:13:58.079291+0000","last_peered":"2024-04-23T19:13:58.079291+0000","last_clean":"2024-04-23T19:13:58.079291+0000","last_became_active":"2024-04-23T19:10:07.499258+0000","last_became_peered":"2024-04-23T19:10:07.499258+0000","last_unstale":"2024-04-23T19:13:58.079291+0000","last_undegraded":"2024-04-23T19:13:58.079291+0000","last_fullsized":"2024-04-23T19:13:58.079291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:13:58.079130+0000","last_clean_scrub_stamp":"2024-04-23T19:13:58.079130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:48:35.830705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"32'10","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:08.883280+0000","last_change":"2024-04-23T19:14:08.883280+0000","last_active":"2024-04-23T19:14:08.883280+0000","last_peered":"2024-04-23T19:14:08.883280+0000","last_clean":"2024-04-23T19:14:08.883280+0000","last_became_active":"2024-04-23T19:10:09.484697+0000","last_became_peered":"2024-04-23T19:10:09.484697+0000","last_unstale":"2024-04-23T19:14:08.883280+0000","last_undegraded":"2024-04-23T19:14:08.883280+0000","last_fullsized":"2024-04-23T19:14:08.883280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'10","last_scrub_stamp":"2024-04-23T19:14:08.883162+0000","last_deep_scrub":"32'10","last_deep_scrub_stamp":"2024-04-23T19:14:08.883162+0000","last_clean_scrub_stamp":"2024-04-23T19:14:08.883162+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:27.108487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"32'3","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.554519+0000","last_change":"2024-04-23T19:14:02.554519+0000","last_active":"2024-04-23T19:14:02.554519+0000","last_peered":"2024-04-23T19:14:02.554519+0000","last_clean":"2024-04-23T19:14:02.554519+0000","last_became_active":"2024-04-23T19:10:07.490236+0000","last_became_peered":"2024-04-23T19:10:07.490236+0000","last_unstale":"2024-04-23T19:14:02.554519+0000","last_undegraded":"2024-04-23T19:14:02.554519+0000","last_fullsized":"2024-04-23T19:14:02.554519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-04-23T19:14:02.554376+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.554376+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:52.658050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"32'2","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:04.844128+0000","last_change":"2024-04-23T19:14:04.844128+0000","last_active":"2024-04-23T19:14:04.844128+0000","last_peered":"2024-04-23T19:14:04.844128+0000","last_clean":"2024-04-23T19:14:04.844128+0000","last_became_active":"2024-04-23T19:10:09.483989+0000","last_became_peered":"2024-04-23T19:10:09.483989+0000","last_unstale":"2024-04-23T19:14:04.844128+0000","last_undegraded":"2024-04-23T19:14:04.844128+0000","last_fullsized":"2024-04-23T19:14:04.844128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:04.844020+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:04.844020+0000","last_clean_scrub_stamp":"2024-04-23T19:14:04.844020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:21:48.661589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:29.951532+0000","last_change":"2024-04-23T19:14:29.951560+0000","last_active":"2024-04-23T19:14:29.951532+0000","last_peered":"2024-04-23T19:14:29.951532+0000","last_clean":"2024-04-23T19:14:29.951532+0000","last_became_active":"2024-04-23T19:10:07.504451+0000","last_became_peered":"2024-04-23T19:10:07.504451+0000","last_unstale":"2024-04-23T19:14:29.951532+0000","last_undegraded":"2024-04-23T19:14:29.951532+0000","last_fullsized":"2024-04-23T19:14:29.951532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:29.951431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:29.951431+0000","last_clean_scrub_stamp":"2024-04-23T19:14:29.951431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:51.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"32'6","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.936694+0000","last_change":"2024-04-23T19:14:00.936694+0000","last_active":"2024-04-23T19:14:00.936694+0000","last_peered":"2024-04-23T19:14:00.936694+0000","last_clean":"2024-04-23T19:14:00.936694+0000","last_became_active":"2024-04-23T19:10:09.498101+0000","last_became_peered":"2024-04-23T19:10:09.498101+0000","last_unstale":"2024-04-23T19:14:00.936694+0000","last_undegraded":"2024-04-23T19:14:00.936694+0000","last_fullsized":"2024-04-23T19:14:00.936694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:00.936576+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.936576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:19.351629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"32'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:12.563709+0000","last_change":"2024-04-23T19:14:12.563709+0000","last_active":"2024-04-23T19:14:12.563709+0000","last_peered":"2024-04-23T19:14:12.563709+0000","last_clean":"2024-04-23T19:14:12.563709+0000","last_became_active":"2024-04-23T19:10:07.488523+0000","last_became_peered":"2024-04-23T19:10:07.488523+0000","last_unstale":"2024-04-23T19:14:12.563709+0000","last_undegraded":"2024-04-23T19:14:12.563709+0000","last_fullsized":"2024-04-23T19:14:12.563709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-04-23T19:14:12.563575+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-04-23T19:14:12.563575+0000","last_clean_scrub_stamp":"2024-04-23T19:14:12.563575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:07:45.234515+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"32'6","reported_seq":52,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.538459+0000","last_change":"2024-04-23T19:13:52.538459+0000","last_active":"2024-04-23T19:13:52.538459+0000","last_peered":"2024-04-23T19:13:52.538459+0000","last_clean":"2024-04-23T19:13:52.538459+0000","last_became_active":"2024-04-23T19:10:09.491632+0000","last_became_peered":"2024-04-23T19:10:09.491632+0000","last_unstale":"2024-04-23T19:13:52.538459+0000","last_undegraded":"2024-04-23T19:13:52.538459+0000","last_fullsized":"2024-04-23T19:13:52.538459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:13:52.538343+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.538343+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:41:34.602802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:14.859491+0000","last_change":"2024-04-23T19:14:14.859491+0000","last_active":"2024-04-23T19:14:14.859491+0000","last_peered":"2024-04-23T19:14:14.859491+0000","last_clean":"2024-04-23T19:14:14.859491+0000","last_became_active":"2024-04-23T19:10:07.490435+0000","last_became_peered":"2024-04-23T19:10:07.490435+0000","last_unstale":"2024-04-23T19:14:14.859491+0000","last_undegraded":"2024-04-23T19:14:14.859491+0000","last_fullsized":"2024-04-23T19:14:14.859491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:14.859367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:14.859367+0000","last_clean_scrub_stamp":"2024-04-23T19:14:14.859367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:04.076971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"32'8","reported_seq":66,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:59.865265+0000","last_change":"2024-04-23T19:13:59.865265+0000","last_active":"2024-04-23T19:13:59.865265+0000","last_peered":"2024-04-23T19:13:59.865265+0000","last_clean":"2024-04-23T19:13:59.865265+0000","last_became_active":"2024-04-23T19:10:09.481661+0000","last_became_peered":"2024-04-23T19:10:09.481661+0000","last_unstale":"2024-04-23T19:13:59.865265+0000","last_undegraded":"2024-04-23T19:13:59.865265+0000","last_fullsized":"2024-04-23T19:13:59.865265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:13:59.865099+0000","last_clean_scrub_stamp":"2024-04-23T19:13:59.865099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:18:21.512041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:03.889509+0000","last_change":"2024-04-23T19:14:03.889509+0000","last_active":"2024-04-23T19:14:03.889509+0000","last_peered":"2024-04-23T19:14:03.889509+0000","last_clean":"2024-04-23T19:14:03.889509+0000","last_became_active":"2024-04-23T19:10:07.501040+0000","last_became_peered":"2024-04-23T19:10:07.501040+0000","last_unstale":"2024-04-23T19:14:03.889509+0000","last_undegraded":"2024-04-23T19:14:03.889509+0000","last_fullsized":"2024-04-23T19:14:03.889509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:03.889376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:03.889376+0000","last_clean_scrub_stamp":"2024-04-23T19:14:03.889376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.241474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"32'4","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:52.731942+0000","last_change":"2024-04-23T19:13:52.731942+0000","last_active":"2024-04-23T19:13:52.731942+0000","last_peered":"2024-04-23T19:13:52.731942+0000","last_clean":"2024-04-23T19:13:52.731942+0000","last_became_active":"2024-04-23T19:10:09.491904+0000","last_became_peered":"2024-04-23T19:10:09.491904+0000","last_unstale":"2024-04-23T19:13:52.731942+0000","last_undegraded":"2024-04-23T19:13:52.731942+0000","last_fullsized":"2024-04-23T19:13:52.731942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:52.731777+0000","last_clean_scrub_stamp":"2024-04-23T19:13:52.731777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:59:37.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:41.850777+0000","last_change":"2024-04-23T19:14:41.850851+0000","last_active":"2024-04-23T19:14:41.850777+0000","last_peered":"2024-04-23T19:14:41.850777+0000","last_clean":"2024-04-23T19:14:41.850777+0000","last_became_active":"2024-04-23T19:10:07.488320+0000","last_became_peered":"2024-04-23T19:10:07.488320+0000","last_unstale":"2024-04-23T19:14:41.850777+0000","last_undegraded":"2024-04-23T19:14:41.850777+0000","last_fullsized":"2024-04-23T19:14:41.850777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:41.850671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:41.850671+0000","last_clean_scrub_stamp":"2024-04-23T19:14:41.850671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:10.257673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:02.863186+0000","last_change":"2024-04-23T19:14:02.863186+0000","last_active":"2024-04-23T19:14:02.863186+0000","last_peered":"2024-04-23T19:14:02.863186+0000","last_clean":"2024-04-23T19:14:02.863186+0000","last_became_active":"2024-04-23T19:10:09.482430+0000","last_became_peered":"2024-04-23T19:10:09.482430+0000","last_unstale":"2024-04-23T19:14:02.863186+0000","last_undegraded":"2024-04-23T19:14:02.863186+0000","last_fullsized":"2024-04-23T19:14:02.863186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:02.863017+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:02.863017+0000","last_clean_scrub_stamp":"2024-04-23T19:14:02.863017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:09:33.372130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:22.951020+0000","last_change":"2024-04-23T19:14:22.951020+0000","last_active":"2024-04-23T19:14:22.951020+0000","last_peered":"2024-04-23T19:14:22.951020+0000","last_clean":"2024-04-23T19:14:22.951020+0000","last_became_active":"2024-04-23T19:10:07.502029+0000","last_became_peered":"2024-04-23T19:10:07.502029+0000","last_unstale":"2024-04-23T19:14:22.951020+0000","last_undegraded":"2024-04-23T19:14:22.951020+0000","last_fullsized":"2024-04-23T19:14:22.951020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:22.950894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:22.950894+0000","last_clean_scrub_stamp":"2024-04-23T19:14:22.950894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:42.201782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"32'6","reported_seq":61,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:06.983645+0000","last_change":"2024-04-23T19:14:06.983645+0000","last_active":"2024-04-23T19:14:06.983645+0000","last_peered":"2024-04-23T19:14:06.983645+0000","last_clean":"2024-04-23T19:14:06.983645+0000","last_became_active":"2024-04-23T19:10:09.495143+0000","last_became_peered":"2024-04-23T19:10:09.495143+0000","last_unstale":"2024-04-23T19:14:06.983645+0000","last_undegraded":"2024-04-23T19:14:06.983645+0000","last_fullsized":"2024-04-23T19:14:06.983645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'6","last_scrub_stamp":"2024-04-23T19:14:06.983514+0000","last_deep_scrub":"32'6","last_deep_scrub_stamp":"2024-04-23T19:14:06.983514+0000","last_clean_scrub_stamp":"2024-04-23T19:14:06.983514+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:29.820347+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"30'2","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:00.057782+0000","last_change":"2024-04-23T19:14:00.057782+0000","last_active":"2024-04-23T19:14:00.057782+0000","last_peered":"2024-04-23T19:14:00.057782+0000","last_clean":"2024-04-23T19:14:00.057782+0000","last_became_active":"2024-04-23T19:10:07.499734+0000","last_became_peered":"2024-04-23T19:10:07.499734+0000","last_unstale":"2024-04-23T19:14:00.057782+0000","last_undegraded":"2024-04-23T19:14:00.057782+0000","last_fullsized":"2024-04-23T19:14:00.057782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2","last_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_deep_scrub":"30'2","last_deep_scrub_stamp":"2024-04-23T19:14:00.057618+0000","last_clean_scrub_stamp":"2024-04-23T19:14:00.057618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:34.694433+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"32'12","reported_seq":64,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:20.779346+0000","last_change":"2024-04-23T19:14:20.779346+0000","last_active":"2024-04-23T19:14:20.779346+0000","last_peered":"2024-04-23T19:14:20.779346+0000","last_clean":"2024-04-23T19:14:20.779346+0000","last_became_active":"2024-04-23T19:10:09.489282+0000","last_became_peered":"2024-04-23T19:10:09.489282+0000","last_unstale":"2024-04-23T19:14:20.779346+0000","last_undegraded":"2024-04-23T19:14:20.779346+0000","last_fullsized":"2024-04-23T19:14:20.779346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'12","last_scrub_stamp":"2024-04-23T19:14:20.779226+0000","last_deep_scrub":"32'12","last_deep_scrub_stamp":"2024-04-23T19:14:20.779226+0000","last_clean_scrub_stamp":"2024-04-23T19:14:20.779226+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:00.435680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:43.914958+0000","last_change":"2024-04-23T19:14:43.914958+0000","last_active":"2024-04-23T19:14:43.914958+0000","last_peered":"2024-04-23T19:14:43.914958+0000","last_clean":"2024-04-23T19:14:43.914958+0000","last_became_active":"2024-04-23T19:10:07.507081+0000","last_became_peered":"2024-04-23T19:10:07.507081+0000","last_unstale":"2024-04-23T19:14:43.914958+0000","last_undegraded":"2024-04-23T19:14:43.914958+0000","last_fullsized":"2024-04-23T19:14:43.914958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:43.914835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:43.914835+0000","last_clean_scrub_stamp":"2024-04-23T19:14:43.914835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:46:43.212848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:24.879645+0000","last_change":"2024-04-23T19:14:24.879645+0000","last_active":"2024-04-23T19:14:24.879645+0000","last_peered":"2024-04-23T19:14:24.879645+0000","last_clean":"2024-04-23T19:14:24.879645+0000","last_became_active":"2024-04-23T19:10:07.502753+0000","last_became_peered":"2024-04-23T19:10:07.502753+0000","last_unstale":"2024-04-23T19:14:24.879645+0000","last_undegraded":"2024-04-23T19:14:24.879645+0000","last_fullsized":"2024-04-23T19:14:24.879645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T19:14:24.879513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T19:14:24.879513+0000","last_clean_scrub_stamp":"2024-04-23T19:14:24.879513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:13:57.197889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"32'8","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:11.585491+0000","last_change":"2024-04-23T19:14:11.585491+0000","last_active":"2024-04-23T19:14:11.585491+0000","last_peered":"2024-04-23T19:14:11.585491+0000","last_clean":"2024-04-23T19:14:11.585491+0000","last_became_active":"2024-04-23T19:10:09.486471+0000","last_became_peered":"2024-04-23T19:10:09.486471+0000","last_unstale":"2024-04-23T19:14:11.585491+0000","last_undegraded":"2024-04-23T19:14:11.585491+0000","last_fullsized":"2024-04-23T19:14:11.585491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:11.585343+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:11.585343+0000","last_clean_scrub_stamp":"2024-04-23T19:14:11.585343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:48:00.802947+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"32'8","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:14:17.561430+0000","last_change":"2024-04-23T19:14:17.561430+0000","last_active":"2024-04-23T19:14:17.561430+0000","last_peered":"2024-04-23T19:14:17.561430+0000","last_clean":"2024-04-23T19:14:17.561430+0000","last_became_active":"2024-04-23T19:10:07.486181+0000","last_became_peered":"2024-04-23T19:10:07.486181+0000","last_unstale":"2024-04-23T19:14:17.561430+0000","last_undegraded":"2024-04-23T19:14:17.561430+0000","last_fullsized":"2024-04-23T19:14:17.561430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-04-23T19:14:17.561302+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-04-23T19:14:17.561302+0000","last_clean_scrub_stamp":"2024-04-23T19:14:17.561302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:47.402903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13,"num_write":40,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"32'4","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T19:13:53.753760+0000","last_change":"2024-04-23T19:13:53.753760+0000","last_active":"2024-04-23T19:13:53.753760+0000","last_peered":"2024-04-23T19:13:53.753760+0000","last_clean":"2024-04-23T19:13:53.753760+0000","last_became_active":"2024-04-23T19:10:09.491218+0000","last_became_peered":"2024-04-23T19:10:09.491218+0000","last_unstale":"2024-04-23T19:13:53.753760+0000","last_undegraded":"2024-04-23T19:13:53.753760+0000","last_fullsized":"2024-04-23T19:13:53.753760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-04-23T19:13:53.753615+0000","last_clean_scrub_stamp":"2024-04-23T19:13:53.753615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:57:13.659941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1524,"ondisk_log_size":1524,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1604,"num_read_kb":1494,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":481,"ondisk_log_size":481,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673028,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39948,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331892,"statfs":{"total":96636764160,"available":96595857408,"internally_reserved":0,"allocated":462848,"data_stored":144617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673027,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1024000,"data_stored":734097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":139,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40468,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331372,"statfs":{"total":96636764160,"available":96595324928,"internally_reserved":0,"allocated":995328,"data_stored":732924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705731,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":425984,"data_stored":144396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T19:15:09.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.776+0000 7f1479d11640 1 Processor -- start 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.776+0000 7f1479d11640 1 -- start start 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f1479d11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1474094380 0x7f147412bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f1479d11640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f147406d320 con 0x7f147412c050 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f1479d11640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f147412b8e0 con 0x7f1474094380 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f14737fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1474094380 0x7f147412bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f1472ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f147412c050 0x7f1474095d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37028/0 (socket says 172.21.15.16:37028) 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f1472ffd640 1 -- 172.21.15.16:0/2205279684 learned_addr learned my addr 172.21.15.16:0/2205279684 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-23T19:15:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f14727fc640 1 -- 172.21.15.16:0/2205279684 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3896515701 0 0) 0x7f147406d320 con 0x7f147412c050 2024-04-23T19:15:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f14727fc640 1 -- 172.21.15.16:0/2205279684 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f145c003520 con 0x7f147412c050 2024-04-23T19:15:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f14737fe640 1 -- 172.21.15.16:0/2205279684 >> v1:172.21.15.16:6789/0 conn(0x7f147412c050 legacy=0x7f1474095d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.777+0000 7f14737fe640 1 -- 172.21.15.16:0/2205279684 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147406d7b0 con 0x7f1474094380 2024-04-23T19:15:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.778+0000 7f14737fe640 1 --2- 172.21.15.16:0/2205279684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f147412bb10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1464009720 tx=0x7f146402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08167e3bd953b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.778+0000 7f14727fc640 1 -- 172.21.15.16:0/2205279684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1464035020 con 0x7f1474094380 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.778+0000 7f14727fc640 1 -- 172.21.15.16:0/2205279684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1464031760 con 0x7f1474094380 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.778+0000 7f14727fc640 1 -- 172.21.15.16:0/2205279684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1464031a40 con 0x7f1474094380 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 -- 172.21.15.16:0/2205279684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 msgr2=0x7f147412bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 --2- 172.21.15.16:0/2205279684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f147412bb10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1464009720 tx=0x7f146402b8d0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 -- 172.21.15.16:0/2205279684 shutdown_connections 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 --2- 172.21.15.16:0/2205279684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f147412bb10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 -- 172.21.15.16:0/2205279684 >> 172.21.15.16:0/2205279684 conn(0x7f147409ad10 msgr2=0x7f147409b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 -- 172.21.15.16:0/2205279684 shutdown_connections 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 -- 172.21.15.16:0/2205279684 wait complete. 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.779+0000 7f1479d11640 1 Processor -- start 2024-04-23T19:15:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.780+0000 7f1479d11640 1 -- start start 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.780+0000 7f1479d11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f1474124140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.780+0000 7f1479d11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f147412c810 con 0x7f1474094380 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.780+0000 7f14737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f1474124140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.780+0000 7f14737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f1474124140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39386/0 (socket says 172.21.15.16:39386) 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.780+0000 7f14737fe640 1 -- 172.21.15.16:0/3442261327 learned_addr learned my addr 172.21.15.16:0/3442261327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.780+0000 7f14737fe640 1 -- 172.21.15.16:0/3442261327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740725b0 con 0x7f1474094380 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.781+0000 7f14737fe640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f1474124140 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1464034040 tx=0x7f14640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.781+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1464003820 con 0x7f1474094380 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.781+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14640039c0 con 0x7f1474094380 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.781+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1474127460 con 0x7f1474094380 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.781+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1464003cc0 con 0x7f1474094380 2024-04-23T19:15:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.781+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1474124680 con 0x7f1474094380 2024-04-23T19:15:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.782+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (secure 0 0 0) 0x7f1464038020 con 0x7f1474094380 2024-04-23T19:15:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.782+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1474063660 con 0x7f1474094380 2024-04-23T19:15:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.782+0000 7f145bfff640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f14600385c0 0x7f146003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:15:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.782+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 6313+0+0 (secure 0 0 0) 0x7f146406e5b0 con 0x7f1474094380 2024-04-23T19:15:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.782+0000 7f1472ffd640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f14600385c0 0x7f146003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:15:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.783+0000 7f1472ffd640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f14600385c0 0x7f146003aa80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1468007fd0 tx=0x7f14680010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:15:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.786+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f146403adf0 con 0x7f1474094380 2024-04-23T19:15:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.970+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f1474067f20 con 0x7f1474094380 2024-04-23T19:15:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.972+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f1464030dd0 con 0x7f1474094380 2024-04-23T19:15:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.973+0000 7f145bfff640 1 -- 172.21.15.16:0/3442261327 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1464030380 con 0x7f1474094380 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f14600385c0 msgr2=0x7f146003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f14600385c0 0x7f146003aa80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1468007fd0 tx=0x7f14680010a0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 msgr2=0x7f1474124140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f1474124140 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1464034040 tx=0x7f14640046c0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 shutdown_connections 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.63:6824/4252129218,v1:172.21.15.63:6825/4252129218] conn(0x7f14600385c0 0x7f146003aa80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 --2- 172.21.15.16:0/3442261327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1474094380 0x7f1474124140 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 >> 172.21.15.16:0/3442261327 conn(0x7f147409ad10 msgr2=0x7f1474094bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 shutdown_connections 2024-04-23T19:15:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-23T19:15:09.974+0000 7f1479d11640 1 -- 172.21.15.16:0/3442261327 wait complete. 2024-04-23T19:15:09.986 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-23T19:15:09.986 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-23T19:15:09.986 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-23T19:15:09.986 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T19:15:10.018 INFO:tasks.ceph.osd.0:Stopped 2024-04-23T19:15:10.018 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-23T19:15:10.019 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T19:15:10.058 INFO:tasks.ceph.osd.1:Stopped 2024-04-23T19:15:10.059 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-23T19:15:10.059 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T19:15:10.138 INFO:tasks.ceph.osd.2:Stopped 2024-04-23T19:15:10.139 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-23T19:15:10.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T19:15:10.206 INFO:tasks.ceph.osd.3:Stopped 2024-04-23T19:15:10.206 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-23T19:15:10.206 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-23T19:15:10.207 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T19:15:10.277 INFO:tasks.ceph.mgr.x:Stopped 2024-04-23T19:15:10.277 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-23T19:15:10.277 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-23T19:15:10.277 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T19:15:10.289 INFO:tasks.ceph.mon.a:Stopped 2024-04-23T19:15:10.289 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-23T19:15:10.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-04-23T19:15:10.367 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-23T19:15:10.368 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-23T19:15:10.503 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:15:10.503 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-23T19:15:10.596 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:15:10.597 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-23T19:15:10.686 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:15:10.686 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-23T19:15:10.742 INFO:tasks.ceph:Archiving mon data... 2024-04-23T19:15:10.743 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/data/mon.a.tgz 2024-04-23T19:15:10.743 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-04-23T19:15:10.761 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.Q3Oh0LiL29 2024-04-23T19:15:10.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Q3Oh0LiL29 2024-04-23T19:15:10.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.Q3Oh0LiL29 2024-04-23T19:15:10.980 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.Q3Oh0LiL29 is 270KB 2024-04-23T19:15:11.076 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.Q3Oh0LiL29 2024-04-23T19:15:11.091 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-23T19:15:11.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T19:15:11.133 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T19:15:11.212 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T19:15:11.230 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T19:15:11.230 INFO:tasks.ceph:Archiving crash dumps... 2024-04-23T19:15:11.231 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/remote/smithi016/crash 2024-04-23T19:15:11.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T19:15:11.276 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/crash to /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/remote/smithi063/crash 2024-04-23T19:15:11.277 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T19:15:11.313 INFO:tasks.ceph:Compressing logs... 2024-04-23T19:15:11.313 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T19:15:11.320 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T19:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29877.log 2024-04-23T19:15:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28844.log 2024-04-23T19:15:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29113.log 2024-04-23T19:15:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28844.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T19:15:11.358 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29877.log.gz 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.28844.log.gz 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30171.log 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29113.log.gz 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27762.log 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30385.log 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.27762.log: gzip 85.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.30171.log.gz --verbose 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.30087.log 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.27762.log.gz 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29834.log 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr: 85.6%/var/log/ceph/ceph-client.admin.30087.log: -- replaced with /var/log/ceph/ceph-client.admin.30385.log.gz 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27802.log 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T19:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29834.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30087.log.gz 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.29834.log.gz 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.27802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29559.log 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.27802.log.gz 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28970.log 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28788.log 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28970.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.29559.log.gz 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30467.log 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.28970.log.gz 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29601.log 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30467.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28788.log.gz 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29283.log 2024-04-23T19:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.30467.log.gz 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29013.log 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29453.log 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.29601.log.gz 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.29283.log.gz 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29740.log 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29013.log.gz 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29697.log 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29740.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29453.log.gz 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28731.log 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.29740.log.gz 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29112.log 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.29697.log.gz 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29919.log 2024-04-23T19:15:11.361 INFO:teuthology.orchestra.run.smithi016.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.28731.log.gz 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28886.log 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29112.log.gz 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29919.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.29919.log.gz 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30129.log 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.28886.log.gz 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29785.log 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30045.log 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29785.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.30129.log.gz 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29391.log 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.29785.log.gz 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30045.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29391.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.30045.log.gz 2024-04-23T19:15:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29431.log 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29391.log.gz 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28928.log 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29281.log 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28928.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29431.log.gz 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25953.log 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.28928.log.gz 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30299.log 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29281.log: /var/log/ceph/ceph.tmp-client.admin.25953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30342.log 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29281.log.gz 0.0% 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.25953.log.gz 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27644.log 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30299.log.gz 2024-04-23T19:15:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30003.log 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30342.log.gz 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.27644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29430.log 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.27644.log.gz 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29643.log 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30003.log: /var/log/ceph/ceph-client.admin.29430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29223.log 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.30003.log.gz 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.29430.log.gz 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29115.log 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29643.log.gz 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29114.log 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29223.log: /var/log/ceph/ceph-client.admin.29115.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29223.log.gz 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30213.log 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29115.log.gz 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30427.log/var/log/ceph/ceph-client.admin.29114.log: 2024-04-23T19:15:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29961.log 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29114.log.gz 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30213.log.gz 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29282.log 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.30427.log.gz 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30257.log 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29282.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29961.log.gz 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29282.log.gz 2024-04-23T19:15:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.30257.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30257.log.gz 2024-04-23T19:15:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60471.log 2024-04-23T19:15:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68336.log 2024-04-23T19:15:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.60471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79226.log 2024-04-23T19:15:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.68336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73781.log 2024-04-23T19:15:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.79226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35672.log 2024-04-23T19:15:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48371.log 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.73781.log: /var/log/ceph/ceph-client.0.35672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88271.log 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.60471.log.gz 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.48371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36277.log 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.79226.log.gz 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.88271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93111.log 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.36277.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.73781.log.gz 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69546.log 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.0.35672.log.gz 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.93111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82856.log 2024-04-23T19:15:11.389 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.48371.log.gz 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.69546.log: 93.3%gzip -- replaced with /var/log/ceph/ceph-client.0.68336.log.gz 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.75596.log 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.88271.log.gz 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.82856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55026.log 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.75596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84665.log 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.36277.log.gz 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.55026.log: gzip 91.0% -- replaced with /var/log/ceph/ceph-client.0.93111.log.gz 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.38697.log 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.84665.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.69546.log.gz 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90691.log 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.75596.log.gz 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.38697.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-23T19:15:11.390 INFO:teuthology.orchestra.run.smithi063.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.84665.log.gz 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76806.log 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.90691.log: /var/log/ceph/ceph-osd.2.log: 90.3% -- replaced with /var/log/ceph/ceph-client.0.38697.log.gz 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70151.log 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.82856.log.gz 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.55026.log.gz 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.76806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49581.log 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.70151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54421.log 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.90691.log.gz 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.49581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64706.log 2024-04-23T19:15:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.54421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78621.log 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.64706.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.76806.log.gz 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62891.log 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.70151.log.gz 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.78621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46556.log 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.49581.log.gz 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.62891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90086.log 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.54421.log.gz 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.46556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83461.log 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.90086.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.64706.log.gz 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85863.log 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.83461.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.62891.log.gz 2024-04-23T19:15:11.411 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50791.log 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.78621.log.gz 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.85863.log: gzip -5 --verbose 91.0% -- /var/log/ceph/ceph-client.0.51396.log -- replaced with /var/log/ceph/ceph-client.0.46556.log.gz 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.50791.log: 90.3%gzip -- replaced with /var/log/ceph/ceph-client.0.85863.log.gz -5 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr: --verbose -- /var/log/ceph/ceph-client.0.74386.log 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41111.log 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.51396.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.83461.log.gz 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.90086.log.gz 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.74386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29983.log 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.41111.log: gzip -5 91.1% -- replaced with /var/log/ceph/ceph-client.0.50791.log.gz 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr: --verbose -- /var/log/ceph/ceph-client.0.62286.log 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.29983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61076.log 2024-04-23T19:15:11.412 INFO:teuthology.orchestra.run.smithi063.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.29983.log.gz 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.51396.log.gz 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.62286.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.74386.log.gz 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95583.log 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.61076.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.41111.log.gz 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53211.log 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.95583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86462.log 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.53211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44136.log 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.62286.log.gz/var/log/ceph/ceph-client.0.86462.log: 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- 91.2% -- replaced with /var/log/ceph/ceph-client.0.61076.log.gz 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr: /var/log/ceph/ceph-client.0.48976.log 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.44136.log: 90.3% -- replaced with /var/log/ceph/ceph-client.0.86462.log.gz 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56236.log 2024-04-23T19:15:11.413 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.53211.log.gz 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.48976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71966.log 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.56236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87061.log 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.71966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76201.log 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.44136.log.gz 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.87061.log: gzip -5 --verbose 93.3% -- replaced with /var/log/ceph/ceph-client.0.95583.log.gz 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.48976.log.gz 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr: -- /var/log/ceph/ceph-client.0.74991.log 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.76201.log: gzip -5 --verbose -- 91.3% -- replaced with /var/log/ceph/ceph-client.0.56236.log.gz 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr: /var/log/ceph/ops-log-ceph-client.0.log 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.74991.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.71966.log.gz 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53816.log 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ops-log-ceph-client.0.log: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30063.log 2024-04-23T19:15:11.414 INFO:teuthology.orchestra.run.smithi063.stderr: -- replaced with /var/log/ceph/ceph-client.0.87061.log.gz 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.76201.log.gz 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.53816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56841.log 91.9% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.74991.log.gz 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.30063.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30063.log.gz 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.56841.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52606.log 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58656.log 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.0.53816.log.gz 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.52606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91296.log 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.58656.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.56841.log.gz 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58051.log 2024-04-23T19:15:11.415 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52001.log 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.58051.log: /var/log/ceph/ceph-client.0.91296.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.52606.log.gz 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.58656.log.gz 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.58051.log.gz 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.91296.log.gz 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34839.log 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.52001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47766.log 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.34839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47161.log 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.47766.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.34839.log.gz 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.47161.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.52001.log.gz 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57446.log 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mgr.x.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.47766.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.84066.log 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-04-23T19:15:11.416 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92506.log 2024-04-23T19:15:11.417 INFO:teuthology.orchestra.run.smithi063.stderr: -- replaced with /var/log/ceph/ceph-client.0.47161.log.gz 2024-04-23T19:15:11.417 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.57446.log: /var/log/ceph/ceph-client.0.84066.log: 90.2% -- replaced with /var/log/ceph/ceph-client.0.84066.log.gz 2024-04-23T19:15:11.417 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91901.log 2024-04-23T19:15:11.417 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.92506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59866.log 2024-04-23T19:15:11.417 INFO:teuthology.orchestra.run.smithi063.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.57446.log.gz 2024-04-23T19:15:11.417 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.91901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89481.log 2024-04-23T19:15:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.59866.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.92506.log.gz 2024-04-23T19:15:11.431 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.91901.log.gz 2024-04-23T19:15:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94926.log 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.89481.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.59866.log.gz 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40506.log 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64101.log 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.94926.log: /var/log/ceph/ceph-client.0.40506.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.89481.log.gz 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65916.log 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.64101.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.94926.log.gz 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.40506.log.gz 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45951.log 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80436.log 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.65916.log: /var/log/ceph/ceph-client.0.45951.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.64101.log.gz 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94321.log 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.80436.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.45951.log.gz 2024-04-23T19:15:11.432 INFO:teuthology.orchestra.run.smithi063.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.65916.log.gz 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81646.log 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37487.log 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.94321.log: /var/log/ceph/ceph-client.0.81646.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.80436.log.gz 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42321.log 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.81646.log.gz 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.37487.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.94321.log.gz 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67126.log 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34799.log 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.67126.log: /var/log/ceph/ceph-client.0.42321.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.37487.log.gz 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93716.log 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.34799.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.67126.log.gz 85.7% -- replaced with /var/log/ceph/ceph-client.admin.34799.log.gz 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.42321.log.gz 2024-04-23T19:15:11.433 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82251.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.93716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78016.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.82251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73176.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.78016.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.93716.log.gz 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59261.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.82251.log.gz 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.73176.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.78016.log.gz 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50186.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87666.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.59261.log: /var/log/ceph/ceph-client.0.50186.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.73176.log.gz 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39901.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.87666.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.50186.log.gz 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.59261.log.gz 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39296.log 2024-04-23T19:15:11.434 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85264.log 2024-04-23T19:15:11.435 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.87666.log.gz 2024-04-23T19:15:11.435 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.39901.log: /var/log/ceph/ceph-client.0.39296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65311.log 2024-04-23T19:15:11.435 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.85264.log: 90.2% -- replaced with /var/log/ceph/ceph-client.0.85264.log.gz 2024-04-23T19:15:11.435 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81041.log 2024-04-23T19:15:11.435 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.39901.log.gz 2024-04-23T19:15:11.438 INFO:teuthology.orchestra.run.smithi063.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.39296.log.gz 2024-04-23T19:15:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.65311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63496.log 2024-04-23T19:15:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.81041.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.65311.log.gz 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44741.log 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.63496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55631.log 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.44741.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.81041.log.gz 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38092.log 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0%/var/log/ceph/ceph-client.0.55631.log: -- replaced with /var/log/ceph/ceph-client.0.63496.log.gz 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.44741.log.gz 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71361.log 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.38092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68941.log 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.71361.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.55631.log.gz 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61681.log 2024-04-23T19:15:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.68941.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.38092.log.gz 2024-04-23T19:15:11.445 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72571.log 2024-04-23T19:15:11.445 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.61681.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.71361.log.gz 2024-04-23T19:15:11.445 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66521.log 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.72571.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.61681.log.gz 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36882.log 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.66521.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.68941.log.gz 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79831.log 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.72571.log.gz 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.36882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43531.log 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.79831.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.66521.log.gz 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45346.log 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.43531.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.79831.log.gz 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.36882.log.gz 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34879.log 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42926.log 2024-04-23T19:15:11.446 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.45346.log: /var/log/ceph/ceph-client.admin.34879.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.43531.log.gz 2024-04-23T19:15:11.447 INFO:teuthology.orchestra.run.smithi063.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.34879.log.gz 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.45346.log.gz 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34759.log 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.42926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30023.log 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.34759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88876.log 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.34759.log.gz 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.30023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67731.log 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.42926.log.gz 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.88876.log: 84.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29943.log 2024-04-23T19:15:11.453 INFO:teuthology.orchestra.run.smithi063.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30023.log.gz 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.67731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34718.log 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.29943.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.29943.log.gz 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 91.2% --verbose -- /var/log/ceph/ceph-client.0.41716.log 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr: -- replaced with /var/log/ceph/ceph-client.0.88876.log.gz 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.34718.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.34718.log.gz 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70756.log 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.41716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77411.log 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.67731.log.gz 2024-04-23T19:15:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.70756.log: /var/log/ceph/ceph-client.0.77411.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.41716.log.gz 2024-04-23T19:15:11.455 INFO:teuthology.orchestra.run.smithi063.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.70756.log.gz 2024-04-23T19:15:11.455 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.77411.log.gz 2024-04-23T19:15:11.455 INFO:teuthology.orchestra.run.smithi063.stderr: 92.1% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-04-23T19:15:11.471 INFO:teuthology.orchestra.run.smithi063.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-23T19:15:11.529 INFO:teuthology.orchestra.run.smithi016.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-23T19:15:12.274 INFO:teuthology.orchestra.run.smithi063.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-23T19:15:12.373 INFO:teuthology.orchestra.run.smithi016.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-23T19:15:12.375 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-23T19:15:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m1.043s 2024-04-23T19:15:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m1.195s 2024-04-23T19:15:12.375 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.070s 2024-04-23T19:15:12.425 INFO:teuthology.orchestra.run.smithi063.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-23T19:15:12.452 INFO:teuthology.orchestra.run.smithi063.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-23T19:15:12.454 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-04-23T19:15:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:real 0m1.083s 2024-04-23T19:15:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:user 0m3.132s 2024-04-23T19:15:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m0.212s 2024-04-23T19:15:12.455 INFO:tasks.ceph:Archiving logs... 2024-04-23T19:15:12.455 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/remote/smithi016/log 2024-04-23T19:15:12.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T19:15:12.680 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/remote/smithi063/log 2024-04-23T19:15:12.681 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T19:15:13.218 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T19:15:13.229 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T19:15:13.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T19:15:13.232 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T19:15:13.301 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T19:15:13.312 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T19:15:13.312 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T19:15:13.315 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T19:15:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-23T19:15:13.330 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-04-23T19:15:13.334 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T19:15:13.334 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-04-23T19:15:13.334 INFO:teuthology.orchestra.run.smithi063.stdout:^- hv01.front.sepia.ceph.com 2 6 377 54 +130us[ +157us] +/- 596ms 2024-04-23T19:15:13.334 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -290us[ -263us] +/- 59ms 2024-04-23T19:15:13.334 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +444us[ +444us] +/- 87ms 2024-04-23T19:15:13.334 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:15:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T19:15:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-23T19:15:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:^- hv01.front.sepia.ceph.com 2 6 377 52 +124us[ +124us] +/- 596ms 2024-04-23T19:15:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -301us[ -303us] +/- 59ms 2024-04-23T19:15:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +457us[ +455us] +/- 87ms 2024-04-23T19:15:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:15:13.337 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T19:15:13.345 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T19:15:13.345 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T19:15:13.386 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T19:15:13.452 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T19:15:13.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T19:15:13.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:15:13.541 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T19:15:13.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:15:13.568 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T19:15:13.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T19:15:13.612 INFO:teuthology.task.internal:Duration was 810.728837 seconds 2024-04-23T19:15:13.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T19:15:13.662 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T19:15:13.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T19:15:13.665 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T19:15:13.703 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T19:15:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T19:15:14.003 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T19:15:14.003 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-04-23T19:15:14.004 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T19:15:14.068 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-04-23T19:15:14.068 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T19:15:14.092 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T19:15:14.092 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T19:15:14.111 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T19:15:14.158 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T19:15:14.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T19:15:14.180 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T19:15:14.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T19:15:14.305 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T19:15:14.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T19:15:14.330 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T19:15:14.355 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T19:15:14.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:15:14.372 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:15:14.393 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-04-23T19:15:14.417 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-04-23T19:15:14.436 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:15:14.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:15:14.482 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:15:14.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:15:14.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T19:15:14.507 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T19:15:14.508 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/remote/smithi016 2024-04-23T19:15:14.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T19:15:14.558 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/7669982/remote/smithi063 2024-04-23T19:15:14.559 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T19:15:14.591 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T19:15:14.592 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T19:15:14.594 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T19:15:14.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T19:15:14.655 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T19:15:14.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T19:15:14.688 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T19:15:14.688 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T19:15:14.690 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T19:15:14.703 INFO:teuthology.orchestra.run.smithi016.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 19:15 /home/ubuntu/cephtest 2024-04-23T19:15:14.705 INFO:teuthology.orchestra.run.smithi063.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 19:15 /home/ubuntu/cephtest 2024-04-23T19:15:14.706 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T19:15:14.757 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T19:15:14.857 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} duration: 810.7288374900818 flavor: default owner: scheduled_suriarte@teuthology success: true 2024-04-23T19:15:14.857 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T19:15:14.930 INFO:teuthology.run:pass